Sign Up

2461 SW Campus Way, Corvallis, OR 97331

View map Free Event

Weakly Supervised Learning for Action Segmentation

In this research report, we address weakly supervised learning for action segmentation. Weakly supervised learning means framewise ground truth labels are not available in training. In particular, there are two types of weakly supervised learning: (1) Transcript-level supervised learning, where ground truth is a transcript that represents the temporal order sequence of actions present in a training video; (2) Set-level supervised learning, where ground truth specifies only a set of actions present in a training video. We address both problem setups by proposing an energy-based learning framework. For each video, we construct a temporal segmentation graph to represent its all possible segmentation patterns, where a path on the graph represents a segmentation pattern for the video, nodes are cuts and edges are video segments between every two temporally ordered cuts. For transcript-level supervised learning, we have developed constrained discriminative forward! loss (CDFL) that efficiently maximizes the distance energies between multiple valid paths that satisfy temporal ordering ground truth and invalid paths that violate temporal ordering ground truth. For set-level supervised learning, we have developed set-constrained Viterbi (SCV) that constructs the temporal segmentation graph by efficiently solving the NP-hard all color problem.

Major Advisor: Sinisa Todorovic
Committee: Fuxin Li
Committee: Alan Fern
Committee: Yue Zhang
GCR: Jimmy Yang

  • Guangyu Zhang

1 person is interested in this event

User Activity

No recent activity