Modeling the Temporal Extent of Actions

Satkin, Scott; Hebert, Martial

doi:10.1007/978-3-642-15549-9_39

Cited by 99 publications

(74 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In activity detection from 2D videos, much previous work has focussed on short video clips, assuming that temporal segmentation has been done apriori. It has been observed that temporal boundaries of actions are not precisely defined in practice, whether they are obtained automatically using weak-supervision [35] or by hand [36]. These works represent the action clips by an orderless bag-offeatures and try to improve classification of the action clips by refining their temporal boundaries.…”

Section: Related Workmentioning

confidence: 99%

Anticipating Human Activities using Object Affordances for Reactive Robotic Response

Koppula

Saxena

2013

Robotics: Science and Systems IX

159

214

View full text Add to dashboard Cite

Abstract-An important aspect of human perception is anticipation, which we use extensively in our day-to-day activities when interacting with other humans as well as with our surroundings. Anticipating which activities will a human do next (and how) can enable an assistive robot to plan ahead for reactive responses. Furthermore, anticipation can even improve the detection accuracy of past activities. The challenge, however, is two-fold: We need to capture the rich context for modeling the activities and object affordances, and we need to anticipate the distribution over a large space of future human activities. In this work, we represent each possible future using an anticipatory temporal conditional random field (ATCRF) that models the rich spatial-temporal relations through object affordances. We then consider each ATCRF as a particle and represent the distribution over the potential futures using a set of particles. In extensive evaluation on CAD-120 human activity RGB-D dataset, we first show that anticipation improves the state-ofthe-art detection results. We then show that for new subjects (not seen in the training set), we obtain an activity anticipation accuracy (defined as whether one of top three predictions actually happened) of 84.1%, 74.4% and 62.2% for an anticipation time of 1, 3 and 10 seconds respectively. Finally, we also show a robot using our algorithm for performing a few reactive responses. †

show abstract

Section: Related Workmentioning

confidence: 99%

Anticipating Human Activities using Object Affordances for Reactive Robotic Response

Koppula

Saxena

2013

Robotics: Science and Systems IX

159

214

View full text Add to dashboard Cite

show abstract

“…For the video shown in the figure it would simply be "clean and jerk" (Figure 1b). But this level of description does not address issues such as the temporal extent of the action [27]. It typically uses only a global featurebased representation to predict the class of action.…”

Section: Introductionmentioning

confidence: 99%

Representing Videos Using Mid-level Discriminative Patches

Jain¹,

Gupta²,

Rodríguez³

et al. 2013

2013 IEEE Conference on Computer Vision and Pattern Recognition

120

View full text Add to dashboard Cite

How should a video be represented? We propose a new representation for videos based on mid-level discriminative spatio-temporal patches. These spatio-temporal patches might correspond to a primitive human action, a semantic object, or perhaps a random but informative spatiotemporal patch in the video. What defines these spatiotemporal patches is their discriminative and representative properties. We automatically mine these patches from hundreds of training videos and experimentally demonstrate that these patches establish correspondence across videos and align the videos for label transfer techniques. Furthermore, these patches can be used as a discriminative vocabulary for action classification where they demonstrate stateof-the-art performance on UCF50 and Olympics datasets.

show abstract

“…This method was found useful when the videos are captured in complex scenes (e.g., supermarket). Similar to the idea of sliding window search, Satkin and Hebert [120] located the best segment in a video for action training by exhaustively checking all possible segments of the video. Fig.…”

Section: Spatio-temporal Localizationmentioning

confidence: 99%

High-level event recognition in unconstrained videos

Jiang

Bhattacharya

Chang

et al. 2012

Int J Multimed Info Retr

160

108

View full text Add to dashboard Cite

The goal of high-level event recognition is to automatically detect complex high-level events in a given video sequence. This is a difficult task especially when videos are captured under unconstrained conditions by nonprofessionals. Such videos depicting complex events have limited quality control, and therefore, may include severe camera motion, poor lighting, heavy background clutter, and occlusion. However, due to the fast growing popularity of such videos, especially on the Web, solutions to this problem are in high demands and have attracted great interest from researchers. In this paper, we review current technologies for complex event recognition in unconstrained videos. While the existing solutions vary, we identify common key modules and provide detailed descriptions along with some insights for each of them, including extraction and representation of low-level features across different modalities, classification strategies, fusion techniques, etc. Publicly available benchmark datasets, performance metrics, and related research forums are also described. Finally, we discuss promising directions for future research.

show abstract

Modeling the Temporal Extent of Actions

Cited by 99 publications

References 18 publications

Anticipating Human Activities using Object Affordances for Reactive Robotic Response

Anticipating Human Activities using Object Affordances for Reactive Robotic Response

Representing Videos Using Mid-level Discriminative Patches

High-level event recognition in unconstrained videos

Contact Info

Product

Resources

About