Online Real-time Multiple Spatiotemporal Action Localisation and Prediction

Singh, Gurkirt; Saha, Suman Kumar; Sapienza, Michael; Han, Junwei; Cuzzolin, Fabio

doi:10.48550/arxiv.1611.08563

Cited by 4 publications

(16 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Almost all recent works [22,26,29,35] for action localization build on CNN object detectors [18,24]. In the following, we review recent CNN object detectors and then state-of-the-art action localization approaches.…”

Section: Related Workmentioning

confidence: 99%

“…They also use multiple regions inside each action proposal and then link the detections across a video based on spatial overlap and classification score. Singh et al [29] perform action localization in real-time using (a) the efficient SSD detector, (b) a fast method [13] to estimate the optical flow for the motion stream, and (c) an online linking algorithm. All these approaches rely on detections at the frame level.…”

Section: Related Workmentioning

confidence: 99%

“…For combining the two streams at test time we compare two approaches: union fusion and late fusion. For the union fusion [29], we consider the set union of the outputs from both streams: the tubelets from the RGB stream with their associated scores and the tubelets from the flow stream with their scores. For the late fusion [6], we average the scores from both streams for each anchor cuboid, as the set of anchors is the same for both streams.…”

Section: Two-stream Act-detectormentioning

confidence: 99%

“…Recently, Convolutional Neural Networks (CNNs) have proven well adapted for action localization, as they provide robust representations of video frames. Indeed, most state-of-the-art action localization approaches [9,22,26,29,35] are based on CNN object detectors [18,24] that detect human actions at the 1 Univ. Grenoble Alpes, Inria, CNRS, Grenoble INP, LJK, 38000 Grenoble, France.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Action Tubelet Detector for Spatio-Temporal Action Localization

Kalogeiton

Weinzaepfel

Ferrari

et al. 2017

2017 IEEE International Conference on Computer Vision (ICCV)

287

373

View full text Add to dashboard Cite

Current state-of-the-art approaches for spatio-temporal action localization rely on detections at the frame level that are then linked or tracked across time. In this paper, we leverage the temporal continuity of videos instead of operating at the frame level. We propose the ACtion Tubelet detector (ACT-detector) that takes as input a sequence of frames and outputs tubelets, i.e., sequences of bounding boxes with associated scores. The same way state-of-theart object detectors rely on anchor boxes, our ACT-detector is based on anchor cuboids. We build upon the SSD framework [18]. Convolutional features are extracted for each frame, while scores and regressions are based on the temporal stacking of these features, thus exploiting information from a sequence. Our experimental results show that leveraging sequences of frames significantly improves detection performance over using individual frames. The gain of our tubelet detector can be explained by both more accurate scores and more precise localization. Our ACT-detector outperforms the state-of-the-art methods for frame-mAP and video-mAP on the J-HMDB [12] and UCF-101 [30] datasets, in particular at high overlap thresholds.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Two-stream Act-detectormentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Action Tubelet Detector for Spatio-Temporal Action Localization

Kalogeiton

Weinzaepfel

Ferrari

et al. 2017

2017 IEEE International Conference on Computer Vision (ICCV)

287

373

View full text Add to dashboard Cite

show abstract

“…Current DNNs algorithms require huge datasets which are mostly recorded by humans, who however select points of view which may completely differ from those adopted by robots. While the performance of these bottom-up recognition methods is continuously improved through new architectures [7] and datasets [8], the high dependency of human activities on multiple contextual factors and actors' mental states suggests that the size of the datasets necessary to achieve a high enough precision for predictive physical interaction would be prohibitive. Furthermore, this approach requires substantial training time and reconfiguration or retraining when new tasks are added.…”

Section: Introductionmentioning

confidence: 99%

Transferring Adaptive Theory of Mind to Social Robots: Insights from Developmental Psychology to Robotics

Bianco

Ognibene

2019

Social Robotics

View full text Add to dashboard Cite

Despite the recent advancement in the social robotic field, important limitations restrain its progress and delay the application of robots in everyday scenarios. In the present paper, we propose to develop computational models inspired by our knowledge of human infants' social adaptive abilities. We believe this may provide solutions at an architectural level to overcome the limits of current systems. Specifically, we present the functional advantages that adaptive Theory of Mind (ToM) systems would support in robotics (i.e., mentalizing for belief understanding, proactivity and preparation, active perception and learning) and contextualize them in practical applications. We review current computational models mainly based on the simulation and teleological theories, and robotic implementations to identify the limitations of ToM functions in current robotic architectures and suggest a possible future developmental pathway. Finally, we propose future studies to create innovative computational models integrating the properties of the simulation and teleological approaches for an improved adaptive ToM ability in robots with the aim of enhancing human-robot interactions and permitting the application of robots in unexplored environments, such as disasters and construction sites. To achieve this goal, we suggest directing future research towards the modern cross-talk between the fields of robotics and developmental psychology.

show abstract

Okutama-Action: An Aerial View Video Dataset for Concurrent Human Action Detection

Barekatain

Martí

Shih

et al. 2017

2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)

149

123

View full text Add to dashboard Cite

Despite significant progress in the development of human action detection datasets and algorithms, no current dataset is representative of real-world aerial view scenarios. We present Okutama-Action, a new video dataset for aerial view concurrent human action detection. It consists of 43 minute-long fully-annotated sequences with 12 action classes. Okutama-Action features many challenges missing in current datasets, including dynamic transition of actions, significant changes in scale and aspect ratio, abrupt camera movement, as well as multi-labeled actors. As a result, our dataset is more challenging than existing ones, and will help push the field forward to enable real-world applications.

show abstract

Online Real-time Multiple Spatiotemporal Action Localisation and Prediction

Cited by 4 publications

References 0 publications

Action Tubelet Detector for Spatio-Temporal Action Localization

Action Tubelet Detector for Spatio-Temporal Action Localization

Transferring Adaptive Theory of Mind to Social Robots: Insights from Developmental Psychology to Robotics

Okutama-Action: An Aerial View Video Dataset for Concurrent Human Action Detection

Contact Info

Product

Resources

About