Deep Sequential Context Networks for Action Prediction

Kong, Yu; Tao, Zhiqiang; Fu, Yun

doi:10.1109/cvpr.2017.390

Cited by 136 publications

(132 citation statements)

References 25 publications

Supporting

Mentioning

129

Contrasting

Order By: Relevance

“…This implies that our proposed soft regression framework is also beneficial for the task of action recognition and can obtain the state-of-the-art recognition result. As expected, our soft regression model outperformed the RankLSTM [36] and DeepSCN [25] approaches again, which demonstrates the efficacy of our soft label learning framework for early action prediction. We also note that the prediction results of most methods on the first 10% frames on this set is much lower than that on the ORGBD and SYSU 3D HOI sets.…”

Section: Results On the Ntu Large Scale Datasetmentioning

confidence: 51%

“…It also worked better than the MSSVM model, which predicts ongoing activities with known progress level using multiple pre-trained predictors. We also observe that our soft regression model performed better than DeepSCN [25] R e d u n d a n t f r a me s Ac t i o n f r a me s Fig. 6.…”

Section: Results On Online Rgb-d Action Datasetsmentioning

confidence: 78%

“…We also compared our soft regression model with the DeepSCN [25] and RankLSTM [36] models. The source codes for DeepSCN and RankLSTM are not available for benchmarking, we re-implemented them strictly by following the descriptions in [25], [36]. For a fair comparison, we fed our LAFF features into their learning frameworks and reported the best results among a large range parameter settings.…”

Section: Compared Methodsmentioning

confidence: 99%

“…For evaluation, we followed the same experimental settings as in [24], [25]. And we used the first 15 groups of videos for training, the next 3 groups for validation, and the rest for testing (note that those groups were pre-partitioned in [25]). It is worth noting that the body parts of many actions are only partially observable (e.g., action "apply eye makeup" and "apply lipstick").…”

Section: Soft Regression For Predicting Unconstrained Rgb Actionsmentioning

confidence: 99%

“…To directly compare to other early action prediction models [2], [3], [23], [24], [25], [40] on RGB videos only, we tested our model on the UCF101 set [48], which contains 13320 unconstrained RGB videos from 101 action classes. For evaluation, we followed the same experimental settings as in [24], [25]. And we used the first 15 groups of videos for training, the next 3 groups for validation, and the rest for testing (note that those groups were pre-partitioned in [25]).…”

Section: Soft Regression For Predicting Unconstrained Rgb Actionsmentioning

confidence: 99%

See 4 more Smart Citations

Early Action Prediction by Soft Regression

Zheng

et al. 2019

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

Abstract-We propose a novel approach for predicting on-going action with the assistance of a low-cost depth camera. Our approach introduces a soft regression-based early prediction framework. In this framework, we estimate soft labels for the subsequences at different progress levels, jointly learned with an action predictor. Our formulation of soft regression framework 1) overcomes a usual assumption in existing early action prediction systems that the progress level of on-going sequence is given in the testing stage; and 2) presents a theoretical framework to better resolve the ambiguity and uncertainty of subsequences at early performing stage. The proposed soft regression framework is further enhanced in order to take the relationships among subsequences and the discrepancy of soft labels over different classes into consideration, so that a Multiple Soft labels Recurrent Neural Network (MSRNN) is finally developed. For real-time performance, we also introduce a new RGB-D feature called "local accumulative frame feature (LAFF)", which can be computed efficiently by constructing an integral feature map. Our experiments on three RGB-D benchmark datasets and an unconstrained RGB action set demonstrate that the proposed regression-based early action prediction model outperforms existing models significantly and also show that the early action prediction on RGB-D sequence is more accurate than that on RGB channel.

show abstract

Section: Results On the Ntu Large Scale Datasetmentioning

confidence: 51%

Section: Results On Online Rgb-d Action Datasetsmentioning

confidence: 78%