Completeness Modeling and Context Separation for Weakly Supervised Temporal Action Localization

Liu, Daochang; Jiang, Tingting; Wang, Yizhou

doi:10.1109/cvpr.2019.00139

Cited by 225 publications

(184 citation statements)

References 39 publications

Supporting

Mentioning

184

Contrasting

Order By: Relevance

“…As demonstrated on two challenging untrimmed video datasets, PreTrimNet achieves superior performance over the state-of-the-art weakly supervised methods, and is even comparable to some fully-supervised methods that leverage temporal annotations during training. In the future, inspired by the most recent advances in action recognition and localization, we will consider improving our framework by explicitly modeling backgrounds (Liu, Jiang, and Wang 2019), and further leveraging external sources (Nguyen, Ramanan, and Fowlkes 2019).…”

Section: Resultsmentioning

confidence: 99%

Multi-Instance Multi-Label Action Recognition and Localization Based on Spatio-Temporal Pre-Trimming for Untrimmed Videos

Zhang

Shi

2020

AAAI

View full text Add to dashboard Cite

Weakly supervised action recognition and localization for untrimmed videos is a challenging problem with extensive applications. The overwhelming irrelevant background contents in untrimmed videos severely hamper effective identification of actions of interest. In this paper, we propose a novel multi-instance multi-label modeling network based on spatio-temporal pre-trimming to recognize actions and locate corresponding frames in untrimmed videos. Motivated by the fact that person is the key factor in a human action, we spatially and temporally segment each untrimmed video into person-centric clips with pose estimation and tracking techniques. Given the bag-of-instances structure associated with video-level labels, action recognition is naturally formulated as a multi-instance multi-label learning problem. The network is optimized iteratively with selective coarse-to-fine pre-trimming based on instance-label activation. After convergence, temporal localization is further achieved with local-global temporal class activation map. Extensive experiments are conducted on two benchmark datasets, i.e. THUMOS14 and ActivityNet1.3, and experimental results clearly corroborate the efficacy of our method when compared with the state-of-the-arts.

show abstract

Section: Resultsmentioning

confidence: 99%

Multi-Instance Multi-Label Action Recognition and Localization Based on Spatio-Temporal Pre-Trimming for Untrimmed Videos

Zhang

Shi

2020

AAAI

View full text Add to dashboard Cite

show abstract

“…The performance is better, but it costs more time and computation. Afterwards, CMCS [35] proposes a multi-branch network architecture with diversity loss for action completeness modeling. At the same time, they propose a scheme generating a hard negative video for separating contexts.…”

Section: ) Current Representative Methodsmentioning

confidence: 99%

A Survey on Temporal Action Localization

Xia

Zhan

2020

IEEE Access

View full text Add to dashboard Cite

Temporal action localization is one of the most crucial and challenging problems for video understanding in computer vision. It has received a lot of attention in recent years because of the extensive application of daily life. Temporal action localization has made some significant progress, especially with the development of deep learning recently. And more demand is for temporal action localization in untrimmed videos. In this paper, our target is to survey the state-of-the-art techniques and models for video temporal action localization. It mainly includes the related techniques, some benchmark datasets and the evaluation metrics of temporal action localization. In addition, we summarize temporal action localization from two aspects: fully-supervised learning and weakly-supervised learning. And we list several representative works and compare their performances respectively. Finally, we make some deep analysis and propose potential research directions, and conclude the survey.

show abstract

“…Further, Shou et al [28] propose a novel Outer-Inner-Contrastive loss to discover the segment-level supervision for action boundary prediction. To keep the completeness of actions, Liu et al [20] employ a multi-branch framework where branches are enforced to discover distinctive parts of actions. And Yu et al [35] explore the temporal action structure and model each action as a multi-phase process.…”

Section: Related Work 21 Temporal Action Localizationmentioning

confidence: 99%

“…Network regularization is widely-used in weakly-supervised tasks [8,20], which injects extra limitations (i.e. prior knowledge) into the network to stabilize the training process and improve the model performance.…”

Section: Introductionmentioning

confidence: 99%

Regularized Two-Branch Proposal Networks for Weakly-Supervised Moment Retrieval in Videos

Zhang

Lin

Zhao

et al. 2020

Proceedings of the 28th ACM International Conference on Multimedia

View full text Add to dashboard Cite

Video moment retrieval aims to localize the target moment in an video according to the given sentence. The weak-supervised setting only provides the video-level sentence annotations during training. Most existing weak-supervised methods apply a MIL-based framework to develop inter-sample confrontment, but ignore the intra-sample confrontment between moments with semantically similar contents. Thus, these methods fail to distinguish the target moment from plausible negative moments. In this paper, we propose a novel Regularized Two-Branch Proposal Network to simultaneously consider the inter-sample and intra-sample confrontments. Concretely, we first devise a language-aware filter to generate an enhanced video stream and a suppressed video stream. We then design the sharable two-branch proposal module to generate positive proposals from the enhanced stream and plausible negative proposals from the suppressed one for sufficient confrontment. Further, we apply the proposal regularization to stabilize the training process and improve model performance. The extensive experiments show the effectiveness of our method. Our code is released at here 1. CCS CONCEPTS • Information systems → Video search; • Computing methodologies → Activity recognition and understanding.

show abstract

Completeness Modeling and Context Separation for Weakly Supervised Temporal Action Localization

Cited by 225 publications

References 39 publications

Multi-Instance Multi-Label Action Recognition and Localization Based on Spatio-Temporal Pre-Trimming for Untrimmed Videos

Multi-Instance Multi-Label Action Recognition and Localization Based on Spatio-Temporal Pre-Trimming for Untrimmed Videos

A Survey on Temporal Action Localization

Regularized Two-Branch Proposal Networks for Weakly-Supervised Moment Retrieval in Videos

Contact Info

Product

Resources

About