BMN: Boundary-Matching Network for Temporal Action Proposal Generation

Lin, Tianwei; Liu, Xiao; Li, Xin; Ding, Errui; Wen, Shifeng

doi:10.48550/arxiv.1907.09702

Cited by 8 publications

(2 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Temporal Action Localization. Temporal action localization in full supervision has gained significant developments in recent years [15,11,16,2,3,6]. However, obtaining precise temporal boundaries (start and end time) is very timeconsuming in long untrimmed videos.…”

Section: Related Workmentioning

confidence: 99%

“…There are many works [1,2,3,4,5,6] in recent years to tackle temporal action localization which aims to localize and classify actions in untrimmed videos. These methods are introduced with full supervision setting: annotations of temporal boundaries (start time and end time) and action category labels are provided in the training procedure as shown in Fig.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Weakly Supervised Temporal Action Localization with Segment-Level Labels

Ding¹,

Wang²,

Gao³

et al. 2020

Preprint

View full text Add to dashboard Cite

Temporal action localization presents a trade-off between test performance and annotation-time cost. Fully supervised methods achieve good performance with time-consuming boundary annotations. Weakly supervised methods with cheaper video-level category label annotations result in worse performance. In this paper, we introduce a new segment-level supervision setting: segments are labeled when annotators observe actions happening here. We incorporate this segment-level supervision along with a novel localization module in the training. Specifically, we devise a partial segment loss regarded as a loss sampling to learn integral action parts from labeled segments. Since the labeled segments are only parts of actions, the model tends to overfit along with the training process. To tackle this problem, we first obtain a similarity matrix from discriminative features guided by a sphere loss. Then, a propagation loss is devised based on the matrix to act as a regularization term, allowing implicit unlabeled segments propagation during training. Experiments validate that our method can outperform the video-level supervision methods with almost same the annotation time.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Weakly Supervised Temporal Action Localization with Segment-Level Labels

Ding¹,

Wang²,

Gao³

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

Temporal Action Proposal Generation via Multi-Task Feature Learning

Duan

2020

2020 IEEE International Conference on Visual Communications and Image Processing (VCIP)

View full text Add to dashboard Cite

BSN++: Complementary Boundary Regressor with Scale-Balanced Relation Modeling for Temporal Action Proposal Generation

Su¹,

Gan²,

Wu³

et al. 2020

Preprint

View full text Add to dashboard Cite

Generating human action proposals in untrimmed videos is an important yet challenging task with wide applications. Current methods often suffer from the noisy boundary locations and the inferior quality of confidence scores used for proposal retrieving. In this paper, we present BSN++, a new framework which exploits complementary boundary regressor and relation modeling for temporal proposal generation. First, we propose a novel boundary regressor based on the complementary characteristics of both starting and ending boundary classifiers. Specifically, we utilize the Ushaped architecture with nested skip connections to capture rich contexts and introduce bi-directional boundary matching mechanism to improve boundary precision. Second, to account for the proposal-proposal relations ignored in previous methods, we devise a proposal relation block to which includes two self-attention modules from the aspects of position and channel. Furthermore, we find that there inevitably exists data imbalanced problems in the positive/negative proposals and temporal durations, which harm the model performance on tail distributions. To relieve this issue, we introduce the scale-balanced re-sampling strategy. Extensive experiments are conducted on two popular benchmarks: ActivityNet-1.3 and THUMOS14, which demonstrate that BSN++ achieves the state-of-the-art performance.

show abstract

BMN: Boundary-Matching Network for Temporal Action Proposal Generation

Cited by 8 publications

References 28 publications

Weakly Supervised Temporal Action Localization with Segment-Level Labels

Weakly Supervised Temporal Action Localization with Segment-Level Labels

Temporal Action Proposal Generation via Multi-Task Feature Learning

BSN++: Complementary Boundary Regressor with Scale-Balanced Relation Modeling for Temporal Action Proposal Generation

Contact Info

Product

Resources

About