Progressive Localization Networks for Language-based Moment Localization

Zheng, Qi; Dong, Jianfeng; Qu, Xiaoye; Yang, Xun; Ji, Shouling; Wang, Xun

doi:10.48550/arxiv.2102.01282

Cited by 1 publication

(1 citation statement)

References 45 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In addition, 2D-TAN [32] enumerate all possible segments as proposal candidates and convert them into 2D feature map, then a temporal adjacent network is proposed to obtain multi-modal representation and encode the video context information. Following this, [19,20,35] design more complicated cross-modal reasoning strategies to learn the video-language semantic alignment from both coarse and finegrained granularities.…”

Section: Short-form Video Temporal Groundingmentioning

confidence: 99%

Scanning Only Once: An End-to-end Framework for Fast Temporal Grounding in Long Videos

Pan¹,

He²,

Gong³

et al. 2023

Preprint

View full text Add to dashboard Cite

Video temporal grounding aims to pinpoint a video segment that matches the query description. Despite the recent advance in short-form videos (e.g., in minutes), temporal grounding in long videos (e.g., in hours) is still at its early stage. To address this challenge, a common practice is to employ a sliding window, yet can be inefficient and inflexible due to the limited number of frames within the window. In this work, we propose an end-to-end framework for fast temporal grounding, which is able to model an hours-long video with one-time network execution. Our pipeline is formulated in a coarse-to-fine manner, where we first extract context knowledge from non-overlapped video clips (i.e., anchors), and then supplement the anchors that highly response to the query with detailed content knowledge. Besides the remarkably high pipeline efficiency, another advantage of our approach is the capability of capturing long-range temporal correlation, thanks to modeling the entire video as a whole, and hence facilitates more accurate grounding. Experimental results suggest that, on the long-form video datasets MAD and Ego4d, our method significantly outperforms state-ofthe-arts, and achieves 14.6× / 102.8× higher efficiency respectively. Project can be found at https://github. com/afcedf/SOONet.git.

show abstract