TSP: Temporally-Sensitive Pretraining of Video Encoders for Localization Tasks

Alwassel, Humam; Giancola, Silvio; Ghanem, Bernard

doi:10.1109/iccvw54120.2021.00356

Cited by 80 publications

(46 citation statements)

References 71 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…0.95. Using the pre-training method from TSP [2] slightly improves our results (36.0% average mAP). Our model thus outperforms the best method with the same features [83] by a small margin.…”

Section: Results On Activitynet-13mentioning

confidence: 75%

“…To show that our method can adapt to different video features, we also consider the pre-training method from [2] using an R(2+1)D network [63].…”

Section: Results On Thumos14mentioning

confidence: 99%

“…Moreover, we combined external classification results from [86] following [4,76,80,84]. Similarly, we consider the pre-training method from [2] and compare our model to the same set of baselines, including our close competitor of single-stage models.…”

Section: Results On Activitynet-13mentioning

confidence: 99%

“…Another limitation is the need for many human labeled videos for training and the constraint of a pre-defined vocabulary of actions. Interesting future directions include pre-training for action localization [2,75], and learning from videos and text corpus [30,53] without human labels.…”

Section: Conclusion and Discussionmentioning

confidence: 99%

“…Classification Head. Given the feature pyramid Z, our classification head examines each moment t across all L levels on the pyramid, and predicts the probability of action p(a t ) at every moment t. 2 This head is realized using a lightweight 1D convolutional network attached to each pyramid level. And the parameters are shared across all levels.…”

Section: Decoding Actions In Timementioning

confidence: 99%

See 4 more Smart Citations

ActionFormer: Localizing Moments of Actions with Transformers

Zhang¹,

Wu²,

Li³

2022

Preprint

View full text Add to dashboard Cite

Self-attention based Transformer models have demonstrated impressive results for image classification and object detection, and more recently for video understanding. Inspired by this success, we investigate the application of Transformer networks for temporal action localization in videos. To this end, we present ActionFormer-a simple yet powerful model to identify actions in time and recognize their categories in a single shot, without using action proposals or relying on pre-defined anchor windows. ActionFormer combines a multiscale feature representation with local selfattention, and uses a light-weighted decoder to classify every moment in time and estimate the corresponding action boundaries. We show that this orchestrated design results in major improvements upon prior works. Without bells and whistles, ActionFormer achieves 65.6% mAP at tIoU=0.5 on THUMOS14, outperforming the best prior model by 8.7 absolute percentage points and crossing the 60% mAP for the first time. Further, ActionFormer demonstrates strong results on ActivityNet 1.3 (36.0% average mAP) and the more recent EPIC-Kitchens 100 (+13.5% average mAP over prior works). Our code is available at https://github. com/happyharrycn/actionformer_release.

show abstract

Section: Results On Activitynet-13mentioning

confidence: 75%

“…To show that our method can adapt to different video features, we also consider the pre-training method from [2] using an R(2+1)D network [63].…”

Section: Results On Thumos14mentioning

confidence: 99%

Section: Results On Activitynet-13mentioning

confidence: 99%

Section: Conclusion and Discussionmentioning

confidence: 99%

Section: Decoding Actions In Timementioning

confidence: 99%

See 3 more Smart Citations

ActionFormer: Localizing Moments of Actions with Transformers

Zhang¹,

Wu²,

Li³

2022

Preprint

View full text Add to dashboard Cite

show abstract

The Solution of Huawei Cloud & Noah’s Ark Lab to the NLPCC-2020 Challenge: Light Pre-Training Chinese Language Model for NLP Task

Zhang

Wang

et al. 2020

Natural Language Processing and Chinese Computing

View full text Add to dashboard Cite

The performance of Large Language Models (LLMs) in reasoning tasks depends heavily on prompt design, with Chain-of-Thought (CoT) and self-consistency being critical methods that enhance this ability. However, these methods do not fully exploit the answers generated by the LLM to guide subsequent responses. This paper proposes a new prompting method, named Progressive-Hint Prompting (PHP), that enables automatic multiple interactions between users and LLMs by using previously generated answers as hints to progressively guide toward the correct answers. PHP is orthogonal to CoT and self-consistency, making it easy to combine with state-of-the-art techniques to further improve performance. We conducted an extensive and comprehensive evaluation to demonstrate the effectiveness of the proposed method. Our experimental results on six benchmarks show that combining CoT and self-consistency with PHP significantly improves accuracy while remaining highly efficient. For instance, with text-davinci-003, we observed a 4.2% improvement on GSM8K with greedy decoding compared to Complex CoT, and a 46.17% reduction in sample paths with self-consistency. With GPT-4 and PHP, we achieve state-of-the-art performances on SVAMP (91.9%), GSM8K (95.5%) and AQuA (79.9%).Preprint. Under review.

show abstract