Concept Propagation via Attentional Knowledge Graph Reasoning for Video-Text Retrieval

Fang, Sheng; Wang, Shuhui; Zhuo, Junbao; Huang, Qingming; Ma, Bin; Wei, Xiaoming; Wei, Xiaolin

doi:10.1145/3503161.3547785

Cited by 2 publications

(2 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…TACo [52] utilizes a token-aware cascade hard negative sampling strategy to select a fixed number of hard negatives within a batch. Moreover, triplet ranking loss with online triplet mining often acts as an auxiliary target to guide the text-video alignment, which usually selects the hardest negative sample to construct triplet samples [7,10,50]. Nevertheless, these strategies may result in sub-optimal learning or missing some hard negatives as the distribution of hard pairs may be either scarce or dense, depending on the batch size.…”

Section: Negative Mining In Contrastive Learningmentioning

confidence: 99%

“…However, hard negatives should make a greater impact on the discrimination between matched and mismatched pairs. Most of the current approaches adopt random sampling strategies [34,57] or a specific sampling strategy [10,50,52] to cut the number of negatives to a fixed number. These strategies may result in sub-optimal learning or overlooking some hard negatives.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Dual-Modal Attention-Enhanced Text-Video Retrieval with Triplet Partial Margin Contrastive Learning

Jiang,

Liu,

et al. 2023

Proceedings of the 31st ACM International Conference on Multimedia

View full text Add to dashboard Cite

In recent years, the explosion of web videos makes text-video retrieval increasingly essential and popular for video filtering, recommendation, and search. Text-video retrieval aims to rank relevant text/video higher than irrelevant ones. The core of this task is to precisely measure the cross-modal similarity between texts and videos. Recently, contrastive learning methods have shown promising results for text-video retrieval, most of which focus on the construction of positive and negative pairs to learn text and video representations. Nevertheless, they do not pay enough attention to hard negative pairs and lack the ability to model different levels of semantic similarity. To address these two issues, this paper improves contrastive learning using two novel techniques. First, to exploit hard examples for robust discriminative power, we propose a novel Dual-Modal Attention-Enhanced Module (DMAE) to mine hard negative pairs from textual and visual clues. By further introducing a Negative-aware InfoNCE (NegNCE) loss, we are able to adaptively identify all these hard negatives and explicitly highlight their impacts in the training loss. Second, our work argues that triplet samples can better model fine-grained semantic similarity compared to pairwise samples. We thereby present a new Triplet Partial Margin Contrastive Learning (TPM-CL) module to

show abstract