Contrastive Learning with Large Memory Bank and Negative Embedding Subtraction for Accurate Copy Detection

Yokoo, Shuhei

doi:10.48550/arxiv.2112.04323

Cited by 5 publications

(12 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Out of the 100 most similar pairs for each metric, the NPCK recovers a far higher fraction of near-duplicates (Figure 16 and Figure 17). Only a specially trained duplicate detection model (Yokoo, 2021) performs comparably.…”

Section: Neural Posterior Correlation Kernelmentioning

confidence: 99%

“…We report two-sided p-values. (Douze et al, 2021) descriptor track winner (Yokoo, 2021). The ResNet penultimate features and CLIP embeddings both match many pairs which are not visually similar.…”

Section: B3 Theoremmentioning

confidence: 99%

“…(Middle:) CLIP (Radford et al, 2021) vision tower embeddings. (Bottom:) FBISC challenge (Douze et al, 2021) descriptor track winner(Yokoo, 2021). The ResNet penultimate features and CLIP embeddings both match many pairs which are not visually similar.…”

mentioning

confidence: 99%

See 2 more Smart Citations

The locus coeruleus broadcasts prediction errors across the cortex to promote sensorimotor plasticity

Jordan

Keller

2023

Preprint

View full text Add to dashboard Cite

Prediction errors are differences between expected and actual sensory input and are thought to be key computational signals that drive learning related plasticity. One way that prediction errors could drive learning is by activating neuromodulatory systems to gate plasticity. The catecholaminergic locus coeruleus (LC) is a major neuromodulatory system involved in neuronal plasticity in the cortex. Using two-photon calcium imaging in mice exploring a virtual environment, we found that the activity of LC axons in the cortex correlated with the magnitude of unsigned visuomotor prediction errors. LC response profiles were similar in both motor and visual cortical areas, indicating that LC axons broadcast prediction errors throughout the dorsal cortex. While imaging calcium activity in layer 2/3 of the primary visual cortex, we found that optogenetic stimulation of LC axons facilitated learning of a stimulus-specific suppression of visual responses during locomotion. This plasticity – induced by minutes of LC stimulation – recapitulated the effect of visuomotor learning on a scale that is normally observed during visuomotor development across days. We conclude that prediction errors drive LC activity, and that LC activity facilitates sensorimotor plasticity in the cortex, consistent with a role in modulating learning rates.

show abstract

Section: Neural Posterior Correlation Kernelmentioning

confidence: 99%

Section: B3 Theoremmentioning

confidence: 99%

See 1 more Smart Citation

The locus coeruleus broadcasts prediction errors across the cortex to promote sensorimotor plasticity

Jordan

Keller

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Another landmark work, MoCo [11], improves upon the InfoNCE work through the use of a momentum contrast mechanism that improves convergence. [43] further advocates to train convolutional neural networks with contrastive learning and hard data augmentation, trying to explore more discriminative representations. Both [5] and [43] employ a similar strategy of data augmentation and take an augmented image as a positive (similar) sample.…”

Section: Contrastive Learningmentioning

confidence: 99%

“…[43] further advocates to train convolutional neural networks with contrastive learning and hard data augmentation, trying to explore more discriminative representations. Both [5] and [43] employ a similar strategy of data augmentation and take an augmented image as a positive (similar) sample. Such data augmentations include cropping, grayscale, blocking part of the picture, and horizontal flipping, which are quite similar to the strategies adopted in video infringement.…”

Section: Contrastive Learningmentioning

confidence: 99%

Video Infringement Detection via Feature Disentanglement and Mutual Information Maximization

Liu,

Yu,

Wang

et al. 2023

Proceedings of the 31st ACM International Conference on Multimedia

View full text Add to dashboard Cite

The self-media era provides us tremendous high quality videos. Unfortunately, frequent video copyright infringements are now seriously damaging the interests and enthusiasm of video creators. Identifying infringing videos is therefore a compelling task. Current state-of-the-art methods tend to simply feed high-dimensional mixed video features into deep neural networks and count on the networks to extract useful representations. Despite its simplicity, this paradigm heavily relies on the original entangled features and lacks constraints guaranteeing that useful task-relevant semantics are extracted from the features.In this paper, we seek to tackle the above challenges from two aspects: (1) We propose to disentangle an original high-dimensional feature into multiple sub-features, explicitly disentangling the feature into exclusive lower-dimensional components. We expect the sub-features to encode non-overlapping semantics of the original feature and remove redundant information. (2) On top of the disentangled sub-features, we further learn an auxiliary feature to enhance the sub-features. We theoretically analyzed the mutual information between the label and the disentangled features, arriving

show abstract

TransVCL: Attention-Enhanced Video Copy Localization Network with Flexible Supervision

et al. 2023

AAAI

View full text Add to dashboard Cite

Video copy localization aims to precisely localize all the copied segments within a pair of untrimmed videos in video retrieval applications. Previous methods typically start from frame-to-frame similarity matrix generated by cosine similarity between frame-level features of the input video pair, and then detect and refine the boundaries of copied segments on similarity matrix under temporal constraints. In this paper, we propose TransVCL: an attention-enhanced video copy localization network, which is optimized directly from initial frame-level features and trained end-to-end with three main components: a customized Transformer for feature enhancement, a correlation and softmax layer for similarity matrix generation, and a temporal alignment module for copied segments localization. In contrast to previous methods demanding the handcrafted similarity matrix, TransVCL incorporates long-range temporal information between feature sequence pair using self- and cross- attention layers. With the joint design and optimization of three components, the similarity matrix can be learned to present more discriminative copied patterns, leading to significant improvements over previous methods on segment-level labeled datasets (VCSL and VCDB). Besides the state-of-the-art performance in fully supervised setting, the attention architecture facilitates TransVCL to further exploit unlabeled or simply video-level labeled data. Additional experiments of supplementing video-level labeled datasets including SVD and FIVR reveal the high flexibility of TransVCL from full supervision to semi-supervision (with or without video-level annotation). Code is publicly available at https://github.com/transvcl/TransVCL.

show abstract

Contrastive Learning with Large Memory Bank and Negative Embedding Subtraction for Accurate Copy Detection

Cited by 5 publications

References 15 publications

The locus coeruleus broadcasts prediction errors across the cortex to promote sensorimotor plasticity

The locus coeruleus broadcasts prediction errors across the cortex to promote sensorimotor plasticity

Video Infringement Detection via Feature Disentanglement and Mutual Information Maximization

TransVCL: Attention-Enhanced Video Copy Localization Network with Flexible Supervision

Contact Info

Product

Resources

About