Circle Loss: A Unified Perspective of Pair Similarity Optimization

Sun, Yifan; Cheng, Changmao; Zhang, Yuhan; Zhang, Chi; Zheng, Liang; Wang, Zhongdao; Wei, Yichen

doi:10.1109/cvpr42600.2020.00643

Cited by 660 publications

(287 citation statements)

References 23 publications

Supporting

Mentioning

287

Contrasting

Order By: Relevance

“…Note that online mining is switched on for all models here. (Basaldella et al, 2020) 64.6 74.6 NCA loss (Goldberger et al, 2005) 65.2 77.0 Lifted-Structure loss (Oh Song et al, 2016) 62.0 72.1 InfoNCE (Oord et al, 2018;He et al, 2020) 63.3 74.2 Circle loss (Sun et al, 2020) 66.7 78.7…”

Section: B2 Comparing Loss Functionsmentioning

confidence: 99%

“…Lifted-Structure loss (Oh Song et al, 2016) and NCA loss (Goldberger et al, 2005) are two very classic metric learning objectives. Multi-Similarity loss (Wang et al, 2019) and Circle loss (Sun et al, 2020) are two recently proposed metric learning objectives and have been considered as SOTA on large-scale visual recognition benchmarks.…”

Section: B2 Comparing Loss Functionsmentioning

confidence: 99%

“…He et al, 2020) * , 0.5 (Oord et al, 2018)} m in Circle loss {0.25 (Sun et al, 2020) * , 0.4 (Sun et al, 2020)} γ in Circle loss{80(Sun et al, 2020), 256(Sun et al, 2020) …”

mentioning

confidence: 99%

See 2 more Smart Citations

Self-Alignment Pretraining for Biomedical Entity Representations

Liu¹,

Shareghi²,

Meng³

et al. 2021

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Langua

102

View full text Add to dashboard Cite

Despite the widespread success of selfsupervised learning via masked language models (MLM), accurately capturing fine-grained semantic relationships in the biomedical domain remains a challenge. This is of paramount importance for entity-level tasks such as entity linking where the ability to model entity relations (especially synonymy) is pivotal. To address this challenge, we propose SAPBERT, a pretraining scheme that selfaligns the representation space of biomedical entities. We design a scalable metric learning framework that can leverage UMLS, a massive collection of biomedical ontologies with 4M+ concepts. In contrast with previous pipelinebased hybrid systems, SAPBERT offers an elegant one-model-for-all solution to the problem of medical entity linking (MEL), achieving a new state-of-the-art (SOTA) on six MEL benchmarking datasets. In the scientific domain, we achieve SOTA even without taskspecific supervision. With substantial improvement over various domain-specific pretrained MLMs such as BIOBERT, SCIBERT and PUB-MEDBERT, our pretraining scheme proves to be both effective and robust. 1

show abstract

Section: B2 Comparing Loss Functionsmentioning

confidence: 99%

Section: B2 Comparing Loss Functionsmentioning

confidence: 99%

See 1 more Smart Citation

Self-Alignment Pretraining for Biomedical Entity Representations

Liu¹,

Shareghi²,

Meng³

et al. 2021

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Langua

102

View full text Add to dashboard Cite

show abstract

“…However, for the open-set problem, the distribution of the unknown class with multiple categories is completely different from the distribution of the predefined classes. After that, some losses with margins are proposed [3], [7], [12], [16], [17], [21]. Liu et al [3], [7] propose a large margin Softmax (LMSoftmax) by adding multiplicative angular constraints to each identity to improve the feature discrimination on vision classification and face verification.…”

Section: Related Workmentioning

confidence: 99%

“…Although these works achieve promising performance on the open-set problem, they only explicitly minimize intra-class distance. In [21], the circle loss explicitly minimizes intra-class distance and maximizes inter-class distance. However, it requires pair-wise features, one of which is the positive sample feature and the other one is the negative sample feature.…”

Section: Related Workmentioning

confidence: 99%

Normalized Maximal Margin Loss for Open-Set Image Classification

Sun

Zhu

et al. 2021

IEEE Access

View full text Add to dashboard Cite

This work aims to address the image classification problem under open-set protocol: classes in test set do not appear in the training set. Intuitively, convolutional Neural Network (CNN) with softmax loss is a straightforward solution. However, the unknown class (is not predefined in the training set) makes the boundaries of intra-class and inter-class more blurred, which brings more challenges for image classification. Although some softmax variants, such as center loss, CosFace loss etc., focus on learning discriminative features by minimizing the intra-class distance, they do not explicitly maximize inter-class distance, which is more important for open-set problem verified by our experiments. Besides, even though deep metric learning, such as with the contrastive loss and the triplet loss, can learn discriminative features of intra-class and inter-class, it needs a time-consuming image sampling process during training. In this paper, we propose a novel normalized maximal margin (NMM) loss for open-set image classification, which not only explicitly minimizes intra-class distance and maximizes inter-class distance, but also defines their margins. Specially, after analyzing the advantage of angular space that the softmax loss normalized by the feature and weights through geometric interpretation, we make NMM work in angular space. Then, the validity of NMM for discriminative features learning is demonstrated from the view of geometric interpretation as well. After that, we innovatively determine the upper bound of inter-class margin by theoretical analysis. Finally, extensive experiments are conducted on popular datasets: CIFAR-100 (object recognition), ImageNet (image classification), LFW (face recognition) and MSMT17 (person re-identification) to verify the effectiveness of NMM. The experimental results show that NMM achieves very competitive performance.

show abstract

Kernel triplet loss for image‐text retrieval

Pan

Zhang

2022

Computer Animation & Virtual

View full text Add to dashboard Cite

Triplet loss is widely used as the objective function in image-text retrieval tasks.However, as all the triplets are treated equally, triplet loss has a bottleneck problem of slow convergence and other unsatisfactory performances. In this article, we propose solutions by appropriately weighting triplets according to the relative similarities among the training samples. Specifically, we present three weighting functions to assign an appropriate weight for the selected informative triplets to accelerate the convergence. We evaluate our approach on two widely used benchmark datasets: Flickr30k and MSCOCO, with results outperforming the previous methods, which demonstrates its superiority.

show abstract

Circle Loss: A Unified Perspective of Pair Similarity Optimization

Cited by 660 publications

References 23 publications

Self-Alignment Pretraining for Biomedical Entity Representations

Self-Alignment Pretraining for Biomedical Entity Representations

Normalized Maximal Margin Loss for Open-Set Image Classification

Kernel triplet loss for image‐text retrieval

Contact Info

Product

Resources

About