AdaCos: Adaptively Scaling Cosine Logits for Effectively Learning Deep Face Representations

Abstract

The cosine-based softmax losses [21,28,39,8] and their variants [40,38,7] achieve great success in deep learning based face recognition. However, hyperparameter settings in these losses have significant influences on the optimization path as well as the final recognition performance. Manually tuning those hyperparameters heavily relies on user experience and requires many training tricks.In this paper, we investigate in depth the effects of two important hyperparameters of cosine-based softmax losses, the scale parameter and angular margin parameter, by analyzing how they modulate the predicted classification probability. Based on these analysis, we propose a novel cosinebased softmax loss, AdaCos, which is hyperparameter-free and leverages an adaptive scale parameter to automatically strengthen the training supervisions during the training process. We apply the proposed AdaCos loss to largescale face verification and identification datasets, including LFW [13], MegaFace [16], and IJB-C [23] 1:1 Verification. Our results show that training deep neural networks with the AdaCos loss is stable and able to achieve high face recognition accuracy. Our method outperforms state-of-the-art softmax losses [28,40,7] on all the three datasets.

Exploring Deep Learning-Based Architecture, Strategies, Applications and Current Trends in Generic Object Detection: A Comprehensive Review

“…It also designs the appropriate loss function that can enhance the discriminative power of DCNNs based, large-scale face recognition. However, cosine-based softmax losses [167][168][169] provide better results in deep learning-based face recognition. High discriminative features were achieved using an Additive Angular Margin Loss(AcrFace) for face recognition [170].…”

GO Loss: A Gaussian Distribution‐Based Orthogonal Decomposition Loss for Classification

“…First, we plan to apply GO loss to other datasets for a thorough evaluation of its performance under different application scenarios. Second, we will propose a method to quantitatively determine the value of the hyperparameters, such as by visual analytics [6] or adaptive scaling [47].…”

LinCos-Softmax: Learning Angle-Discriminative Face Representations With Linearity-Enhanced Cosine Logits

“…The choice of the scaling hyper-parameter usually relies on heuristic trials, which are both time consuming and inconvenient to use. The automatic selection of has been discussed in [29]. Inspired by these efforts, we designed a simple scheme to automatically determine this scale parameter for different logits, so that their scale ranges are the same.…”

“…Similarly, NormFace [24] normalized both feature vectors and weight vectors to optimize cosine similarity instead of the inner products in the softmax loss, thereby effectively improving the angular discrimination of the features. Besides, the adaptive selection of the scale and margin hyper-parameters were studied in [28,29,47]. Zhang et al studied the settings of scale and angular margin parameter in cosine-based softmax losses and proposed AdaCos [29] to adaptively scale cosine logits to enhance the supervision during training.…”

“…Besides, the adaptive selection of the scale and margin hyper-parameters were studied in [28,29,47]. Zhang et al studied the settings of scale and angular margin parameter in cosine-based softmax losses and proposed AdaCos [29] to adaptively scale cosine logits to enhance the supervision during training. Liu et al proposed an adaptive margin softmax [28] to adaptively adjust the margins for different classes to tackle the problem of imbalanced training data in face recognition.…”