Ring Loss: Convex Feature Normalization for Face Recognition

Zheng, Yuejiu; Pal, Dipan K.; Savvides, Marios

doi:10.1109/cvpr.2018.00534

Cited by 193 publications

(122 citation statements)

References 25 publications

Supporting

Mentioning

122

Contrasting

Order By: Relevance

“…on the Source Only model. This technique is widely used to characterize the feature embeddings under the softmax-related objectives [47,21,51]. Specifically, we set the task-specific features to be two-dimensional and retrain the model.…”

Section: Introductionmentioning

confidence: 99%

Larger Norm More Transferable: An Adaptive Feature Norm Approach for Unsupervised Domain Adaptation

Yang

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

416

236

View full text Add to dashboard Cite

Domain adaptation enables the learner to safely generalize into novel environments by mitigating domain shifts across distributions. Previous works may not effectively uncover the underlying reasons that would lead to the drastic model degradation on the target task. In this paper, we empirically reveal that the erratic discrimination of the target domain mainly stems from its much smaller feature norms with respect to that of the source domain. To this end, we propose a novel parameter-free Adaptive Feature Norm approach. We demonstrate that progressively adapting the feature norms of the two domains to a large range of values can result in significant transfer gains, implying that those task-specific features with larger norms are more transferable. Our method successfully unifies the computation of both standard and partial domain adaptation with more robustness against the negative transfer issue. Without bells and whistles but a few lines of code, our method substantially lifts the performance on the target task and exceeds state-of-the-arts by a large margin (11.5% on Office-Home [45] and 17.1% on VisDA2017 [31]). We hope our simple yet effective approach will shed some light on the future research of transfer learning. Code is available at https

show abstract

Section: Introductionmentioning

confidence: 99%

Larger Norm More Transferable: An Adaptive Feature Norm Approach for Unsupervised Domain Adaptation

Yang

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

416

236

View full text Add to dashboard Cite

show abstract

“…Metric learning is common in few-shot learning. Multiple improvements of the standard softmax and cross-entropy loss are proposed by [49,22,53,45,9] to this end. Traditional methods like siamese networks are also considered [3,40,18] along with models that learn by comparing multiple samples at once [44,50,42].…”

Section: Related Workmentioning

confidence: 99%

Dense Classification and Implanting for Few-Shot Learning

Lifchitz

Avrithis

Picard

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

191

132

View full text Add to dashboard Cite

Training deep neural networks from few examples is a highly challenging and key problem for many computer vision tasks. In this context, we are targeting knowledge transfer from a set with abundant data to other sets with few available examples. We propose two simple and effective solutions: (i) dense classification over feature maps, which for the first time studies local activations in the domain of few-shot learning, and (ii) implanting, that is, attaching new neurons to a previously trained network to learn new, taskspecific features. On miniImageNet, we improve the prior state-of-the-art on few-shot classification, i.e., we achieve 62.5%, 79.8% and 83.8% on 5-way 1-shot, 5-shot and 10shot settings respectively. 1 We use the notation [i] := {1, . . . , i} for i ∈ N.

show abstract

“…The L2-constraint based deep length normalization explained in Section 3.2 uses the norm constraint right before the softmax loss. However, according to [22], such a direct approach through the hard normalization operation results in a nonconvex formulation. It results in local minima generated by the loss function itself and leads to difficulties in optimization.…”

Section: Ring Loss-based Deep Length Normalizationmentioning

confidence: 99%

“…The final speaker representation is produced by aggregating the embeddings from each sub-region. Furthermore, we apply convex length normalization using ring loss [22] to normalize the speaker embedding. We show that ring loss-based deep length normalization performs better than the L2-constraint based one.…”

Section: Introductionmentioning

confidence: 99%

Spatial Pyramid Encoding with Convex Length Normalization for Text-Independent Speaker Verification

Jung¹,

Kim

Lim³

et al. 2019

Interspeech 2019

View full text Add to dashboard Cite

In this paper, we propose a new pooling method called spatial pyramid encoding (SPE) to generate speaker embeddings for text-independent speaker verification. We first partition the output feature maps from a deep residual network (ResNet) into increasingly fine sub-regions and extract speaker embeddings from each sub-region through a learnable dictionary encoding layer. These embeddings are concatenated to obtain the final speaker representation. The SPE layer not only generates a fixed-dimensional speaker embedding for a variable-length speech segment, but also aggregates the information of feature distribution from multi-level temporal bins. Furthermore, we apply deep length normalization by augmenting the loss function with ring loss. By applying ring loss, the network gradually learns to normalize the speaker embeddings using model weights themselves while preserving convexity, leading to more robust speaker embeddings. Experiments on the VoxCeleb1 dataset show that the proposed system using the SPE layer and ring loss-based deep length normalization outperforms both ivector and d-vector baselines. Index Terms: speaker verification, spatial pyramid encoding, learnable dictionary encoding, ring loss, length normalization d-vector systemsWe can classify d-vector based SV systems according to the loss function used. The first one is based on the softmax loss defined in [23] as the combination of a cross-entropy loss, a softmax function and the last fully connected layer [7,8,24]. In this system, a speaker classifier is trained to classify speakers in the training set. The softmax loss encourages the separability of speaker embeddings. However, the softmax loss is not sufficient to learn the discriminative embedding with a large margin, and more researchers began to explore discriminative loss functions for enhanced generalization ability.

show abstract

Ring Loss: Convex Feature Normalization for Face Recognition

Cited by 193 publications

References 25 publications

Larger Norm More Transferable: An Adaptive Feature Norm Approach for Unsupervised Domain Adaptation

Larger Norm More Transferable: An Adaptive Feature Norm Approach for Unsupervised Domain Adaptation

Dense Classification and Implanting for Few-Shot Learning

Spatial Pyramid Encoding with Convex Length Normalization for Text-Independent Speaker Verification

Contact Info

Product

Resources

About