Variational Prototype Learning for Deep Face Recognition

Abstract

Deep face recognition has achieved remarkable improvements due to the introduction of margin-based softmax loss, in which the prototype stored in the last linear layer represents the center of each class. In these methods, training samples are enforced to be close to positive prototypes and far apart from negative prototypes by a clear margin. However, we argue that prototype learning only employs sample-to-prototype comparisons without considering sample-to-sample comparisons during training and the low loss value gives us an illusion of perfect feature embedding, impeding the further exploration of SGD. To this end, we propose Variational Prototype Learning (VPL), which represents every class as a distribution instead of a point in the latent space. By identifying the slow feature drift phenomenon, we directly inject memorized features into prototypes to approximate variational prototype sampling. The proposed VPL can simulate sample-to-sample comparisons within the classification framework, encour-aging the SGD solver to be more exploratory, while boosting performance. Moreover, VPL is conceptually simple, easy to implement, computationally efficient and memory saving. We present extensive experimental results on popular benchmarks, which demonstrate the superiority of the proposed VPL method over the state-of-the-art competitors.

AnchorFace: Boosting TAR@FAR for Practical Face Recognition

“…Moreover, CurricularFace (Huang et al 2020) and MV-Arc-Softmax (Wang et al 2020) are used to introduce the mining-based strategies to emphasize the mis-classified samples. The recent work VPL (Deng et al 2021) first analyzes the limitations of previous methods, which employ sample-to-prototype comparisons during training without considering sample-to-sample comparisons, and then introduces the sample-to-sample comparisons into the classification framework for FR. In contrast to existing works, our proposed AnchorFace discusses the necessity of the optimization under the Anchor FAR (i.e., Anchor Optimization) for practical FR from a new perspective, and introduces a pair of loss functions to and reduce the gap of the training and evaluation for FR.…”

“…Meanwhile, as discussed in VPL (Deng et al 2021), features drift slowly for FR models, which indicates that features extracted previously can be considered as an approximation of the output of the current network within a certain number of training steps. Therefore, we also create a validness indicator V ∈ R N ×K to represent the validness of each feature in the online-updating set S. Each item in V is a scalar value, which denotes the remaining valid steps for the corresponding feature in the online-updating set S. The maximum number of valid steps for each feature is M , and we initialize all items of V as 0 at the beginning of the training process.…”

“…There are also some works that analyze the gaps between the training and evaluation for FR. For example, for most softmax-based loss functions, sampleto-prototype similarities are optimized in the training process, while sample-to-sample similarities are used in practice (Deng et al 2021). Furthermore, domain shift is another common gap (Sohn et al 2017), where the FR models per- form poorly on unseen ethnicity if the distribution of the training data is severely biased.…”

Integrating Language Guidance into Vision-based Deep Metric Learning

“…However, as the usage of tuples introduces an exponential increase in sample complex-ity [94,115], similar emphasis has also been placed in tuple selection heuristics to boost training speeds and generalization, either based on sample distances [94,100,110,115], hierarchical arrangements [33] or adapted to the training process [38,89]. Tuple complexity can also be addressed using proxies as stand-in replacements in the generation of tuples [20,50,71,82,102,125]. However, while literature results suggests increasing generalization performance based on simple changes in re-ranking and tuple selection, recent work has instead highlighted a much stronger saturation in method performance [30,72,91], underlining the importance of fair and comparable training and evaluation protocols with fixed backbone network and pipeline parameter choices.…”

Face Morphing: Fooling a Face Recognition System Is Simple!

“…Recent FR systems [8,9,22] report face verification results exceeding 99.5% on the arguably simple labeled faces in the wild (LFW) dataset [17], yet also reach 92% under more challenging cross-age and cross-pose scenarios. First analyses of FR performance under morphing attacks have been published before [23,27,35,37,39,40,44].…”