Hard Example Mining with Auxiliary Embeddings

Smirnov, Evgeny; Ivanova, Elizaveta; Melnikov, Aleksandr; Kalinovskiy, Ilya; Oleinik, Andrei; Luckyanets, Eugene

doi:10.1109/cvprw.2018.00013

Cited by 31 publications

(24 citation statements)

References 46 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As observed from Table 1, most of the datasets are moderately sized having controlled disguise variations. Other than disguised face datasets, a lot of recent [5,26] research in face recognition has focused on large-scale datasets captured in unconstrained environments [21], [22], [23], [24], [25].…”

Section: Introductionmentioning

confidence: 99%

Recognizing Disguised Faces in the Wild

Singh

Vatsa

et al. 2019

IEEE Trans. Biom. Behav. Identity Sci.

View full text Add to dashboard Cite

Research in face recognition has seen tremendous growth over the past couple of decades. Beginning from algorithms capable of performing recognition in constrained environments, the current face recognition systems achieve very high accuracies on large-scale unconstrained face datasets. While upcoming algorithms continue to achieve improved performance, a majority of the face recognition systems are susceptible to failure under disguise variations, one of the most challenging covariate of face recognition. In literature, some algorithms demonstrate promising results on existing disguise datasets, however, most of the disguise datasets contain images with limited variations, often captured in controlled settings. This does not simulate a real world scenario, where both intentional and unintentional unconstrained disguises are encountered by a face recognition system. In this paper, a novel Disguised Faces in the Wild (DFW) dataset is proposed which contains over 11,000 images of 1,000 identities with variations across different types of disguise accessories. The dataset is collected from the Internet, resulting in unconstrained face images similar to real world settings. This is the first-of-a-kind dataset with the availability of impersonator and genuine obfuscated face images for each subject. The proposed DFW dataset has been analyzed in terms of three levels of difficulty: (i) easy, (ii) medium, and (iii) hard in order to showcase the challenging nature of the problem. It is our view that the research community can greatly benefit from the DFW dataset in terms of developing algorithms robust to such adversaries. The proposed dataset was released as part of the First International Workshop and Competition on Disguised Faces in the Wild at the International Conference on Computer Vision and Pattern Recognition, 2018. This paper presents the DFW dataset in detail, including the evaluation protocols, baseline results, performance analysis of the submissions received as part of the competition, and three levels of difficulties of the DFW challenge dataset. Index Terms-FaceRecognition, Disguise in the Wild. ! arXiv:1811.08837v1 [cs.CV]

show abstract

Section: Introductionmentioning

confidence: 99%

Recognizing Disguised Faces in the Wild

Singh

Vatsa

et al. 2019

IEEE Trans. Biom. Behav. Identity Sci.

View full text Add to dashboard Cite

show abstract

“…There are also many complementary methods proposed to build better face recognition models by promoting desired properties of the produced face representations, such as robustness to noisy labels [8] and low image resolution [50], invariance to age [51] and pose [52], ability to mitigate racial bias [53] and domain imbalance [35], [54], to improve the fairness of representations [55]. There are also methods, proposed to overcome the problems with the situations of difficult face appearance variations, like deliberately disguised faces [31], [56] or faces in medical masks [57].…”

Section: Related Work a Face Recognitionmentioning

confidence: 99%

“…These methods could be combined with algorithms of hard class mining [30] and hard example mining [31], or used together as parts of a composite mini-batch [56].…”

Section: Solution To Prototype Obsolescencementioning

confidence: 99%

“…Prototype Memory is orthogonal to loss functions, neural network architectures, hard example mining methods, etc. It can be combined with many different methods, including smart class [27], [29], [30] and example [29], [31] sampling algorithms, margin-enhanced softmax variants [7], [32], [33] and other state-of-the-art face recognition methods [34]- [38].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Prototype Memory for Large-Scale Face Representation Learning

Smirnov¹,

Garaev²,

Galyuk³

2022

IEEE Access

Self Cite

View full text Add to dashboard Cite

Face representation learning using datasets with a massive number of identities requires appropriate training methods. Softmax-based approach, currently the state-of-the-art in face recognition, in its usual "full softmax" form is not suitable for datasets with millions of persons. Several methods, based on the "sampled softmax" approach, were proposed to remove this limitation. These methods, however, have a set of disadvantages. One of them is a problem of "prototype obsolescence": classifier weights (prototypes) of the rarely sampled classes, receive too scarce gradients and become outdated and detached from the current encoder state, resulting in incorrect training signals. This problem is especially serious in ultra-large-scale datasets. In this paper, we propose a novel face representation learning model called Prototype Memory, which alleviates this problem and allows training on a dataset of any size. Prototype Memory consists of the limited-size memory module for storing recent class prototypes and employs a set of algorithms to update it in appropriate way. New class prototypes are generated on the fly using exemplar embeddings in the current mini-batch. These prototypes are enqueued to the memory and used in a role of classifier weights for softmax classification-based training. To prevent obsolescence and keep the memory in close connection with the encoder, prototypes are regularly refreshed, and oldest ones are dequeued and disposed of. Prototype Memory is computationally efficient and independent of dataset size. It can be used with various loss functions, hard example mining algorithms and encoder architectures. We prove the effectiveness of the proposed model by extensive experiments on popular face recognition benchmarks.

show abstract

“…Negative pairs A common and effective strategy in similarity metric learning setting is to focus on hard negative examples during training. For example, for facial features learning, comparing similar identities help differentiating them [10,13,9,7]. This often implies selecting negative pairs with similar embeddings.…”

Section: Triplet Formationmentioning

confidence: 99%