Facial masks and soft‐biometrics: Leveraging face recognition CNNs for age and gender prediction on mobile ocular images

Abstract

We address the use of selfie ocular images captured with smartphones to estimate age and gender. Partial face occlusion has become an issue due to the mandatory use of face masks. Also, the use of mobile devices has exploded, with the pandemic further accelerating the migration to digital services. However, state-of-the-art solutions in related tasks such as identity or expression recognition employ large Convolutional Neural Networks, whose use in mobile devices is infeasible due to hardware limitations and size restrictions of downloadable applications. To counteract this, we adapt two existing lightweight CNNs proposed in the context of the ImageNet Challenge, and two additional architectures proposed for mobile face recognition. Since datasets for softbiometrics prediction using selfie images are limited, we counteract over-fitting by using networks pre-trained on ImageNet. Furthermore, some networks are further pretrained for face recognition, for which very large training databases are available. Since both tasks employ similar input data, we hypothesise that such strategy can be beneficial for soft-biometrics estimation. A comprehensive study of the effects of different pretraining over the employed architectures is carried out, showing that, in most cases, a better accuracy is obtained after the networks have been fine-tuned for face recognition. | INTRODUCTIONRecent research has explored the automatic extraction of information such as gender, age, ethnicity, etc. of an individual, known as soft-biometrics [1]. It can be deduced from biometric data like face photos, voice, gait, hand or body images, etc. One of the most natural ways is face analysis [2], but given the use of masks due to the COVID-19 pandemic, the face appears occluded even in cooperative settings, leaving the ocular region as the only visible part. In recent years, the ocular region has gained attention as a stand-alone modality for a variety of tasks, including person recognition [3], softbiometrics estimation [4], or liveness detection [5]. Accordingly, this work is concerned with the challenge of estimating soft-biometrics when only the ocular region is available. Additionally, we are interested in mobile environments [6]. The pandemic has accelerated the migration to the digital domain, converting mobiles in data hubs used for all type of This is an open access article under the terms of the Creative Commons Attribution-NonCommercial License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited and is not used for commercial purposes.

Facial Recognition System for People with and without Face Mask in Times of the COVID-19 Pandemic

“…Table 5 presents a comparison of this proposal with the other works. Other proposals in face detection have also reviewed, but with the ocular recognition variant, as seen in [52,53]. However, it should be clarified that focusing on the human eye represents a different approach than the one analyzed in this document.…”

“…The problem of facial recognition due to the use of face masks during the COVID-19 pandemic has caused new horizons to be explored in artificial intelligence, representing a challenge for researchers, which has motivated the development of ocular recognition systems, as a parallel response. In [52], a facial recognition system using eye information and CNN trained by ImageNet is presented. The results present an accuracy of between 90-95%.…”

Investigating Fairness of Ocular Biometrics Among Young, Middle-Aged, and Older Adults

“…Decades of research have been conducted in extracting representative features from biometric modalities, such as the face, fingerprint, and ocular region, for user recognition and soft-biometric estimation such as gender, race, and agegroup [2], [10], [19]. Biometric technology has been widely adopted in forensics, surveillance, border-control, humancomputer interaction, anonymous customized advertisement system, and image retrieval systems.…”

“…For person authentication and soft-biometric prediction, ocular biometrics in the visible spectrum include scanning regions in and around the eye, such as the iris, conjunctival and episcleral vasculature, and periocular region [1], [2], [16], [19], [20] has been well-established. Due to its high accuracy, privacy, the convenience of capture with a standard RGB camera in mobile devices, and in the presence of a facial mask [5], this modality has received a lot of attention from academia and industry.…”

“…Due to its high accuracy, privacy, the convenience of capture with a standard RGB camera in mobile devices, and in the presence of a facial mask [5], this modality has received a lot of attention from academia and industry. A number of deep learning methods based on fine-tuned convolutional neural networks (CNNs) have been proposed for user authentication as well as softbiometric estimation from ocular regions [1], [2], [16], [19], [20]. Studies [1], [2] suggest gender classification accuracy from the ocular region in the range [71%, 90%] using finetuned VGG-16, ResNet-50, DenseNet, and MobileNet-V2.…”