An Overview of Multimodal Biometrics Using the Face and Ear

Abstract

In the recent years, we have witnessed the rapid development of face recognition, though it is still plagued by variations such as facial expressions, pose, and occlusion. In contrast to the face, the ear has a stable 3D structure and is nearly unaffected by aging and expression changes. Both the face and ear can be captured from a distance and in a nonintrusive manner, which makes them applicable to a wider range of application domains. Together with their physiological structure and location, the ear can readily serve as supplement to the face for biometric recognition. It has been a trend to combine the face and ear to develop nonintrusive multimodal recognition for improved accuracy, robustness, and security. However, when either the face or the ear suffers from data degeneration, if the fusion rule is fixed or with inferior flexibility, a multimodal system may perform worse than the unimodal system using only the modality with better quality sample. The biometric quality-based adaptive fusion is an avenue to address this issue. In this paper, we present an overview of the literature about multimodal biometrics using the face and ear. All the approaches are classified into categories according to their fusion levels. In the end, we pay particular attention to an adaptive multimodal identification system, which adopts a general biometric quality assessment (BQA) method and dynamically integrates the face and ear via sparse representation. Apart from a refinement of the BQA and fusion weights selection, we extend the experiments for a more thorough evaluation by using more datasets and more types of image degeneration.

Face Recognition for Automatic Border Control: A Systematic Literature Review

“…What needs to be done to overcome this is to extract features and synthesize faces simultaneously because both provide mutual benefits to each other [8]. The ear in face detection has a fairly good level of stability in the 3D structure and is almost not affected by aging and changing facial expressions [71]. Meanwhile, determining racial and ethnic requires a database containing the characteristics of each racial and ethnic group based on facial features, expressions, gender and so on [72].…”

Boosting the Performance of Deep Ear Recognition Systems Using Generative Adversarial Networks and Mean Class Activation Maps

“…Ear and face features can be used as complementary information in multimodal biometric systems, as both can be captured simultaneously in a single scan. This integration enhances recognition accuracy, frequently in surveillance and tracking applications [6]. For these reasons, more and more research is being conducted into the application of ear recognition in various fields, such as security, surveillance and forensics.…”

Image Fusion and Deep Learning Based Ear Recognition Using Thermal and Visible Images

“…Furthermore, ear recognition systems can complement other biometric modalities, providing identification clues when information from other sources is unreliable or unavailable. Recent studies have demonstrated the importance of ear recognition technology in multimodal biometric systems (Sarangi et al, 2018;Sarangi et al, 2022;Ma et al, 2020;Maity et al, 2020). While ear recognition offers many advantages, it also faces challenges that can impact the recognition process.…”