Ear Detection under Uncontrolled Conditions with Multiple Scale Faster Region-Based Convolutional Neural Networks

Abstract

Abstract:Ear detection is an important step in ear recognition approaches. Most existing ear detection techniques are based on manually designing features or shallow learning algorithms. However, researchers found that the pose variation, occlusion, and imaging conditions provide a great challenge to the traditional ear detection methods under uncontrolled conditions. This paper proposes an efficient technique involving Multiple Scale Faster Region-based Convolutional Neural Networks (Faster R-CNN) to detect ears from 2D profile images in natural images automatically. Firstly, three regions of different scales are detected to infer the information about the ear location context within the image. Then an ear region filtering approach is proposed to extract the correct ear region and eliminate the false positives automatically. In an experiment with a test set of 200 web images (with variable photographic conditions), 98% of ears were accurately detected. Experiments were likewise conducted on the Collection J2 of University of Notre Dame Biometrics Database (UND-J2) and University of Beira Interior Ear dataset (UBEAR), which contain large occlusion, scale, and pose variations. Detection rates of 100% and 98.22%, respectively, demonstrate the effectiveness of the proposed approach.

Local and Holistic Feature Fusion for Occlusion-Robust 3D Ear Recognition

“…The accuracies of the methods in [17] and [18] are 100% and 98.4%, respectively, for 252 images from the XM2VTS database. The detection accuracies of the method in [20] are 78.8% and 85.5% for 415 images, that in [24] is 100% for 203 images, and that in [14] is 100% for the UND-J2 database from the UND database. The method in [21] achieved an accuracy rate of 76.43% for 700 images from the ColorFERET database, and the best accuracy rate of the method in [22] for 240 images from the CMU PIE database was 92.92%.…”

“…In this section, we summarize the different aspects of relevant published studies. Most automatic 2D ear detection approaches depend on the mutual morphological properties of the ear, for instance, the characteristic edges [12], model matching [13], and learning algorithms [14].…”

“…Learning-based detection methods have been proposed owing to their accuracy and robustness advantages for ear detection, e.g., the AdaBoost algorithm and its modified version [24,25] and ear detection involving faster region-based convolutional neural network (Faster R-CNN) frameworks [26], a modified multiple scale faster region-based convolutional neural network [14], geometrics morphometrics and deep learning [27] and convolutional encoder-decoder networks for ear detection [28]. Automatic ear detection methods have achieved good performance under no occlusion or noise conditions; for example, the accuracy of the method in [12] is 93.34% for 700 images, and that in [19] is 97.57% for 267 images.…”

“…Although the ear is partially occluded, the performance of ear detection approaches decreases; for example, the method in [17] was tested with 252 images, each with occlusion rates of 40% and 50%, and achieved accuracies of 83% and 66%, respectively. The detector failed in the case where the majority of the ear is occluded [24], and the method in [14] achieved an accuracy rate of 94.7% for a self-built dataset containing many images, each with an occlusion rate greater than 60%. In [28], average IoU values around 50% were achieved for difficult images with variations across pose, race, and occlusion.…”

“…The ear is detected in 2D profile images using the Fast-R-CNN-based detector developed by Zhang and Mu [14]. This detector was chosen because it is fully automatic, fast, and robust against occlusion and has been reported to yield an accuracy of 100% for the UND-J2 profile face database with 1800 images of 415 subjects [14]. The corresponding 3D data were then extracted from the coregistered 3D profile data, as described in [8].…”