Face Recognition With Masks Based on Spatial Fine-Grained Frequency Domain Broadening

Abstract

Along with social distancing, wearing masks is an effective method of preventing the transmission of COVID-19 in the ongoing pandemic. However, masks occlude a large number of facial features, preventing facial recognition. The recognition rate of existing methods may be significantly reduced by the presence of masks. In this paper, we propose a method to effectively solve the problem of the lack of facial feature information needed to perform facial recognition on people wearing masks. The proposed approach uses image super-resolution technology to perform image preprocessing along with a deep bilinear module to improve EfficientNet. It also combines feature enhancement with frequency domain broadening, fuses the spatial features and frequency domain features of the unoccluded areas of the face, and classifies the fused features. The features of the unoccluded area are increased to improve the accuracy of recognition of masked faces. The results of a cross-validation show that the proposed approach achieved an accuracy of 98% on the RMFRD dataset, as well as a higher recognition rate and faster speed than previous methods. In addition, we also performed an experimental evaluation in an actual facial recognition system and achieved an accuracy of 99%, which demonstrates the effectiveness and practicability of the proposed method.INDEX TERMS Face recognition with mask, convolutional neural network, frequency domain widening, bilinear module, RMFRD dataset.

Visual Privacy: Current and Emerging Regulations Around Unconsented Video Analytics in Retail

“…Singh et al (2017) found that obscuring one's face with a mask, beard, sun glasses or scarf does not ensure one's face cannot still be recorded and identified. As mask mandates of the COVID-19 epidemic became commonplace, facial recognition models have since been optimized to work around face coverings, achieving only a 1% drop in accuracy as compared to unobscured faces (Chen et al, 2022). Even if there was a practical way for a customer to signal non-participation, retailers may not even be required to allow customers to opt-out.…”

Deep Learning Mask Face Recognition with Annealing Mechanism

“…First, the features of the unobscured areas of the eyes and forehead were obtained using a trained CNN model, and then a bag-of-features paradigm was used in the last layer of the network to quantify the features for classification. HUA-QUAN Chen et al [11] first obtained the unmasked eyes and forehead areas in the face image and used ESRGAN [12] to perform an image super-resolution. Next, the steps are divided into two parts based on the neural network.…”

Combining Classifiers for Deep Learning Mask Face Recognition

“…After obtaining the features from the three models, the EUM recognition results are evaluated using a self-restrained triplet loss. Chen et al [24] divided the process into four steps. The first step is to obtain the eye and forehead regions of the face image and perform image super-resolution using ESRGAN; the second step is to use the YCbCr color space in the image for frequency domain broadening analysis and then perform fast independent component analysis feature reduction to obtain features.…”