Masked Face Recognition and Temperature Monitoring Systems for Airplane Passenger Using Sensor Fusion

Abstract

Transportation is currently an unavoidable necessity. However, the COVID-19 pandemic has impacted all lines of industry, including the Indonesian aviation transportation industry. Technology is one of the solutions to deal with these problems. The monitoring system of masked face recognition and body temperature detection for the check-in process of passengers at the airport is aimed to be developed in this research. The contribution of this research is that the system can distinguish the type of face mask used. Therefore, this monitoring system classified only medical masks and N95/KN95 respirator masks as ‘Good Masked’. IP camera and thermal camera are used to identify a masked face and body temperature, respectively. The sensor fusion method was used for decision-making on passengers whether they can be departed or not. The decision was taken based on the measured body temperature, the use of standardized face masks, and the face recognition of the airport passengers. Convolutional neural network (CNN) method was used for face and face mask recognition. The CNN model training was conducted four times according to the four proposed scenarios. The CNN model that has been trained can distinguish a masked face and a face without a mask. The best results were obtained in the fourth scenario with the comparison of the training dataset to the testing dataset was 9:1 and the epoch was 500 times. The basic deep learning model used for face detection was the single shot multibox detector (SSD) using the ResNet-10 architecture. Meanwhile, the CNN method with the MobileNetV2 architecture was used to detect the use of masks. The accuracy of the CNN model for face recognition and mask recognition was 100%. All check-in monitoring and verification process data were displayed on the web application which was built on the localhost.