Aerial Gaze Target Recognition Based on Head and Eye Movements

Abstract

Aerial gaze target recognition is an important step in aerial eye control interaction. In order to achieve accurate aerial gaze target recognition, the VT4LM recognition algorithm was constructed in this paper. By this algorithm, the facial images containing the eyes of the flight operators were first inputted into Vision Transformer (ViT) to extract the local features, and the head posture of the flight operators was inputted into four LSTMs to extract the global features. Then, the local and global features of the flight operators were inputted into three fully connected layers, two dropout layers and one softmax classifier. Finally, the recognition result of aerial gaze target was obtained. In this paper the effectiveness of the VT4LM recognition algorithm was verified through the identification of four aerial line-of-sight gaze targets named Head-up display, Accelerator push rod, Control lever and Rudder by four flight operators during simulated flight. The experimental results showed that the accuracy of the VT4LM algorithm for aerial gaze target recognition reached 89.29% and the Cross Entropy Loss was 1.45. Compared to the other three recognition methods, the VT4LM algorithm had the highest recognition accuracy and minimum loss. When using the VT4LM algorithm to detect four simulated flight operators staring at four aerial gaze targets, the recognition accuracy was all higher than 85.00%. It could be seen that the VT4LM algorithm had a good performance in aerial gaze target recognition.