Integrated neural network-based pupil tracking technology for wearable gaze tracking devices in flight training

Abstract

Pupil tracking technology is a tracking and detection method that uses eye image information to extract real-time position information of the pupil. Detecting the pilot's eye movement patterns and characteristics through pupil movement signals is an important part of monitoring the pilot's physiological characteristics. The current pupil tracking algorithm is prone to problems such as insufficient tracking accuracy and discontinuous pupil signals when faced with problems such as pupil occlusion caused by frequent blinking and loss of pupil information in dark light environments that occur during flight training for pilot students. To increase the tracking accuracy of pilots' pupils, this paper designs an integrated neural network-based pupil tracking technology for wearable gaze tracking devices in flight training. To solve the above problems, this paper builds a pupil positioning model based on the hybrid neural network by combining the feature pyramid and ViT network. On this basis, we built a hybrid neural network pupil tracking model for occluded pupil images based on the pilot eye data characteristics collected during flight training and designed a new loss function suitable for pupil detection. After verification, the pupil tracking algorithm we proposed has significantly improved the visual tracking accuracy with an error range of less than 5 pixels compared with existing methods, and the tracking accuracy can reach up to 85%. In pilot flight training, this algorithm has better pupil tracking stability, can effectively reduce pupil signal interference caused by pupil occlusion, and can achieve more accurate real-time tracking of pupils.