Understanding Human Behaviour in Flight Operation Using Eye-Tracking Technology

Korek, Wojciech Tomasz; Mendez, A.E.; Asad, Hafiz ul; Li, Wen‐Chin; Lone, Mudassir

doi:10.1007/978-3-030-49183-3_24

Cited by 7 publications

(5 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…When evaluating the relationship between humans and equipment through eye-tracking devices, eye movement data were used to analyse human behaviours. This hypothesis was confirmed by many previous researchers [14], such as mental fatigue [15], cognitive load [16], information acquisition [17], situational awareness [18], scanning behaviours [19], attention [20], physiological measurements [21], workload [22, 23], interface evaluation and human-computer interaction [24, 25]. Research on eye movement data that predicted pilots’ behaviour included how different role assignments influenced decisions in high-risk environments, and how to predict pilots’ decisions by paying attention to relevant information about pilots’ choices [26].…”

Section: Introductionsupporting

confidence: 79%

“…scanning behaviours [19], attention [20], physiological measurements [21], workload [22,23], interface evaluation and human-computer interaction [24,25].Research on eye movement data that predicted pilots' behaviour included how different role assignments influenced decisions in high-risk environments, and how to predict pilots' decisions by paying attention to relevant information about pilots' choices [26]. When using eye movement to predict dangerous situations, Francisco [27] found that saccade, gaze, blink and gaze dispersion in horizontal and vertical dimensions in eye movement features were more likely to predict the occurrence of dangerous situations.…”

Section: Application Of Eye Movement Datamentioning

confidence: 99%

See 1 more Smart Citation

A methodology to detect pilot perception of warning information by eye movement data and deep residual shrinkage networks

et al. 2023

View full text Add to dashboard Cite

This paper studied the use of eye movement data to form criteria for judging whether pilots perceive emergency information such as cockpit warnings. In the experiment, 12 subjects randomly encountered different warning information while flying a simulated helicopter, and their eye movement data were collected synchronously. Firstly, the importance of the eye movement features was calculated by ANOVA (analysis of variance). According to the sorting of the importance and the Euclidean distance of each eye movement feature, the warning information samples with different eye movement features were obtained. Secondly, the residual shrinkage network modules were added to CNN (convolutional neural network) to construct a DRSN (deep residual shrinkage networks) model. Finally, the processed warning information samples were used to train and test the DRSN model. In order to verify the superiority of this method, the DRSN model was compared with three machine learning models, namely SVM (support vector machine), RF (radom forest) and BPNN (backpropagation neural network). Among the four models, the DRSN model performed the best. When all eye movement features were selected, this model detected pilot perception of warning information with an average accuracy of 90.4%, of which the highest detection accuracy reached 96.4%. Experiments showed that the DRSN model had advantages in detecting pilot perception of warning information.

show abstract

Section: Introductionsupporting

confidence: 79%

Section: Application Of Eye Movement Datamentioning

confidence: 99%

A methodology to detect pilot perception of warning information by eye movement data and deep residual shrinkage networks

et al. 2023

View full text Add to dashboard Cite

show abstract

“…The study recorded eye movement data and analysed key indicators related to discernibility, perceptibility, and accessibility of HUD information and explored the optimal colour coding scheme for key elements of the HUD against different background brightness levels, with their findings providing insights for improving HUD design and optimising the cognitive performance of pilots in various flight environments, offering guidance for HUD enhancement and reference for future research in this area. Additionally, the analysis of the pilots' gaze distribution is used to measure how the pilots' task load influence visual behaviour and performance [20,21] and, generally, understanding of how the pilot processes the information in the cockpit while carrying out particular tasks [22]. A particular case is [23] as it examines pilot eye movements during approach phases, with results indicating similarities in eye behaviour between the pilot flying and the pilot monitoring.…”

Section: Use Of Eye Tracking In Aviationmentioning

confidence: 99%

Using Eye-Tracking for Adaptive Human-Machine Interfaces for Pilots: A Literature Review and Sample Cases

Xenos,

Mallas,

Minas

2024

J. Phys.: Conf. Ser.

View full text Add to dashboard Cite

This paper explores the potential of eye-tracking technology in adaptive human-machine interfaces for pilots in aviation. We argue that an interface able to adjust its layout and elements based on pilots’ real-time eye-tracking data can prevent errors and enhance their performance. The study presents a literature review on the use of eye-tracking for various pilot cases, including flight simulator games, drone pilots, and cockpit pilots. Results in most cases showed that eye-tracking has been employed to improve interactions, enhance spatial awareness, guide pilots’ gaze to relevant areas, and provide insights into pilots’ information processing and task load. The paper discusses two sample cases demonstrating the potential of eye-tracking in adaptive human-machine interfaces. In the first case, during challenging drone simulations, eye-tracking identified areas where an adaptive human-machine interface could aid navigation and reduce cognitive load. In the second one, based on real drone flights, when signal loss incidents occurred, eye-tracking data showed that the interface should adapt to pilots’ needs by providing critical information to help them to improve situational awareness. The paper concludes that eye-tracking technology has significant potential in adaptive human-machine interfaces for aviation, emphasising the importance of refining these technologies to meet pilots’ specific needs and enhance flight safety.

show abstract

“…The effect of this engagement is one-off and requires redefining the AOI both when changing the operator and the flight task being recorded. This functionality is important, especially in the case of complex Human-Machine Interfaces (HMI) systems, where it is indispensable to transfer large amounts of data in the shortest possible time, e.g., production line operators or operators of ground air traffic control stations of unmanned systems [7][8][9][10][11]. The approach proposed in this paper is based on well-known and used eye-tracking systems which, due to their universality and targeting a different audience, do not allow for application in the areas and tasks that are the purpose of this paper.…”

Section: Introductionmentioning

confidence: 99%

Use of a DNN in Recording and Analysis of Operator Attention in Advanced HMI Systems

et al. 2022

View full text Add to dashboard Cite

The main objective of this research was to propose a smart technology to record and analyse the attention of operators of transportation devices where human–machine interaction occurs. Four simulators were used in this study: General Aviation (GA), Remotely Piloted Aircraft System (RPAS), AS 1600, and Czajka, in which a spatio-temporal trajectory of system operator attention describing the histogram distribution of cockpit instrument observations was sought. Detection of the position of individual instruments in the video stream recorded by the eyetracker was accomplished using a pre-trained Fast R-CNN deep neural network. The training set for the network was constructed using a modified Kanade–Lucas–Tomasi (KLT) algorithm, which was applied to optimise the labelling of the cockpit instruments of each simulator. A deep neural network allows for sustained instrument tracking in situations where classical algorithms stop their work due to introduced noise. A mechanism for the flexible selection of Area Of Interest (AOI) objects that can be tracked in the recorded video stream was used to analyse the recorded attention using a mobile eyetracker. The obtained data allow for further analysis of key skills in the education of operators of such systems. The use of deep neural networks as a detector for selected instrument types has made it possible to universalise the use of this technology for observer attention analysis when applied to a different objects-sets of monitoring and control instruments.

show abstract

Understanding Human Behaviour in Flight Operation Using Eye-Tracking Technology

Cited by 7 publications

References 10 publications

A methodology to detect pilot perception of warning information by eye movement data and deep residual shrinkage networks

A methodology to detect pilot perception of warning information by eye movement data and deep residual shrinkage networks

Using Eye-Tracking for Adaptive Human-Machine Interfaces for Pilots: A Literature Review and Sample Cases

Use of a DNN in Recording and Analysis of Operator Attention in Advanced HMI Systems

Contact Info

Product

Resources

About