Registration and Analysis of a Pilot’s Attention Using a Mobile Eyetracking System

Gomółka, Zbigniew; Twaróg, Bogusław; Zeslawska, Ewa; Kordos, Damian

doi:10.1007/978-3-030-19501-4_21

Cited by 3 publications

(5 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Existing mobile eyetracker software does not provide seamless AOI analysis in recorded video streams. The products of leading eye-tracking system manufacturers do not provide such functionality [1][2][3]. This paper extends earlier research's related with pilot attention tracking during key procedures, i.e., take-off and landing, which was carried out using the SMI RED 500 and Tobii T60 stationary eyetrackers and the Tobii Glasses mobile eyetracker.…”

Section: Introductionmentioning

confidence: 62%

“…To construct the spatio-temporal trajectory, the mechanism of coincidence detection and contour blurring of the observed instruments described in [1,2] was used. Each frame from the recorded video stream can be used as the background of the resulting graph, regarding limitations that all instruments used by the operator involved in attention trajectory analysis are displayed correctly.…”

Section: Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Use of a DNN in Recording and Analysis of Operator Attention in Advanced HMI Systems

et al. 2022

View full text Add to dashboard Cite

The main objective of this research was to propose a smart technology to record and analyse the attention of operators of transportation devices where human–machine interaction occurs. Four simulators were used in this study: General Aviation (GA), Remotely Piloted Aircraft System (RPAS), AS 1600, and Czajka, in which a spatio-temporal trajectory of system operator attention describing the histogram distribution of cockpit instrument observations was sought. Detection of the position of individual instruments in the video stream recorded by the eyetracker was accomplished using a pre-trained Fast R-CNN deep neural network. The training set for the network was constructed using a modified Kanade–Lucas–Tomasi (KLT) algorithm, which was applied to optimise the labelling of the cockpit instruments of each simulator. A deep neural network allows for sustained instrument tracking in situations where classical algorithms stop their work due to introduced noise. A mechanism for the flexible selection of Area Of Interest (AOI) objects that can be tracked in the recorded video stream was used to analyse the recorded attention using a mobile eyetracker. The obtained data allow for further analysis of key skills in the education of operators of such systems. The use of deep neural networks as a detector for selected instrument types has made it possible to universalise the use of this technology for observer attention analysis when applied to a different objects-sets of monitoring and control instruments.

show abstract

Section: Introductionmentioning

confidence: 62%

Section: Resultsmentioning

confidence: 99%

Use of a DNN in Recording and Analysis of Operator Attention in Advanced HMI Systems

et al. 2022

View full text Add to dashboard Cite

show abstract

“…The presented research is a continuation of the research project related to the examination of sight activity of a pilot during the flight. The analysis and visualization of the registered pilot's eye attention using the SMI RED500 stationary eye tracker was described in [3,22]. During the tests performed on the simulator, the following assumptions were introduced:…”

Section: Test Stand Description and The Assumed Air Taskmentioning

confidence: 99%

“…At present, there is a lack of objective systems that would allow for the determination of the level of training of persons operating (not only) mechanical equipment [1][2][3][4][5]. An interesting example is the system examining pilots' scanning behavior when executing maneuvers at airports in varying visibility conditions [6][7][8].…”

Section: Introductionmentioning

confidence: 99%

“…As an example, the work enabling statistical elaboration of observers attention for the three minute video sequence (MJPG2000, 640 × 480 pixels at 30 fps) including nine instruments in a cockpit requires the effort of a few days of manpower (ca. 48 h of work) [3,22,23]. The effect of this work is non-recurrent and requires redefinition of AOIs both when changing an operator and the registered aeronautical task.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

The Application of Flexible Areas of Interest to Pilot Mobile Eye Tracking

Gomółka

Kordos

Zeslawska

2020

Sensors

Self Cite

View full text Add to dashboard Cite

Recent progress in the development of mobile Eye Tracking (ET) systems shows that there is a demand for modern flexible solutions that would allow for dynamic tracking of objects in the video stream. The paper describes a newly developed tool for work with ET glasses, and its advantages are outlined with the example of a pilot study. A flight task is performed on the FNTP II MCC simulator, and the pilots are equipped with the Mobile Tobii Glasses. The proposed Smart Trainer tool performs dynamic object tracking in a registered video stream, allowing for an interactive definition of Area of Interest (AOI) with blurred contours for the individual cockpit instruments and for the construction of corresponding histograms of pilot attention. The studies are carried out on a group of experienced pilots with a professional pilot CPL(A) license with instrumental flight (Instrument Rating (IR)) certification and a group of pilots without instrumental training. The experimental section shows the differences in the perception of the flight process between two distinct groups of pilots with varying levels in flight training for the ATPL(A) line pilot license. The proposed Smart Trainer tool might be exploited in order to assess and improve the process of training operators of advanced systems with human machine interfaces.

show abstract