Detection of mental fatigue state using heart rate variability and eye metrics during simulated flight

Abstract

Pilot mental fatigue is a growing concern in the aviation field due to its significant contributions to human errors and aviation accidents. Long work hours, sleep loss, circadian rhythm disruption, and workload are well‐known reasons, but there is a need to accurately detect pilot mental fatigue to improve aviation safety. However, due to the highly restricted cockpit environment and the complex nature of mental fatigue, feasible in‐flight detection remains under‐investigated. The objective of this study is to define a promising approach for mental fatigue detection based on psychophysiological measurements in flying‐relevant environments. Eleven participants engaged in a simulated flight experiment, where several conventional heart rate variability (HRV) and ocular indices were examined to study their relevance to mental fatigue. Additionally, a Toeplitz Inverse Covariance‐Based Clustering (TICC) method was performed to determine the ground truth, after which supervised machine learning was adopted to enable automated mental fatigue detection using HRV and eye metrics. Results showed that HRV and eye metrics were sensitive to the mental fatigue induced by prolonged flight‐relevant tasks. The TICC method helped determine the ground truth for mental fatigue and identify its three distinct levels. Furthermore, a supervised learning‐based detection of mental fatigue was achieved using a support vector machine with the greatest detection accuracy of 91.8%. The findings and methodology of this study provide new insights into the fatigue countermeasures in restricted cockpit environment and lay the groundwork for further explorations into the mental fatigue induced by prolonged flight missions.

Mental workload evaluation model of receiver aircraft pilots based on multiple resource theory

“…Consequently, there is a notable research focus on enhancing the efficiency of air combat missions (Li, Zakarija, et al, 2020; Prinzel et al, 2012). The rapid advancement of electronic information technology has led to the development and production of a wide range of new electronic equipment specifically designed for fighter aircraft (Qin et al, 2021). As a consequence, modern air force aircraft are equipped with advanced levels of information and automation, resulting in features like multitasking, multi‐level control, and high resource utilization during flight combat missions (Li, Zakarija, et al, 2020) As fighter structures, compositions, technologies, and performances continue to improve, the complexity of the pilot‐aircraft human–machine system increases.…”

“…All subjects were right-handed, exhibited normal audiovisual reaction capabilities, and did not have color blindness. Similar sample size has been used before in similar experiments(Qin et al, 2021;Scannella et al, 2018). All subjects were military enthusiasts with a basic background in flight simulation.They had flight simulation experience ranging from 70 to 600 h (M = 320.0, SD = 219.27).…”

Gender differences in professional drivers’ fatigue level measured with BAlert mobile app: A psychophysiological, time efficient, accessible, and innovative approach to fatigue management

“…All in all, the Stroop effect isn’t the only objective measure of exhaustion (i.e., central and peripheral) that holds promise. Furthermore, not only has HRV proven to be a valid and accurate metric for assessing physical fatigue, but it has also shown reliability and validity as a metric for assessing mental exhaustion (e.g., Egelund, 1982 ; Laborde et al, 2011 ; Melo et al, 2017 ; Anwer et al, 2021 ; Qin et al, 2021 ). According to Huang et al (2018) , the following HRV parameters are the most significant markers (i.e., sensitive): NN.mean (mean of normal to normal interval), PNN50 (percentage of NN50 divided by total number of NNs), TP (total spectral power), and LF (low frequency from 0.04 to 0.15 Hz).…”

Generalisable machine learning models trained on heart rate variability data to predict mental fatigue

“…In line with this, research on the biomarkers of fatigue has become increasingly important and suggests that fatigue can be estimated on the basis of markers that reflect the activity of the cerebral cortex (e.g., theta activity obtained by means of neuroimaging techniques) or the autonomous nervous system (e.g. Heart Rate Variability [HRV]) to prevent its negative consequences 14 .Machine learning is a relatively novel way of utilising biomarkers to detect fatigue and this approach has captured the attention of science and practice alike [15][16][17][18] . A few studies have used biological signals obtained with electroencephalography (EEG) to train machine learning models that are capable of effectively detecting mental fatigue (i.e., classification models that successfully distinguish between fatigued and non-fatigued states; for a review, see Ref.…”

“…Machine learning is a relatively novel way of utilising biomarkers to detect fatigue and this approach has captured the attention of science and practice alike [15][16][17][18] . A few studies have used biological signals obtained with electroencephalography (EEG) to train machine learning models that are capable of effectively detecting mental fatigue (i.e., classification models that successfully distinguish between fatigued and non-fatigued states; for a review, see Ref.…”