A rapid, non-invasive method for fatigue detection based on voice information

Abstract

Fatigue results from a series of physiological and psychological changes due to continuous energy consumption. It can affect the physiological states of operators, thereby reducing their labor capacity. Fatigue can also reduce efficiency and, in serious cases, cause severe accidents. In addition, it can trigger pathological-related changes. By establishing appropriate methods to closely monitor the fatigue status of personnel and relieve the fatigue on time, operation-related injuries can be reduced. Existing fatigue detection methods mostly include subjective methods, such as fatigue scales, or those involving the use of professional instruments, which are more demanding for operators and cannot detect fatigue levels in real time. Speech contains information that can be used as acoustic biomarkers to monitor physiological and psychological statuses. In this study, we constructed a fatigue model based on the method of sleep deprivation by collecting various physiological indexes, such as P300 and glucocorticoid level in saliva, as well as fatigue questionnaires filled by 15 participants under different fatigue procedures and graded the fatigue levels accordingly. We then extracted the speech features at different instances and constructed a model to match the speech features and the degree of fatigue using a machine learning algorithm. Thus, we established a method to rapidly judge the degree of fatigue based on speech. The accuracy of the judgment based on unitary voice could reach 94%, whereas that based on long speech could reach 81%. Our fatigue detection method based on acoustic information can easily and rapidly determine the fatigue levels of the participants. This method can operate in real time and is non-invasive and efficient. Moreover, it can be combined with the advantages of information technology and big data to expand its applicability.

Air Traffic Controller Fatigue Detection Based on Facial and Vocal Features Using Long Short-Term Memory

“…Additionally, in this study, similar to the studies reported in [23], [37], and [70], the fatigue scale was introduced as a label value for model learning. Previous authors divided the test values of the scale into intervals to form fatigue and wakefulness intervals before inputting this information into their proposed models.…”

“…Shen et al [36] proposed a high-precision feature extraction network to detect controller fatigue states using improved deep learning techniques. Gao et al [37] developed a rapid and non-invasive method to assess the degree of fatigue based on connections between voice features and the level of fatigue as determined by the SSS. Their work highlighted that vocal characteristics could also reflect the level of fatigue, exhibiting strong correlations with the scale test values.…”

“…The MFCC can access relevant dimensions depending on the requirements. In this study, based on a previous study [37], we considered using 12 MFCCs (1-12) as a part of the speech features.…”

Voice Technology to Identify Fatigue from Japanese Speech

“…The vitality section of the SF-36 questionnaire captures the energy and fatigue aspects of health [1]. Fatigue can refer to a variety of conditions such as sleep fatigue [2], work fatigue [3,4,5] and COVID-19 fatigue [6,7]. According to Ware et al [1], the vitality part of the questionnaire was added to capture "differences in subjective well-being".…”

“…To our knowledge, no direct research using audio to detect fatigue based on the SF-36 questionnaire has been done so far. In the area of sleep fatigue detection, the authors of [2] col-lected data from 15 participants and a Support Vector Machine (SVM) was trained on common audio features such as F0, Mel-Frequency Cepstral Coefficients (MFCCs), and loudness. Similarly, for work fatigue detection Krajewski et al [3] collected data from 12 participants and common audio features were used to train an ensemble of machine learning models such as a linear SVM, logistic regression, decision trees, and multi-layer perceptrons.…”