Predicting Fatigue and Psychophysiological Test Performance from Speech for Safety-Critical Environments

Abstract

Automatic systems for estimating operator fatigue have application in safety-critical environments. A system which could estimate level of fatigue from speech would have application in domains where operators engage in regular verbal communication as part of their duties. Previous studies on the prediction of fatigue from speech have been limited because of their reliance on subjective ratings and because they lack comparison to other methods for assessing fatigue. In this paper, we present an analysis of voice recordings and psychophysiological test scores collected from seven aerospace personnel during a training task in which they remained awake for 60 h. We show that voice features and test scores are affected by both the total time spent awake and the time position within each subject’s circadian cycle. However, we show that time spent awake and time-of-day information are poor predictors of the test results, while voice features can give good predictions of the psychophysiological test scores and sleep latency. Mean absolute errors of prediction are possible within about 17.5% for sleep latency and 5–12% for test scores. We discuss the implications for the use of voice as a means to monitor the effects of fatigue on cognitive performance in practical applications.

Neural Network Architecture That Combines Temporal and Summative Features for Infant Cry Classification in the Interspeech 2018 Computational Paralinguistics Challenge

“…In this paper, we develop and evaluate some neuralnetwork architectures for infant cry classification. In this we have built on our previous investigations into the classification of other paralinguistic properties of the voice: for Cognitive Load [3], for Fatigue [4], and for the Common Cold [5]. The strategy in these previous studies has been to create wellmotivated feature sets that capture temporal, spectral and modulational properties of each audio token relevant for the task, and then summarize those features over each token using a number of statistical functions.…”

Rail worker fatigue: Identification, management and countermeasures

“…More recent research has examined the potential of speech analysis to detect fatigue. The research has found a good association between voice production and measures of fatigue, and several studies [46][47][48] have suggested that the analysis of speech can offer a reliable, continuous and objective measure of fatigue. However, the quality of speech recognition is heavily dependent on ambient noise in the environment.…”

It Sounds Like You Have a Cold! Testing Voice Features for the Interspeech 2017 Computational Paralinguistics Cold Challenge

“…Our previous computational paralinguistics studies have shown the importance of feature normalisation to aid learning [3,4]. Without normalisation, learning may be affected by large differences in dynamic range across features, poor distributional properties or outlier values.…”

“…For studying the effect of fatigue on voice, Baykaner et al used a combination of time-domain, spectral domain and modulation domain features [4].…”

“…In this paper, we build on previous approaches we have investigated for the classification of other paralinguistic properties of the voice: for Cognitive Load [3], for Fatigue [4], and for Laughter [5]. Our strategy has been to create wellmotivated feature sets that summarise temporal, spectral and modulational properties of each recording that are then used to train support-vector machine (SVM) or deep-neural network (DNN) classifiers.…”