Detecting Fatigue From Voice Using Speech Recognition

Abstract

Military and civilian experience has shown that longduration assignments present increased risk of performance failures as the mission progresses. This is due to interruption of normal sleep cycles and psychological pressures of the work environment. There continues to be a need for a non-intrusive fatigue assessment system to successfully monitor the level of alertness of personnel during critical missions and activities. Experimental results on human voice show that specific phones have a predictable dependence on fatigue. Hence, precise phonetic identification and alignment are important to voice-based fatigue detection. This paper explores techniques for detecting fatigue from voice using speech recognition to obtain phonetic alignments. A confidence measure was used to filter out less likely word hypotheses from the ASR's output. In this paper we restricted our analysis to dealing with out-of-vocabulary words. The results obtained from voice show strong correlation with other standardized tests such as Sleep Onset Latency and Sleep, Activity, Fatigue, and Task Effectiveness.

Acoustic analysis of the effects of sustained wakefulness on speech

“…Further, although absolute values of F4 variation do not provide direct information on spectral energy, they do give an indication that an acoustic correlate of vocal quality varies as a consequence of sustained wakefulness. Within the sustained wakefulness literature, Greeley et al (2006) examined the correlation between measures of F1-F4 and standardized assessments of fatigue (e.g., SOL). However, unlike the current study, Greeley and colleagues only observed strong correlations (>0.7) between standard measures of fatigue and F3 (/u/, /O/).…”

“…In contrast, speech samples elicited over 36 h of sustained wakefulness revealed a link between fundamental frequency ( f0), cognitive observations, and subjective measures (Whitmore and Fisher, 1996). More recently, Greeley et al (2006Greeley et al ( , 2007 looked at the potential of automatic speech recognition (ASR) software for identifying aspects of speech that may change in the context of sustained wakefulness. Measures of the first four formants (F1-F4) were derived from a list of words that were produced at regular intervals over the course of the experiment.…”

Analysis of affective speech for fatigue detection

“…H.P. Greeley, E. Friets, J.P. Wilson, S. Raghavan, J. Picone J. Berg [3] introduced the Automatic Speech Recognition (ASR) system for detecting fatigue from voice. Milan Sigmund carried out Spectral Analysis of Speech for stress detection [5].…”

“…Experimental results revealed that, there is a dependence of fatigue on human voice [3]. If a person is carrying fatigue or feeling depressed, then it is clearly reflected from his speech.…”

Automatic vocal effort detection for reliable speech recognition

“…ASR performance relation to various stress conditions and emotional states of a user has received an increasing amount of interest in the last decade [6]- [9]. Other sources of variability may arise from speaker's fatigue level [10], [11], various illnesses, alcohol or drug intoxication, etc. The Lombard effect is also an important aspect affecting speech production [6], since every speaker adjusts his/her pronunciation and voice level according to the immediate ambient noise level as well as to the subjectively assessed ability of a listener to understand.…”