Cross-Domain Classification of Drowsiness in Speech: The Case of Alcohol Intoxication and Sleep Deprivation

Abstract

In this work, we study the drowsy state of a speaker, induced by alcohol intoxication or sleep deprivation. In particular, we investigate the coherence between the two pivotal causes of drowsiness, as featured in the Intoxication and Sleepiness tasks of the INTERSPEECH Speaker State Challenge. In this way, we aim to exploit the interrelations between these different, yet highly correlated speaker states, which need to be reliably recognised in safety and security critical environments. To this end, we perform cross-domain classification of alcohol intoxication and sleepiness, thus leveraging the acoustic similarities of these speech phenomena for transfer learning. Further, we conducted in-depth feature analysis to quantitatively assess the task relatedness and to determine the most relevant features for both tasks. To test our methods in realistic contexts, we use the Alcohol Language Corpus and the Sleepy Language Corpus containing in total 60 hours of genuine intoxicated and sleepy speech. In the result, cross-domain classification combined with feature selection yields up to 60.3 % unweighted average recall, which is significantly above-chance (50 %) and highly notable given the mismatch in the training and validation data. Finally, we show that an effective, general drowsiness classifier can be obtained by aggregating the training data from both domains.

Holistic Affect Recognition Using PaNDA: Paralinguistic Non-Metric Dimensional Analysis

“…Importantly, it has been shown that, where MT-SHL suffers a performance drop, incorporating task correlations helps mitigate the effects of negative transfer. Thus, our findings corroborate the importance of task relatedness for inductive transfer learning, dovetailing with previous work [7,8,9]. For example, in prior work [8], it was hypothesized that sleepiness and alcohol intoxication are related, yet, transfer learning did not improve the performance.…”

“…This demonstrates that our notion of task relatedness abstracts away from pure acoustic similarity, so is not simply a function of recording conditions within a dataset. Moreover, the tasks of sleepiness and intoxication are located far away from each other, indicating their dissimilarity that was also found in the work [8]. This pattern is also reflected in Fig.…”

“…Transient speaker states and permanent speaker traits fall under the umbrella of paralinguistic speech phenomena. In paralinguistic research, there are a few works on the joint learning of speaker attributes, such as deception and sincerity [7], inebriation and sleepiness [8], and native language and non-native English prosody [9]. However, the interrelations between manifold affective and other human phenomena remain hitherto under-explored.…”

Sleepiness detection on read speech using simple features

“…The KSS being a semi-continuous measure, the choice is made to split the dataset into two classes: following [4], [6], [9], [15], the samples with a mean KSS>7.5 will be considered as Sleepy Language (SL). On the contrary, the samples with a KSS≤7.5 are labelled as Non Sleepy Language (NSL).…”

“…Since then, there have been only few reported research on the subject [2]. The best performing automatic classification approach is still a system proposed in 2011 [3] and shares the same feature set as most research [4]- [6] computed with openSmile [7]. While some have tried to suggest using different features [8], [9], the common drawback to all of these systems is the lack of possible physiological interpretation of the features due to feature combination techniques.…”