Classification of cognitive load using voice features: A preliminary investigation

Šarlija, Marko; Petrinovic, Davor

doi:10.1109/coginfocom.2017.8268268

Cited by 3 publications

(9 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Audio features from each trial were extracted using the openSMILE Version 2.1.0 feature extraction toolkit (Eyben et al, 2013). This toolkit extracts a wide array of vocal features suitable for signal processing and machine learning analyses (Mijić et al, 2017). The toolkit was configured to use 10 ms moving window, a time period where vocal features can be considered stationary (Rao, 2011), and the "emo_large" feature set was selected.…”

Section: Discussionmentioning

confidence: 99%

“…The sequence of numbers for each digit span trial was fully randomized to avoid providing the opportunity for machine learning algorithms to learn to differentiate digit sequences and not cognitive load. The importance of this randomization is discussed in detail in Mijić et al (2017).…”

Section: Cognitive Assessmentmentioning

confidence: 99%

“…Vocal features change under increased task demand, for example when participants are doing two tasks at the same time, they show more variable or shorter silence periods, and increased pitch and volume (Segbroeck et al, 2014;Lopes et al, 2018). Speech data has emerged as a non-invasive measure of cognitive load and may prove a valuable extension to tests in which vocal responses are already required (Mijić et al, 2017).…”

Section: Introductionmentioning

confidence: 99%

“…Instantaneous data can be recorded in a nonintrusive setting using simple devices such as smartphones. Data from small samples under experimental settings manipulating cognitive load, have shown the ability to distinguish mental effort based on various voice parameters, and analyzed using machine learning classifiers (Yin et al, 2008;Segbroeck et al, 2014;Magnúsdóttir et al, 2017;Mijić et al, 2017). Nonetheless, further work is needed to validate vocal features as a measure of cognitive load and this necessitates larger datasets to improve detection accuracy.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Prediction of mental effort derived from an automated vocal biomarker using machine learning in a large-scale remote sample

Taptiklis

Barnett

et al. 2023

Front. Artif. Intell.

View full text Add to dashboard Cite

IntroductionBiomarkers of mental effort may help to identify subtle cognitive impairments in the absence of task performance deficits. Here, we aim to detect mental effort on a verbal task, using automated voice analysis and machine learning.MethodsAudio data from the digit span backwards task were recorded and scored with automated speech recognition using the online platform NeuroVocalixTM, yielding usable data from 2,764 healthy adults (1,022 male, 1,742 female; mean age 31.4 years). Acoustic features were aggregated across each trial and normalized within each subject. Cognitive load was dichotomized for each trial by categorizing trials at >0.6 of each participants' maximum span as “high load.” Data were divided into training (60%), test (20%), and validate (20%) datasets, each containing different participants. Training and test data were used in model building and hyper-parameter tuning. Five classification models (Logistic Regression, Naive Bayes, Support Vector Machine, Random Forest, and Gradient Boosting) were trained to predict cognitive load (“high” vs. “low”) based on acoustic features. Analyses were limited to correct responses. The model was evaluated using the validation dataset, across all span lengths and within the subset of trials with a four-digit span. Classifier discriminant power was examined with Receiver Operating Curve (ROC) analysis.ResultsParticipants reached a mean span of 6.34 out of 8 items (SD = 1.38). The Gradient Boosting classifier provided the best performing model on test data (AUC = 0.98) and showed excellent discriminant power for cognitive load on the validation dataset, across all span lengths (AUC = 0.99), and for four-digit only utterances (AUC = 0.95).DiscussionA sensitive biomarker of mental effort can be derived from vocal acoustic features in remotely administered verbal cognitive tests. The use-case of this biomarker for improving sensitivity of cognitive tests to subtle pathology now needs to be examined.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Cognitive Assessmentmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Prediction of mental effort derived from an automated vocal biomarker using machine learning in a large-scale remote sample

Taptiklis

Barnett

et al. 2023

Front. Artif. Intell.

View full text Add to dashboard Cite

show abstract

“…In our previous research we have focused on developing optimized stimulation paradigms for elicitation of multimodal responses related to stress resilience (Dropuljić et al, 2017;Ćosić et al, 2019b) and cognitive functioning estimation (Mijić et al, 2017(Mijić et al, , 2019. Acoustic feature sets were related to: (a) fundamental frequency, an estimate of the base harmonic that the vibrating vocal cords are producing during vocalization; RMS energy, a signal-processing-based estimate of the energy of the sound recorded by the microphone; (b) formant frequencies (f1-f4) and mel-frequency-cepstral-coefficients which describe the spectral and cepstral behavior of the recorded utterances; (c) jitter and shimmer, representing voice perturbations; and (d) number of voiced segments per second and mean voiced/unvoiced segment lengths in seconds, which are related to speech rate.…”

Section: Acoustic Featuresmentioning

confidence: 99%

AI-Based Prediction and Prevention of Psychological and Behavioral Changes in Ex-COVID-19 Patients

et al. 2021

View full text Add to dashboard Cite

The COVID-19 pandemic has adverse consequences on human psychology and behavior long after initial recovery from the virus. These COVID-19 health sequelae, if undetected and left untreated, may lead to more enduring mental health problems, and put vulnerable individuals at risk of developing more serious psychopathologies. Therefore, an early distinction of such vulnerable individuals from those who are more resilient is important to undertake timely preventive interventions. The main aim of this article is to present a comprehensive multimodal conceptual approach for addressing these potential psychological and behavioral mental health changes using state-of-the-art tools and means of artificial intelligence (AI). Mental health COVID-19 recovery programs at post-COVID clinics based on AI prediction and prevention strategies may significantly improve the global mental health of ex-COVID-19 patients. Most COVID-19 recovery programs currently involve specialists such as pulmonologists, cardiologists, and neurologists, but there is a lack of psychiatrist care. The focus of this article is on new tools which can enhance the current limited psychiatrist resources and capabilities in coping with the upcoming challenges related to widespread mental health disorders. Patients affected by COVID-19 are more vulnerable to psychological and behavioral changes than non-COVID populations and therefore they deserve careful clinical psychological screening in post-COVID clinics. However, despite significant advances in research, the pace of progress in prevention of psychiatric disorders in these patients is still insufficient. Current approaches for the diagnosis of psychiatric disorders largely rely on clinical rating scales, as well as self-rating questionnaires that are inadequate for comprehensive assessment of ex-COVID-19 patients’ susceptibility to mental health deterioration. These limitations can presumably be overcome by applying state-of-the-art AI-based tools in diagnosis, prevention, and treatment of psychiatric disorders in acute phase of disease to prevent more chronic psychiatric consequences.

show abstract

Food Choices after Cognitive Load: An Affective Computing Approach

Kappattanavar

Hecker

Moontaha

et al. 2023

Sensors

View full text Add to dashboard Cite

Psychology and nutritional science research has highlighted the impact of negative emotions and cognitive load on calorie consumption behaviour using subjective questionnaires. Isolated studies in other domains objectively assess cognitive load without considering its effects on eating behaviour. This study aims to explore the potential for developing an integrated eating behaviour assistant system that incorporates cognitive load factors. Two experimental sessions were conducted using custom-developed experimentation software to induce different stimuli. During these sessions, we collected 30 h of physiological, food consumption, and affective states questionnaires data to automatically detect cognitive load and analyse its effect on food choice. Utilising grid search optimisation and leave-one-subject-out cross-validation, a support vector machine model achieved a mean classification accuracy of 85.12% for the two cognitive load tasks using eight relevant features. Statistical analysis was performed on calorie consumption and questionnaire data. Furthermore, 75% of the subjects with higher negative affect significantly increased consumption of specific foods after high-cognitive-load tasks. These findings offer insights into the intricate relationship between cognitive load, affective states, and food choice, paving the way for an eating behaviour assistant system to manage food choices during cognitive load. Future research should enhance system capabilities and explore real-world applications.

show abstract

Classification of cognitive load using voice features: A preliminary investigation

Cited by 3 publications

References 18 publications

Prediction of mental effort derived from an automated vocal biomarker using machine learning in a large-scale remote sample

Prediction of mental effort derived from an automated vocal biomarker using machine learning in a large-scale remote sample

AI-Based Prediction and Prevention of Psychological and Behavioral Changes in Ex-COVID-19 Patients

Food Choices after Cognitive Load: An Affective Computing Approach

Contact Info

Product

Resources

About