Cognitive Load Estimation From Speech Commands to Simulated Aircraft

Vukovic, Maria; Stolar, Melissa N.; Lech, Margaret

doi:10.1109/taslp.2021.3057492

Cited by 12 publications

(14 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As shown in Table VI, the results reported in this study are consistent with previously reported speech classification based on objective measures [24]. However, most previous studies applied subjective labels to categorize either cognitive load [22,24] or interpersonal trust [7]. This could potentially be due to the fact that the injection of objective workload ratings into the task increases workload due to having to complete an extra sub-task during the experiments.…”

Section: E Comparison With Related Studiessupporting

confidence: 89%

“…The current study continues the speech-based line of research presented in [22,23]. However, objective rather than subjective labeling is investigated to perform workload and performance prediction.…”

Section: Related Workmentioning

confidence: 96%

“…The increasing ubiquity of speech-based communication is a rich source of physiological and mental state information that affords an opportunity for automatic real-time detection of a speaker's workload. Speech classification has been used to predict workload [22,23] and inter-personal trust and familiarity of team members [7]. In [23], a person-dependent approach based on two subjectively labeled workload categories was investigated using an SVM algorithm and a conventional set of low-level acoustic speech parameters.…”

Section: Related Workmentioning

confidence: 99%

“…In [23], a person-dependent approach based on two subjectively labeled workload categories was investigated using an SVM algorithm and a conventional set of low-level acoustic speech parameters. The outcomes of this study were improved in [22] by replacing the traditional acoustic speech features with amplitude spectrograms.…”

Section: Related Workmentioning

confidence: 99%

“…It can be observed that the logarithmic scale reveals more details of the low-frequency range where the fundamental frequency and the first formant information are located. The logarithmic scale was shown to be most effective in related speech classification tasks [22,24,25].…”

Section: Conversion From Time Waveforms To Spectrogram Imagesmentioning

confidence: 99%

See 4 more Smart Citations

Real-Time Team Performance and Workload Prediction From Voice Communications

et al. 2022

Self Cite

View full text Add to dashboard Cite

Automatic prediction of team performance and workload plays a crucial role in team selection, training, evaluation, and re-training processes. This study investigated the potential of using voice analysis of team-based communication for predicting team workload (TW) and team performance (TP). Both the TW and TP categories were labeled objectively. Ten teams of three participants were tasked with completing a computer-based command-and-control simulation that required communication of task-specific information to each team member. Recordings of each participant's voice communications were used to train Convolution Neural Network (CNN) models for each team separately. It was hypothesized that integrating TW and TP information into the prediction process would support the prediction of both TW and TP categories. Two experiments were conducted. In the first experiment, the TP prediction networks were fine-tuned to predict TW, and conversely, the TW prediction networks were fine-tuned to predict TP. In the second experiment, the TP or TW prediction based on the assembly of interconnected TP and TW classifiers was tested. Both experiments confirmed the hypothesis. It was shown that task-related pre-requisite knowledge embedded into the neural network reduced neural network model training time and improved performance without the need to increase the training data size. Predictions based on combined TW and TP classification outcomes -using either separate or interconnected TW or TP classifiers -outperformed the baseline method using a single CNN model trained to predict either TW or TP alone. The classification accuracy was consistent with previously reported cognitive load prediction based on objective measures.

show abstract

Section: E Comparison With Related Studiessupporting

confidence: 89%

Section: Related Workmentioning

confidence: 96%