Automatic identification of experts and performance prediction in the multimodal math data corpus through analysis of speech interaction

Abstract

An analysis of multiparty interaction in the problem solving sessions of the Multimodal Math Data Corpus is presented. The analysis focuses on non-verbal cues extracted from the audio tracks. Algorithms for expert identification and performance prediction (correctness of solution) are implemented based on patterns of speech activity among session participants. Both of these categorisation algorithms employ an underlying graph-based representation of dialogues for each individual problem solving activities. The proposed Bayesian approach to expert prediction proved quite effective, reaching accuracy levels of over 92% with as few as 6 dialogues of training data. Performance prediction was not quite as effective. Although the simple graphmatching strategy employed for predicting incorrect solutions improved considerably over a Monte Carlo simulated baseline (F1 score increased by a factor of 2.3), there is still much room for improvement in this task.

An Assessment of Paralinguistic Acoustic Features for Detection of Alzheimer's Dementia in Spontaneous Speech

“…They used a multilayered perceptron network, achieving a classifier accuracy of 88% for AD vs. healthy subjects based on lexical and acoustic features. A more recent study by Luz et al [40] extracted graph-based features encoding turn-taking patterns and speech rate [46] from the Carolina Conversations Collection [47] (spontaneous interviews of participants with and without an AD). They used these features to create an additive logistic regression model that obtained 85% accuracy in distinguishing dialogues involving an AD speaker.…”

Towards Automatic Collaboration Analytics for Group Speech Data Using Learning Analytics

“…Most studies on CC in the past focused on automated analysis using temporal (time domain features like the energy of the signal, amplitude), spectral indicators of speech (frequency-based features like pitch, rhythm) [ 16 , 20 ] and other non-verbal indicators like total speaking time [ 17 , 18 ], frequency of turn taking [ 21 ] or using machine learning classifiers to analyse these features of speech [ 16 , 22 ]. Therefore, most of these studies focused on the analysis of the non-verbal indicators of audio instead of looking at the verbal audio indicators such as the content of the conversation, actual keywords used, dialogues and the main themes of conversation.…”

Introducing Low-Cost Sensors into the Classroom Settings: Improving the Assessment in Agile Practices with Multimodal Learning Analytics

“…For instance, low-cost sensors were already used to track and measure students motion to evaluate their level of attention in the classroom [40], to estimate their skills in seminars [41] and to classify their postures in oral presentations [6,42]. Moreover, low-cost sensors were also used to register and detect students’ voiced interventions during the resolution of collaborative work activities [28,43].…”