Classification of Formal and Informal Dialogues Based on Turn-Taking and Intonation Using Deep Neural Networks

Abstract

Abstract. Here, we introduce a classification method for distinguishing between formal and informal dialogues using feature sets based on prosodic data. One such feature set is the raw fundamental frequency values paired with speaker information (i.e. turn-taking). The other feature set we examine is the prosodic labels extracted from the raw F0 values via the ProsoTool algorithm, which is also complemented by turn-taking. We evaluated the two feature sets by comparing the accuracy scores our classification method got, which uses them to classify dialogue-excerpts taken from the HuComTech corpus. With the ProsoTool features we achieved an average accuracy score of 85.2%, which meant a relative error rate reduction of 24% compared to the accuracy scores attained using F0 features. Regardless of the feature set applied, however, our method yields better accuracy scores than those got by human listeners, who only managed to distinguish between formal and informal dialogue to an accuracy level of 56.5%.

Classification of Formal and Informal Dialogues Based on Emotion Recognition Features

“…In our experiments for the classification of segments taken from these dialogues, we used the train/development/test partitioning established earlier for the task [8]. The reason for this decision was to make our results comparable with those reported by Szekrényes and Kovács [8].…”

“…In our experiments for the classification of segments taken from these dialogues, we used the train/development/test partitioning established earlier for the task [8]. The reason for this decision was to make our results comparable with those reported by Szekrényes and Kovács [8]. In their study, they attempted to distinguish between dialogue segments from these scenarios based on prosodic information, and turn-taking (attaining a classification error rate of 14.8%).…”

“…Here, unlike Szekrényes and Kovács [8] we did not cut dialogue segments where the interviewer and the interviewee discuss one topic on their agenda into smaller, overlapping segments, but used each dialogue segment as a training, development or test instance. This means that we had 1410 examples altogether.…”

“…We also tried to combine the posterior probability estimates obtained from the Deep Rectifier Neural Net (DRN) models trained by Szekrényes and Kovács [8] on features resulting from the ProsoTool [19] algorithm. For this combination, we also used the method of late fusion.…”

“…1 A detailed discussion of the meta-parameters of the DRN applied by Szekrényes and Kovács is beyond the scope of this paper. For more details, see [8].…”