Context-aware Neural-based Dialog Act Classification on Automatically Generated Transcriptions

Ortega, Daniel; Li, Chia‐Yu; Vallejo, Gisela; Denisov, Pavel; Vu, Ngoc Thang

doi:10.1109/icassp.2019.8682881

Cited by 12 publications

(10 citation statements)

References 21 publications

(37 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…[96] integrate the whole spoken dialogue history using a variety of sentence embeddings for a semantic slot filling task. [76] present a dialog act classification system on automatically generated transcriptions that combines convolutional neural networks and conditional random fields for context modeling. [61] also perform a dialogue act classification via a hierarchical deep learning model that takes into account the dialogue context.…”

Section: Control Mechanisms To Strengthen the Long-term Coherence Of ...mentioning

confidence: 99%

A Multilingual Neural Coaching Model with Enhanced Long-term Dialogue Structure

Zorrilla

Torres

2022

ACM Trans. Interact. Intell. Syst.

View full text Add to dashboard Cite

In this work we develop a fully data driven conversational agent capable of carrying out motivational coaching sessions in Spanish, French, Norwegian and English. Unlike the majority of coaching, and in general, well-being related conversational agents that can be found in the literature, ours is not designed by hand-crafted rules. Instead, we directly model the coaching strategy of professionals with end users. To this end, we gather a set of virtual coaching sessions through a Wizard of Oz platform, and apply state of the art Natural Language Processing techniques. We employ a transfer learning approach, pretraining GPT2 neural language models and fine-tuning them on our corpus. However, since these only take as input a local dialogue history, a simple fine-tuning procedure is not capable of modeling the long-term dialogue strategies that appear in coaching sessions. To alleviate this issue, we first propose to learn dialogue phase and scenario embeddings in the fine-tuning stage. These indicate to the model at which part of the dialogue it is and which kind of coaching session it is carrying out. Second, we develop global deep learning system which controls the long-term structure of the dialogue. We also show that this global module can be used to visualize and interpret the decisions taken by the the conversational agent, and that the learnt representations are comparable to dialogue acts. Automatic and human evaluation show that our proposals serve to improve the baseline models. Finally, interaction experiments with coaching experts indicate that system is usable and gives raise to positive emotions in Spanish, French and English, while the results in Norwegian point out that there is still work to be done in fully data driven approaches with very low resource languages.

show abstract

Section: Control Mechanisms To Strengthen the Long-term Coherence Of ...mentioning

confidence: 99%

A Multilingual Neural Coaching Model with Enhanced Long-term Dialogue Structure

Zorrilla

Torres

2022

ACM Trans. Interact. Intell. Syst.

View full text Add to dashboard Cite

show abstract

“…Earlier studies like Reithinger and Klesen [36] and Grau et al [13] focused on lexical, syntactical, and prosodic features for classification. In another work, Ortega et al [29] employed CNNs [18] and CRFs [17]. Lee and Dernoncourt [19] proposed a method based on CNNs and RNNs [37] that used the previous contextual utterances to predict the dialogue-act of the current utterance.…”

Section: Related Workmentioning

confidence: 99%

Speaker and Time-aware Joint Contextual Learning for Dialogue-act Classification in Counselling Conversations

Malhotra¹,

Waheed²,

Srivastava³

et al. 2021

Preprint

View full text Add to dashboard Cite

The onset of the COVID-19 pandemic has brought the mental health of people under risk. Social counselling has gained remarkable significance in this environment. Unlike general goal-oriented dialogues, a conversation between a patient and a therapist is considerably implicit, though the objective of the conversation is quite apparent. In such a case, understanding the intent of the patient is imperative in providing effective counselling in therapy sessions, and the same applies to a dialogue system as well. In this work, we take forward a small but an important step in the development of an automated dialogue system for mental-health counselling. We develop a novel dataset, named HOPE, to provide a platform for the dialogue-act classification in counselling conversations. We identify the requirement of such conversation and propose twelve domainspecific dialogue-act (DAC) labels. We collect ∼ 12.9𝐾 utterances from publicly-available counselling session videos on YouTube, extract their transcripts, clean, and annotate them with DAC labels. Further, we propose SPARTA, a transformer-based architecture with a novel speaker-and time-aware contextual learning for the dialogue-act classification. Our evaluation shows convincing performance over several baselines, achieving state-of-the-art on HOPE. We also supplement our experiments with extensive empirical and qualitative analyses of SPARTA.

show abstract

“…Considerable works have been done on classical Machine Learning (ML) based DAC , (Stolcke et al, 2000), (Verbree et al, 2006), etc. and Deep Learning (DL) based DAC (Kalchbrenner and Blunsom, 2013), (Papalampidi et al, 2017), (Liu et al, 2017), (Ribeiro et al, 2019), (Ortega et al, 2019), (Saha et al, 2019) etc.…”

Section: Introductionmentioning

confidence: 99%

Towards Emotion-aided Multi-modal Dialogue Act Classification

Saha¹,

Patra²,

Saha³

et al. 2020

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics

View full text Add to dashboard Cite

The task of Dialogue Act Classification (DAC) that purports to capture communicative intent has been studied extensively. But these studies limit themselves to text. Non-verbal features (change of tone, facial expressions etc.) can provide cues to identify DAs, thus stressing the benefit of incorporating multi-modal inputs in the task. Also, the emotional state of the speaker has a substantial effect on the choice of the dialogue act, since conversations are often influenced by emotions. Hence, the effect of emotion too on automatic identification of DAs needs to be studied. In this work, we address the role of both multi-modality and emotion recognition (ER) in DAC. DAC and ER help each other by way of multi-task learning. One of the major contributions of this work is a new dataset-multimodal Emotion aware Dialogue Act dataset called EMOTyDA, collected from open-sourced dialogue datasets. To demonstrate the utility of EMOTyDA, we build an attention based (self, inter-modal, inter-task) multi-modal, multi-task Deep Neural Network (DNN) for joint learning of DAs and emotions. We show empirically that multimodality and multi-tasking achieve better performance of DAC compared to uni-modal and single task DAC variants.

show abstract

Context-aware Neural-based Dialog Act Classification on Automatically Generated Transcriptions

Cited by 12 publications

References 21 publications

A Multilingual Neural Coaching Model with Enhanced Long-term Dialogue Structure

A Multilingual Neural Coaching Model with Enhanced Long-term Dialogue Structure

Speaker and Time-aware Joint Contextual Learning for Dialogue-act Classification in Counselling Conversations

Towards Emotion-aided Multi-modal Dialogue Act Classification

Contact Info

Product

Resources

About