Audio Embedding-Aware Dialogue Policy Learning

Abstract

Following the success of Natural Language Processing (NLP) transformers pretrained via self-supervised learning, similar models have been proposed recently for speech processing such as Wav2Vec2, HuBERT and UniSpeech-SAT. An interesting yet unexplored area of application of these models is Spoken Dialogue Systems, where the users' audio signals are typically just mapped to word-level features derived from an Automatic Speech Recogniser (ASR), and then processed using NLP techniques to generate system responses. This paper reports a comprehensive comparison of dialogue policies trained using ASR-based transcriptions and extended with the aforementioned audio processing transformers in the DSTC2 task. Whilst our dialogue policies are trained with supervised and policy-based deep reinforcement learning, they are assessed using both automatic task completion metrics and a human evaluation. Our results reveal that using audio embeddings is more beneficial than detrimental in most of our trained dialogue policies, and that the benefits are stronger for supervised learning than reinforcement learning.

Dialogue Management and Language Generation for a Robust Conversational Virtual Coach: Validation and User Study

“…However, the amount of data needed to build such models and the little or no control of their behaviour once they have been built make them infeasible to implement for many practical applications. Even though deep-learning-based chatbots can be built with rather a few amount of domain-specific data for goal-oriented tasks [ 17 , 18 , 19 ], the problem of the lack of control of the model still remains. This often causes a lack of robustness which can hardly be avoided, and that is the reason why health-related data-driven DMs are extremely scarce in the literature.…”

A survey on deep reinforcement learning for audio-based applications

“…These models bring a new dimension of understanding and contextuality to conversations, not only in text but also in audio interactions, opening doors to even more sophisticated and dynamic interactions between humans and machines. For example, several recent studies have explored the utilisation of Large Audio Models in SDSs for task-oriented dialogue policy learning [212] and also for non-task-oriented dialogue including SpeechGPT [91], SoundStorm [114], AudioGPT [143], and dGSLM [213]. Other recent studies such as ANGIE [214], Multimodal-GPT [215], and Large Multimodal Models [216] have integrated vision and LLMs for training multimodal dialogue systems-which will be potentially transferable efforts to LLM-based robot dialogue systems.…”