Dialogue History Integration into End-to-End Signal-to-Concept Spoken Language Understanding Systems

Tomashenko, Natalia; Raymond, Christian; Caubrière, Antoine; Mori, Renato De; Estève, Yannick

doi:10.1109/icassp40776.2020.9053247

Cited by 11 publications

(10 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…1. This use of side information is similar to past approaches where speaker identity has been provided as i-vector embeddings [30] for ASR systems or h-vector embeddings for dialog history in SLU systems [25]. We conduct experiments on two SLU tasks, dialog action prediction and intent recognition, to demonstrate the usefulness of our proposed method in integrating dialog history for E2E SLU systems.…”

Section: Introductionmentioning

confidence: 92%

See 1 more Smart Citation

Integrating Dialog History into End-to-End Spoken Language Understanding Systems

Ganhotra¹,

Thomas²,

Kuo³

et al. 2021

Preprint

View full text Add to dashboard Cite

End-to-end spoken language understanding (SLU) systems that process human-human or human-computer interactions are often context independent and process each turn of a conversation independently. Spoken conversations on the other hand, are very much context dependent, and dialog history contains useful information that can improve the processing of each conversational turn. In this paper, we investigate the importance of dialog history and how it can be effectively integrated into endto-end SLU systems. While processing a spoken utterance, our proposed RNN transducer (RNN-T) based SLU model has access to its dialog history in the form of decoded transcripts and SLU labels of previous turns. We encode the dialog history as BERT embeddings, and use them as an additional input to the SLU model along with the speech features for the current utterance. We evaluate our approach on a recently released spoken dialog data set, the HARPERVALLEYBANK corpus. We observe significant improvements: 8% for dialog action and 30% for caller intent recognition tasks, in comparison to a competitive context independent end-to-end baseline system.

show abstract

Section: Introductionmentioning

confidence: 92%

“…Recently, in [25] the benefits of integrating dialog history into a speech based E2E SLU system have been investigated. The authors explore different representations, both supervised and unsupervised, to generate dialog history embedding vectors, and they observe significant improvements on the semantic slot filling task.…”

Section: Introductionmentioning

confidence: 99%

Integrating Dialog History into End-to-End Spoken Language Understanding Systems

Ganhotra¹,

Thomas²,

Kuo³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Result examples and comparison using SLU in negotiation dialogues are reported in [5]. Examples of attempts to integrate ASR and SLU functions in end-to-end (E2E) compact trainable DNN architectures are proposed in [8,9,10,11,12,13].…”

Section: Related Workmentioning

confidence: 99%

“…the slot fillers, needed for the evaluation in CVER, are obtained with a set of manually designed regular expressions in AllWords-C and SupWords-C configurations. Expressions are applied to the outputs of each concept support, as in [12,23,30,5]. Using the NormValues-C outputs, these handmade rules are not necessary anymore.…”

Section: Evaluation Protocolmentioning

confidence: 99%

End2End Acoustic to Semantic Transduction

Pelloin

Camelin

Laurent

et al. 2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Self Cite

View full text Add to dashboard Cite

In this paper, we propose a novel end-to-end sequence-tosequence spoken language understanding model using an attention mechanism. It reliably selects contextual acoustic features in order to hypothesize semantic contents. An initial architecture capable of extracting all pronounced words and concepts from acoustic spans is designed and tested. With a shallow fusion language model, this system reaches a 13.6 concept error rate (CER) and an 18.5 concept value error rate (CVER) on the French MEDIA corpus, achieving an absolute 2.8 points reduction compared to the state-of-the-art. Then, an original model is proposed for hypothesizing concepts and their values. This transduction reaches a 15.4 CER and a 21.6 CVER without any new type of context.

show abstract

“…18,19,20] directly computes a serialization of the semantics without intermediate text output. Another common approach uses transfer learning of pretrained ASR models to SLU tasks by replacing the final layer.…”

mentioning

confidence: 99%

DO as I Mean, Not as I Say: Sequence Loss Training for Spoken Language Understanding

Rao

Dheram

Tiwari

et al. 2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Spoken language understanding (SLU) systems extract transcriptions, as well as semantics of intent or named entities from speech, and are essential components of voice activated systems. SLU models, which either directly extract semantics from audio or are composed of pipelined automatic speech recognition (ASR) and natural language understanding (NLU) models, are typically trained via differentiable cross-entropy losses, even when the relevant performance metrics of interest are word or semantic error rates. In this work, we propose non-differentiable sequence losses based on SLU metrics as a proxy for semantic error and use the REINFORCE trick to train ASR and SLU models with this loss. We show that custom sequence loss training is the state-of-the-art on open SLU datasets and leads to 6% relative improvement in both ASR and NLU performance metrics on large proprietary datasets. We also demonstrate how the semantic sequence loss training paradigm can be used to update ASR and SLU models without transcripts, using semantic feedback alone.

show abstract

Dialogue History Integration into End-to-End Signal-to-Concept Spoken Language Understanding Systems

Cited by 11 publications

References 36 publications

Integrating Dialog History into End-to-End Spoken Language Understanding Systems

Integrating Dialog History into End-to-End Spoken Language Understanding Systems

End2End Acoustic to Semantic Transduction

DO as I Mean, Not as I Say: Sequence Loss Training for Spoken Language Understanding

Contact Info

Product

Resources

About