Improving End-to-End Speech-to-Intent Classification with Reptile

Abstract

End-to-end spoken language understanding (SLU) systems have many advantages over conventional pipeline systems, but collecting in-domain speech data to train an end-to-end system is costly and time consuming. One question arises from this: how to train an end-to-end SLU with limited amounts of data? Many researchers have explored approaches that make use of other related data resources, typically by pre-training parts of the model on high-resource speech recognition. In this paper, we suggest improving the generalization performance of SLU models with a non-standard learning algorithm, Reptile. Though Reptile was originally proposed for model-agnostic meta learning, we argue that it can also be used to directly learn a target task and result in better generalization than conventional gradient descent. In this work, we employ Reptile to the task of end-to-end spoken intent classification. Experiments on four datasets of different languages and domains show improvement of intent prediction accuracy, both when Reptile is used alone and used in addition to pre-training.

One Shot Learning for Speech Separation

“…Meta-learning has been applied in computer vision research and achieved great success in image classification [5,6,7,8]. Meanwhile, several natural language and speech processing tasks also adopt meta-learning and attain promising results, such as neural machine translation [9], dialogue generation [10], text classification [11], word sense disambiguation [12], and so on, speaker adaptive training [13], speech-to-intent classification [14], code-switched speech recognition [15], and speech recognition [16].…”

End-to-End Spoken Language Understanding for Generalized Voice Assistants

“…Rather than containing discrete ASR and NLU modules, E2E SLU models are trained to infer the utterance semantics directly from the spoken signal [13][14][15][16][17][18][19][20]. These models are trained to maximize the SLU prediction accuracy where the predicted semantic targets vary from just the intent [21,22], to a full interpretation with domain, intents, and slots [13].…”

“…A similar collection of French spoken NER and slot filling datasets has been investigated [26]. Over the last year the state-of-the-art on FSC has progressed to over 99% test set accuracy for several E2E approaches [14][15][16][17][18][19][20]. However, there remains a gap between the capabilities demonstrated thus far and the E2E SLU requirements for a generalized VA [27].…”

RNN Transducer Models for Spoken Language Understanding

“…Recently there has been a significant effort to build end-to-end (E2E) models for spoken language understanding [1][2][3][4][5][6][7][8][9][10][11][12][13][14][15]. Instead of using an ASR system in tandem with a text-based natural language understanding system [2,16,17], these systems directly process speech to produce spoken language understanding (SLU) entity or intent label targets.…”