From Speech Signals to Semantics — Tagging Performance at Acoustic, Phonetic and Word Levels

Ye, Qian; Ubale, Rutuja; Lange, Patrick; Evanini, Keelan; Soong, Frank K.

doi:10.1109/iscslp.2018.8706581

Cited by 6 publications

(2 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While end-to-end SLU is an active area of research, currently the most promising results under-perform or just barely outperform traditional cascaded systems [7,10]. One reason is that deep learning models require a large amount of appropriate training data.…”

Section: Introductionmentioning

confidence: 99%

Leveraging Unpaired Text Data for Training End-To-End Speech-to-Intent Systems

Huang

Kuo

Thomas

et al. 2020

ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Training an end-to-end (E2E) neural network speech-to-intent (S2I) system that directly extracts intents from speech requires large amounts of intent-labeled speech data, which is time consuming and expensive to collect. Initializing the S2I model with an ASR model trained on copious speech data can alleviate data sparsity. In this paper, we attempt to leverage NLU text resources. We implemented a CTC-based S2I system that matches the performance of a state-ofthe-art, traditional cascaded SLU system. We performed controlled experiments with varying amounts of speech and text training data. When only a tenth of the original data is available, intent classification accuracy degrades by 7.6% absolute. Assuming we have additional text-to-intent data (without speech) available, we investigated two techniques to improve the S2I system: (1) transfer learning, in which acoustic embeddings for intent classification are tied to fine-tuned BERT text embeddings; and (2) data augmentation, in which the textto-intent data is converted into speech-to-intent data using a multispeaker text-to-speech system. The proposed approaches recover 80% of performance lost due to using limited intent-labeled speech.

show abstract

Section: Introductionmentioning

confidence: 99%

Leveraging Unpaired Text Data for Training End-To-End Speech-to-Intent Systems

Huang

Kuo

Thomas

et al. 2020

ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

show abstract

“…Similarly, several other end-to-end learning methods have been proposed for spoken language understanding (SLU) task [5,6,7,8,9,10,4,11,12,13,14] without requiring intermediate text and show promising results on multiple tasks. However, their success heavily depends on a large amount of labeled training data.…”

Section: Introductionmentioning

confidence: 99%

Representation Based Meta-Learning for Few-Shot Spoken Intent Recognition

Mittal¹,

Bharadwaj²,

Khare³

et al. 2020

Interspeech 2020

View full text Add to dashboard Cite

Spoken intent detection has become a popular approach to interface with various smart devices with ease. However, such systems are limited to the preset list of intents-terms or commands, which restricts the quick customization of personal devices to new intents. This paper presents a few-shot spoken intent classification approach with task-agnostic representations via meta-learning paradigm. Specifically, we leverage the popular representation based meta-learning learning to build a taskagnostic representation of utterances, that then use a linear classifier for prediction. We evaluate three such approaches on our novel experimental protocol developed on two popular spoken intent classification datasets: Google Commands and the Fluent Speech Commands dataset. For a 5-shot (1-shot) classification of novel classes, the proposed framework provides an average classification accuracy of 88.6% (76.3%) on the Google Commands dataset, and 78.5% (64.2%) on the Fluent Speech Commands dataset. The performance is comparable to traditionally supervised classification models with abundant training samples.

show abstract

Spoken Language Understanding of Human-Machine Conversations for Language Learning Applications

Ubale

Lange

et al. 2019

J Sign Process Syst

View full text Add to dashboard Cite

From Speech Signals to Semantics — Tagging Performance at Acoustic, Phonetic and Word Levels

Cited by 6 publications

References 20 publications

Leveraging Unpaired Text Data for Training End-To-End Speech-to-Intent Systems

Leveraging Unpaired Text Data for Training End-To-End Speech-to-Intent Systems

Representation Based Meta-Learning for Few-Shot Spoken Intent Recognition

Spoken Language Understanding of Human-Machine Conversations for Language Learning Applications

Contact Info

Product

Resources

About