Sequence-discriminative training of deep neural networks

Veselý, Karel; Ghoshal, Arnab; Burget, Lukáš; Povey, Daniel

doi:10.21437/interspeech.2013-548

Cited by 413 publications

(161 citation statements)

References 22 publications

Supporting

Mentioning

157

Contrasting

Order By: Relevance

“…The REINFORCE framework [24] enables model training where the probabilities of hypotheses are boosted if they perform well on arbitrary chosen metrics. In [25], sequence-discriminative criteria such as minimum word, phone error or minimum Bayes risk is used for ASR. Similar to mWER training for ASR, [22] showed that training with non-differentiable semantic criteria directly optimized SLU metrics.…”

Section: Transcriptmentioning

confidence: 99%

On joint training with interfaces for spoken language understanding

Raju¹,

Mv²,

Tiwari³

et al. 2021

Preprint

View full text Add to dashboard Cite

We propose an end-to-end trained spoken language understanding (SLU) system that extracts transcripts, intents and slots from an input speech utterance. It consists of a streaming recurrent neural network transducer (RNNT) based automatic speech recognition (ASR) model connected to a neural natural language understanding (NLU) model through a neural interface. This interface allows for end-toend training using multi-task RNNT and NLU losses. Additionally, we introduce semantic sequence loss training for the joint RNNT-NLU system that allows direct optimization of non-differentiable SLU metrics. This end-to-end SLU model paradigm can leverage state-of-the-art advancements and pretrained models in both ASR and NLU research communities, outperforming recently proposed direct speech-to-semantics models, and conventional pipelined ASR and NLU systems. We show that this method improves both ASR and NLU metrics on both public SLU datasets and large proprietary datasets.

show abstract

Section: Transcriptmentioning

confidence: 99%

On joint training with interfaces for spoken language understanding

Raju¹,

Mv²,

Tiwari³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…For all our experiments, we use 2 microphones from the opposite sides of the microphone array. All the models are trained with the cross-entropy objective followed by the sMBR objective function [16]. Our test data consists of 33,000 far-field speech utterances, and our development set consists of 17,000 similar utterances.…”

Section: System Descriptionmentioning

confidence: 99%

Multi-channel Acoustic Modeling using Mixed Bitrate OPUS Compression

Khare¹,

Sundaram²,

Wu³

2020

Preprint

View full text Add to dashboard Cite

Recent literature has shown that a learned front end with multi-channel audio input can outperform traditional beamforming algorithms for automatic speech recognition (ASR). In this paper, we present our study on multi-channel acoustic modeling using OPUS compression with different bitrates for the different channels. We analyze the degradation in word error rate (WER) as a function of the audio encoding bitrate and show that the WER degrades by 12.6% relative with 16kpbs as compared to uncompressed audio. We show that its always preferable to have a multi-channel audio input over a single channel audio input given limited bandwidth. Our results show that for the best WER, when one of the two channels can be encoded with a bitrate higher than 32kbps, its optimal to encode the other channel with the highest bitrate possible. For bitrates lower than that, its preferable to distribute the bitrate equally between the two channels. We further show that by training the acoustic model on mixed bitrate input, upto 50% of the degradation can be recovered using a single model.

show abstract

“…Next, the features were subsampled by factor 3 and NNs were trained with Lattice Free MMI (LF-MMI) objective and bi-phone targets as suggested in [9]. Finally, the NN are further trained with sequence Minimum Bayes Risk sMBR objective [10].…”

Section: Hybrid Acoustics Modelsmentioning

confidence: 99%

BUT Opensat 2019 Speech Recognition System

Karafiát¹,

Baskar²,

Szöke³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

The paper describes the BUT Automatic Speech Recognition (ASR) systems submitted for OpenSAT evaluations under two domain categories such as low resourced languages and public safety communications. The first was challenging due to lack of training data, therefore various architectures and multilingual approaches were employed. The combination led to superior performance. The second domain was challenging due to recording in extreme conditions such as specific channel, speaker under stress and high levels of noise. Data augmentation process was inevitable to get reasonably good performance.

show abstract

Sequence-discriminative training of deep neural networks

Cited by 413 publications

References 22 publications

On joint training with interfaces for spoken language understanding

On joint training with interfaces for spoken language understanding

Multi-channel Acoustic Modeling using Mixed Bitrate OPUS Compression

BUT Opensat 2019 Speech Recognition System

Contact Info

Product

Resources

About