Semi-supervised and unsupervised discriminative language model training for automatic speech recognition

Dikici, Erinç; Saraçlar, Murat

doi:10.1016/j.specom.2016.07.004

Cited by 12 publications

(7 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…our claim of rapid adaptability of the system to varying mismatched acoustic and linguistic conditions. The extreme mismatched conditions involved in our experiments supports the possibility of going one step further and training our system on artificially generated data of noisy transformations of phrases as in [35,36,38,57–59]. Thus possibly eliminating the need for an ASR for training purposes.…”

Section: Resultsmentioning

confidence: 58%

Learning from past mistakes: improving automatic speech recognition output via noisy-clean phrase context modeling

Shivakumar

Knight

et al. 2019

SIP

View full text Add to dashboard Cite

Automatic speech recognition (ASR) systems often make unrecoverable errors due to subsystem pruning (acoustic, language and pronunciation models); for example pruning words due to acoustics using short-term context, prior to rescoring with long-term context based on linguistics. In this work we model ASR as a phrase-based noisy transformation channel and propose an error correction system that can learn from the aggregate errors of all the independent modules constituting the ASR and attempt to invert those. The proposed system can exploit long-term context using a neural network language model and can better choose between existing ASR output possibilities as well as re-introduce previously pruned or unseen (out-of-vocabulary) phrases. It provides corrections under poorly performing ASR conditions without degrading any accurate transcriptions; such corrections are greater on top of out-of-domain and mismatched data ASR. Our system consistently provides improvements over the baseline ASR, even when baseline is further optimized through recurrent neural network language model rescoring. This demonstrates that any ASR improvements can be exploited independently and that our proposed system can potentially still provide benefits on highly optimized ASR. Finally, we present an extensive analysis of the type of errors corrected by our system.

show abstract

Section: Resultsmentioning

confidence: 58%

Learning from past mistakes: improving automatic speech recognition output via noisy-clean phrase context modeling

Shivakumar

Knight

et al. 2019

SIP

View full text Add to dashboard Cite

show abstract

“…It should be noticed that the lack of textual corpus is also a major challenge when training language models. To address this problem, various of methods have been carried out to expand corpus in the past decade [28,29,30]. In addition, textual level transfer learning strategy by merging a pre-trained representation to the decoder is also explored.…”

Section: Introductionmentioning

confidence: 99%

Large-Scale Transfer Learning for Low-Resource Spoken Language Understanding

Jia¹,

Wang²,

Zhang³

et al. 2020

Interspeech 2020

View full text Add to dashboard Cite

End-to-end Spoken Language Understanding (SLU) models are made increasingly large and complex to achieve the state-ofthe-art accuracy. However, the increased complexity of a model can also introduce high risk of over-fitting, which is a major challenge in SLU tasks due to the limitation of available data. In this paper, we propose an attention-based SLU model together with three encoder enhancement strategies to overcome data sparsity challenge. The first strategy focuses on the transferlearning approach to improve feature extraction capability of the encoder. It is implemented by pre-training the encoder component with a quantity of Automatic Speech Recognition annotated data relying on the standard Transformer architecture and then fine-tuning the SLU model with a small amount of target labelled data. The second strategy adopts multitask learning strategy, the SLU model integrates the speech recognition model by sharing the same underlying encoder, such that improving robustness and generalization ability. The third strategy, learning from Component Fusion (CF) idea, involves a Bidirectional Encoder Representation from Transformer (BERT) model and aims to boost the capability of the decoder with an auxiliary network. It hence reduces the risk of over-fitting and augments the ability of the underlying encoder, indirectly. Experiments on the FluentAI dataset show that cross-language transfer learning and multi-task strategies have been improved by up to 4.52% and 3.89% respectively, compared to the baseline.

show abstract

“…The two steps here are conducted iteratively: (a) a Criticizing Language Model (CLM) is trained to evaluate the quality score given a text sequence, and (b) and ASR model is trained to minimize the sequence loss calculated with ground truth while maximizing the scores given by CLM. 16,17] have been developed to address such problem by involving unpaired text data (which are relatively easy to obtain) in the training progress. One approach is to utilize unpaired text data to produce a separately trained language model (LM) to rescore the output of the end-to-end approach [18,13,19,20], but at the price of extra computation during testing.…”

Section: Introductionmentioning

confidence: 99%

Adversarial Training of End-to-end Speech Recognition Using a Criticizing Language Model

Liu

Lee

2019

ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

In this paper we proposed a novel Adversarial Training (AT) approach for end-to-end speech recognition using a Criticizing Language Model (CLM). In this way the CLM and the automatic speech recognition (ASR) model can challenge and learn from each other iteratively to improve the performance. Since the CLM only takes the text as input, huge quantities of unpaired text data can be utilized in this approach within end-to-end training. Moreover, AT can be applied to any end-to-end ASR model using any deep-learning-based language modeling frameworks, and compatible with any existing end-to-end decoding method. Initial results with an example experimental setup demonstrated the proposed approach is able to gain consistent improvements efficiently from auxiliary text data under different scenarios.

show abstract

Semi-supervised and unsupervised discriminative language model training for automatic speech recognition

Cited by 12 publications

References 13 publications

Learning from past mistakes: improving automatic speech recognition output via noisy-clean phrase context modeling

Learning from past mistakes: improving automatic speech recognition output via noisy-clean phrase context modeling

Large-Scale Transfer Learning for Low-Resource Spoken Language Understanding

Adversarial Training of End-to-end Speech Recognition Using a Criticizing Language Model

Contact Info

Product

Resources

About