Improved ASR for Under-resourced Languages through Multi-task Learning with Acoustic Landmarks

He, Di; Lim, Boon Pang; Yang, Xuesong; Hasegawa‐Johnson, Mark; Chen, Deming

doi:10.21437/interspeech.2018-1124

Cited by 5 publications

(6 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As our approach addresses both, a contrastive learning task and speech recognition task, this paper is related to the field of multi-task learning [23,24]. Recent approaches to multi-task learning [25,26] solve the tasks by minimizing a loss, containing multiple terms, on the same supervised datasets. Whereas, in our method, the unsupervised and supervised losses are minimized on their respective datasets.…”

Section: Related Workmentioning

confidence: 99%

Joint Masked CPC And CTC Training For ASR

Talnikar

Likhomanenko

Collobert

et al. 2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Self-supervised learning (SSL) has shown promise in learning representations of audio that are useful for automatic speech recognition (ASR). But, training SSL models like wav2vec 2.0 requires a two-stage pipeline. In this paper we demonstrate a single-stage training of ASR models that can utilize both unlabeled and labeled data. During training, we alternately minimize two losses: an unsupervised masked Contrastive Predictive Coding (CPC) loss and the supervised audio-to-text alignment loss Connectionist Temporal Classification (CTC). We show that this joint training method directly optimizes performance for the downstream ASR task using unsupervised data while achieving similar word error rates to wav2vec 2.0 on the Librispeech 100-hours dataset. Finally, we postulate that solving the contrastive task is a regularization for the supervised CTC loss.

show abstract

Section: Related Workmentioning

confidence: 99%

Joint Masked CPC And CTC Training For ASR

Talnikar

Likhomanenko

Collobert

et al. 2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

show abstract

“…Landmark-based ASR has been shown to slightly reduce the WER of a large-vocabulary speech recognizer, but only in a rescoring paradigm using a very small test set [18]. Landmarks can reduce computational load for DNN/HMM hybrid models [12,13] and can improve recognition accuracy [11]. Previous works [11,12,13,19] annotated landmark positions mostly following experimental findings presented in [20,21].…”

Section: Acoustic Landmarksmentioning

confidence: 99%

“…Landmarks can reduce computational load for DNN/HMM hybrid models [12,13] and can improve recognition accuracy [11]. Previous works [11,12,13,19] annotated landmark positions mostly following experimental findings presented in [20,21]. Four different landmarks are defined to capture positions of vowel peak, glide valley in glide-like consonants, oral closure and oral release.…”

Section: Acoustic Landmarksmentioning

confidence: 99%

“…Furthermore, another investigation [9] reveals that the spiky predictions of CTC models tend to overlap with the vicinity of acoustic landmarks where abrupt manner changes of articulation occur [10]. The possible coincidence of CTC peaks overlapping acoustic landmarks suggests a number of possible approaches for reducing the data requirements of CTC, including cross-language transfer (using the relative language-independence of acoustic landmarks [11]) and informative priors.…”

Section: Introductionmentioning

confidence: 99%

“…Many efforts have been attempted to augment acoustic modeling with acoustic landmarks [11,12,13] which are detected by accurate time-aligned phonetic transcriptions. To the best of our knowledge, only TIMIT [14] (5.4 hours) provides such fine-grained transcriptions.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

When CTC Training Meets Acoustic Landmarks

Yang

Lim³

et al. 2019

ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Self Cite

View full text Add to dashboard Cite

Connectionist temporal classification (CTC) provides an endto-end acoustic model (AM) training strategy. CTC learns accurate AMs without time-aligned phonetic transcription, but sometimes fails to converge, especially in resourceconstrained scenarios. In this paper, the convergence properties of CTC are improved by incorporating acoustic landmarks. We tailored a new set of acoustic landmarks to help CTC training converge more rapidly and smoothly while also reducing recognition error rates. We leveraged new target label sequences mixed with both phone and manner changes to guide CTC training. Experiments on TIMIT demonstrated that CTC based acoustic models converge significantly faster and smoother when they are augmented by acoustic landmarks. The models pretrained with mixed target labels can be further finetuned, resulting in phone error rates 8.72% below baseline on TIMIT. Consistent performance gain is also observed on WSJ (a larger corpus) and reduced TIMIT (smaller). With WSJ, we are the first to succeed in verifying the effectiveness of acoustic landmark theory on a mid-sized ASR task.

show abstract

Analytical Review of Methods for Solving Data Scarcity Issues Regarding Elaboration of Automatic Speech Recognition Systems for Low-Resource Languages

Kipyatkova¹,

Kagirov²

2022

View full text Add to dashboard Cite

In this paper, principal methods for solving training data issues for the so-called low-resource languages are discussed, regarding elaboration of automatic speech recognition systems. The notion of low-resource languages is studied and a working definition is coined on the basis of a number of papers on this topic. The main difficulties associated with the application of classical approaches to automatic speech recognition to the material of low-resource languages are determined, and the principal methods used to solve these problems are outlined. The paper discusses the methods for data augmentation, transfer learning and collection of new language data in detail. Depending on specific tasks, methods for audio material and text data augmentation, transfer learning and multi-task learning are distinguished. In Section 4 of the paper the current information support methods, databases and the basic principles of their architecture are discussed with regard to low-resource languages. Conclusions are drawn about the justification of augmentation and knowledge transfer methods for languages with low information support. In the case of unavailability of language data or structurally similar parent models, the preferred option is to collect a new database, including the crowdsourcing technique. Multilanguage learning models are effective for small datasets. If big language data are available, the most efficient method is transfer learning within a language pair. The conclusions made in the course of this this review will be applied to the data of the low-resource Karelian language, for which an automatic speech recognition system has been being created by the authors of this paper since the beginning of the year 2022.

show abstract

Improved ASR for Under-resourced Languages through Multi-task Learning with Acoustic Landmarks

Cited by 5 publications

References 17 publications

Joint Masked CPC And CTC Training For ASR

Joint Masked CPC And CTC Training For ASR

When CTC Training Meets Acoustic Landmarks

Analytical Review of Methods for Solving Data Scarcity Issues Regarding Elaboration of Automatic Speech Recognition Systems for Low-Resource Languages

Contact Info

Product

Resources

About