Machine translation based data augmentation for Cantonese keyword spotting

Huang, Guangpu; Gorin, Arseniy; Gauvain, Jean‐Luc; Lamel, Lori

doi:10.1109/icassp.2016.7472833

Cited by 11 publications

(2 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Fazel et al [21] use synthetic speech generated with T2S to improve accuracy in ASR. Huang et al [62] use a machine translation technique to generate text to train an ASR language model in a low-resource language. At the same time, Shah et al [20] and Huybrechts et al [19] employ S2S voice conversion to improve the quality of speech synthesis in the data reduction scenario.…”

Section: Synthetic Speech Generation For Pronunciation Error Detectionmentioning

confidence: 99%

Computer-assisted Pronunciation Training -- Speech synthesis is almost all you need

Korzekwa,

Lorenzo-Trueba,

Drugman

et al. 2022

Preprint

View full text Add to dashboard Cite

The research community has long studied computer-assisted pronunciation training (CAPT) methods in non-native speech. Researchers focused on studying various model architectures, such as Bayesian networks and deep learning methods, as well as on the analysis of different representations of the speech signal. Despite significant progress in recent years, existing CAPT methods are not able to detect pronunciation errors with high accuracy (only 60% precision at 40%-80% recall). One of the key problems is the low availability of mispronounced speech that is needed for the reliable training of pronunciation error detection models. If we had a generative model that could mimic non-native speech and produce any amount of training data, then the task of detecting pronunciation errors would be much easier. We present three innovative techniques based on phoneme-to-phoneme (P2P), text-to-speech (T2S), and speech-to-speech (S2S) conversion to generate correctly pronounced and mispronounced synthetic speech. We show that these techniques not only improve the accuracy of three machine learning models for detecting pronunciation errors but also help establish a new state-of-the-art in the field. Earlier studies have used simple speech generation techniques such as P2P conversion, but only as an additional mechanism to improve the accuracy of pronunciation error detection. We, on the other hand, consider speech generation to be the first-class method of detecting pronunciation errors. The effectiveness of these techniques is assessed in the tasks of detecting pronunciation and lexical stress errors. Non-native English speech corpora of German, Italian, and Polish speakers are used in the evaluations. The best proposed S2S technique improves the accuracy of detecting pronunciation errors in AUC metric by 41% from 0.528 to 0.749 compared to the state-of-the-art approach.

show abstract

Section: Synthetic Speech Generation For Pronunciation Error Detectionmentioning

confidence: 99%

Computer-assisted Pronunciation Training -- Speech synthesis is almost all you need

Korzekwa,

Lorenzo-Trueba,

Drugman

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Short texts (STs) refer to examples that contain fewer tokens within sequences. 1 Translating STs from one language to another plays a crucial part in natural language processing (NLP) scenarios, including the modeling of query (Huang et al 2016;Song, Kim, and Park 2017;Saleh and Pecina 2020;Bi et al 2020;Yao et al 2020aYao et al , 2020b, dialogue (Wang et al 2017;Liu et al 2018), title (Kreutzer et al 2018;Karakanta, Dehdari, and van Genabith 2018;Darwish and Sayaheen 2019;Etchegoyhen and Gete 2020;Banar, Daelemans, and Kestemont 2020), entity name (Jiang et al 2007;Zhao et al 2020), and text matching (Chen et al 2020;Lyu et al 2021).…”

Section: Introductionmentioning

confidence: 99%

Challenges of Neural Machine Translation for Short Texts

Wan

Yang

Wong

et al. 2022

Computational Linguistics

View full text Add to dashboard Cite

Short texts (STs) present in a variety of scenarios, including query, dialog, and entity names. Most of the exciting studies in neural machine translation (NMT) are wild about tackling open problems concerning long sentences rather than short ones. The intuition behind is that, with respect to human learning and processing, short sequences are generally regarded as easy examples. In this paper, we first dispel this speculation via conducting preliminary experiments, showing that the conventional state-of-the-art NMT approach, i.e. Transformer (Vaswani et al. 2017), still severely suffers from over-translation and mistranslation errors over STs. After empirically investigating the rationale behind, we summarized two challenges in NMT for STs associated with translation error types above, respectively: 1) the imbalanced length distribution in training set intensifies model inference calibration over STs, leading to more over-translation cases on STs; 2) the lack of contextual information makes NMT have higher data uncertainty on short sentences, and NMT model is troubled by considerable mistranslation errors. Some existing approaches, like balancing data distribution for training (e.g. data upsampling) and complementing contextual information (e.g. introducing translation memory) can pertinently alleviate the translation issues in NMT for STs. We are delighted to embrace researchers investigate other challenges in NMT for STs, thus reducing ST translation errors and enhancing translation quality.

show abstract

Cantonese natural language processing in the transformers era: a survey and current challenges

Xiang,

Chersoni,

et al. 2024

Lang Resources & Evaluation

View full text Add to dashboard Cite

Despite being spoken by a large population of speakers worldwide, Cantonese is under-resourced in terms of the data scale and diversity compared to other major languages. This limitation has excluded it from the current “pre-training and fine-tuning” paradigm that is dominated by Transformer architectures. In this paper, we provide a comprehensive review on the existing resources and methodologies for Cantonese Natural Language Processing, covering the recent progress in language understanding, text generation and development of language models. We finally discuss two aspects of the Cantonese language that could make it potentially challenging even for state-of-the-art architectures: colloquialism and multilinguality

show abstract

Machine translation based data augmentation for Cantonese keyword spotting

Cited by 11 publications

References 22 publications

Computer-assisted Pronunciation Training -- Speech synthesis is almost all you need

Computer-assisted Pronunciation Training -- Speech synthesis is almost all you need

Challenges of Neural Machine Translation for Short Texts

Cantonese natural language processing in the transformers era: a survey and current challenges

Contact Info

Product

Resources

About