Synthesizing Dysarthric Speech Using Multi-Speaker Tts For Dysarthric Speech Recognition

Soleymanpour, Mohammad; Johnson, Michael T.; Soleymanpour, Rahim; Berry, Jeffrey

doi:10.1109/icassp43922.2022.9746585

Cited by 18 publications

(16 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We claim that most teaching and speaking intentions or objectives, such as acquisition (Derwing, Munro, & Thomson, 2022) or reproduction of information. Mukhtoraliyevna and Madaminkhonqizi (2022) synthesizing (Soleymanpour, Johnson, Soleymanpour, & Berry, 2022) elaborating (Dong, Gui, Zhang, & Zhong, 2022) creating (Esmaili & Mohammadi Rakati, 2022) exploring (Limeranto & Bram, 2022). In addition to being insightful, transactional communication helps in recognizing and expanding our understanding of the implicit or indirect significance of what is said.…”

Section: Discussionmentioning

confidence: 99%

Teachers’ perspectives on teaching EFL speaking virtually: A case study of COVID-19 pandemic survival

Butarbutar

Leba

2023

Am J Soc Sci Hum

View full text Add to dashboard Cite

The goals of this study are to analyze and provide in-depth detail about how teachers felt and how they taught during the COVID-19 pandemic. The study used a qualitative approach via the Atlas.ti application and used a semi-structured interview to collect data. Five teachers were recruited to answer the research questions to determine what they thought, how they felt, and what they did during the pandemic. To improve the consistency of theme analysis coding (validity or trustworthiness), the study was adjusted based on feedback from external independent reviewers. The results reveal that teaching speaking has a direct effect on four things: (1) the teaching method, (2) the teaching tools, (3) the goals or aims, and (4) how well the students do. As a result, pupils perform better academically, cognitively, and socio-emotionally. This study has flaws, such as the fact that it was conducted in a remote area with few facilities and only used a small sample of teachers with good digital literacy.

show abstract

Section: Discussionmentioning

confidence: 99%

Teachers’ perspectives on teaching EFL speaking virtually: A case study of COVID-19 pandemic survival

Butarbutar

Leba

2023

Am J Soc Sci Hum

View full text Add to dashboard Cite

show abstract

“…In contrast, only a few prior researches on applying GAN to dysarthric speech processing tasks, represented by dysarthric speech augmentation [109]- [112] and dysarthric speech reconstruction [113], [114] have been conducted to date. To the best of our knowledge, no previous study on GAN based data augmentation approaches designed for elderly speech recognition tasks has been conducted.…”

Section: Corpusmentioning

confidence: 99%

Personalized Adversarial Data Augmentation for Dysarthric and Elderly Speech Recognition

Jin¹,

Deng²,

Wang³

et al. 2022

Preprint

View full text Add to dashboard Cite

Despite the rapid progress of automatic speech recognition (ASR) technologies targeting normal speech, accurate recognition of dysarthric and elderly speech remains highly challenging tasks to date. It is difficult to collect large quantities of such data for ASR system development due to the mobility issues often found among these users. To this end, data augmentation techniques play a vital role. In contrast to existing data augmentation techniques only modifying the speaking rate or overall shape of spectral contour, fine-grained spectro-temporal differences between dysarthric, elderly and normal speech are modelled using a novel set of speaker dependent (SD) generative adversarial networks (GAN) based data augmentation approaches in this paper. These flexibly allow both: a) temporal or speed perturbed normal speech spectra to be modified and closer to those of an impaired speaker when parallel speech data is available; and b) for non-parallel data, the SVD decomposed normal speech spectral basis features to be transformed into those of a target elderly speaker before being re-composed with the temporal bases to produce the augmented data for state-of-the-art TDNN and Conformer ASR system training. Experiments are conducted on four tasks: the English UASpeech and TORGO dysarthric speech corpora; the English DementiaBank Pitt and Cantonese JCCOCC MoCA elderly speech datasets. The proposed GAN based data augmentation approaches consistently outperform the baseline speed perturbation method by up to 0.91% and 3.0% absolute (9.61% and 6.4% relative) WER reduction on the TORGO and DementiaBank data respectively. Consistent performance improvements are retained after applying LHUC based speaker adaptation.

show abstract

“…In this paper we build upon previous work on TTS for dysarthric speech [10]. They introduced a dysarthria embedding for the FastSpeech 2 TTS system [11] that allows to explicitly model and generate speech of different severity levels.…”

Section: Introductionmentioning

confidence: 99%

Few-shot Dysarthric Speech Recognition with Text-to-Speech Data Augmentation

Hermann¹,

Magimai.-Doss²

2023

Interspeech 2023

View full text Add to dashboard Cite

Speakers with dysarthria could particularly benefit from assistive speech technology, but are underserved by current automatic speech recognition (ASR) systems. The differences of dysarthric speech pose challenges, while recording large amounts of training data can be exhausting for patients. In this paper, we synthesise dysarthric speech with a FastSpeech 2based multi-speaker text-to-speech (TTS) system for ASR data augmentation. We evaluate its few-shot capability by generating dysarthric speech with as few as 5 words from an unseen target speaker and then using it to train speaker-dependent ASR systems. The results indicated that, while the TTS output is not yet of sufficient quality, this could allow easy development of personalised acoustic models for new dysarthric speakers and domains in the future.

show abstract

Synthesizing Dysarthric Speech Using Multi-Speaker Tts For Dysarthric Speech Recognition

Cited by 18 publications

References 15 publications

Teachers’ perspectives on teaching EFL speaking virtually: A case study of COVID-19 pandemic survival

Teachers’ perspectives on teaching EFL speaking virtually: A case study of COVID-19 pandemic survival

Personalized Adversarial Data Augmentation for Dysarthric and Elderly Speech Recognition

Few-shot Dysarthric Speech Recognition with Text-to-Speech Data Augmentation

Contact Info

Product

Resources

About