Intelligibility Improvement of Dysarthric Speech using MMSE DiscoGAN

Purohit, Mirali; Patel, Maitreya; Malaviya, Harshit; Patil, Ankur T.; Parmar, Mihir; Shah, Nirmesh J.; Doshi, Savan; Patil, Hemant A.

doi:10.1109/spcom50965.2020.9179511

Cited by 12 publications

(14 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our results cannot be directly compared to those reported in [10] and their state-of-the-art DiscoGAN model because of the differences in the denoising of the dysarthric speech that was applied in our work but not in theirs (see Section 2.2).…”

Section: Comparison Of the Results To State-of-the-art And Disco-ganmentioning

confidence: 86%

“…The PER results are shown for individual speakers separately and averaged over all speakers and blocked by model type (see the table caption for an explanation). Note that empty cells refer to results that were not given or specified by [10].…”

Section: Resultsmentioning

confidence: 99%

“…Frameworks allowing non-parallel training thus have higher practical use. It is therefore not surprising that non-parallel CycleGAN-based approaches recently attracted some attention for DVC [10]. However, it is unclear what variant of CycleGAN-VC is the most ideal for DVC as [11] found the DiscoGAN and CycleGAN-VC architectures comparable on objective metrics, while Mask-CycleGAN-VC [12] seems to be better than CycleGAN-VC.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

The Effectiveness of Time Stretching for Enhancing Dysarthric Speech for Improved Dysarthric Speech Recognition

Prananta¹,

Halpern²,

Feng³

et al. 2022

Preprint

View full text Add to dashboard Cite

In this paper, we investigate several existing and a new stateof-the-art generative adversarial network-based (GAN) voice conversion method for enhancing dysarthric speech for improved dysarthric speech recognition. We compare key components of existing methods as part of a rigorous ablation study to find the most effective solution to improve dysarthric speech recognition. We find that straightforward signal processing methods such as stationary noise removal and vocoder-based time stretching lead to dysarthric speech recognition results comparable to those obtained when using state-of-the-art GANbased voice conversion methods as measured using a phoneme recognition task. Additionally, our proposed solution of a combination of MaskCycleGAN-VC and time stretched enhancement is able to improve the phoneme recognition results for certain dysarthric speakers compared to our time stretched baseline.

show abstract

Section: Comparison Of the Results To State-of-the-art And Disco-ganmentioning

confidence: 86%

Section: Resultsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

The Effectiveness of Time Stretching for Enhancing Dysarthric Speech for Improved Dysarthric Speech Recognition

Prananta¹,

Halpern²,

Feng³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…The phoneme error rate (PER) calculated with a phoneme recognizer evaluates the intelligibility, which is also related to severity and naturalness (RQ1 and RQ3). We use a pre-trained Kaldi ASR model with the same specifications as the one used in [22] for phoneme recognition. The ASR was trained with the TIMIT dataset and used an HMM acoustic model.…”

Section: Phoneme Error Ratementioning

confidence: 99%

Towards Identity Preserving Normal to Dysarthric Voice Conversion

Huang¹,

Halpern²,

Violeta³

et al. 2021

Preprint

View full text Add to dashboard Cite

We present a voice conversion framework that converts normal speech into dysarthric speech while preserving the speaker identity. Such a framework is essential for (1) clinical decision making processes and alleviation of patient stress, (2) data augmentation for dysarthric speech recognition. This is an especially challenging task since the converted samples should capture the severity of dysarthric speech while being highly natural and possessing the speaker identity of the normal speaker. To this end, we adopted a two-stage framework, which consists of a sequence-to-sequence model and a nonparallel frame-wise model. Objective and subjective evaluations were conducted on the UASpeech dataset, and results showed that the method was able to yield reasonable naturalness and capture severity aspects of the pathological speech. On the other hand, the similarity to the normal source speaker's voice was limited and requires further improvements.

show abstract

“…Rule-based transformation based on signal processing [3] is limited in that each patient needs to be individually considered. Statistical approaches adopt models ranging from Gaussian mixture models [4], exemplar-based methods [5,6] and deep neural networks [7,8,9].…”

Section: Introductionmentioning

confidence: 99%

A Preliminary Study of a Two-Stage Paradigm for Preserving Speaker Identity in Dysarthric Voice Conversion

Huang¹,

Kobayashi²,

Peng³

et al. 2021

Interspeech 2021

View full text Add to dashboard Cite

We propose a new paradigm for maintaining speaker identity in dysarthric voice conversion (DVC). The poor quality of dysarthric speech can be greatly improved by statistical VC, but as the normal speech utterances of a dysarthria patient are nearly impossible to collect, previous work failed to recover the individuality of the patient. In light of this, we suggest a novel, two-stage approach for DVC, which is highly flexible in that no normal speech of the patient is required. First, a powerful parallel sequence-to-sequence model converts the input dysarthric speech into a normal speech of a reference speaker as an intermediate product, and a nonparallel, frame-wise VC model realized with a variational autoencoder then converts the speaker identity of the reference speech back to that of the patient while assumed to be capable of preserving the enhanced quality. We investigate several design options. Experimental evaluation results demonstrate the potential of our approach to improving the quality of the dysarthric speech while maintaining the speaker identity.

show abstract

Intelligibility Improvement of Dysarthric Speech using MMSE DiscoGAN

Cited by 12 publications

References 21 publications

The Effectiveness of Time Stretching for Enhancing Dysarthric Speech for Improved Dysarthric Speech Recognition

The Effectiveness of Time Stretching for Enhancing Dysarthric Speech for Improved Dysarthric Speech Recognition

Towards Identity Preserving Normal to Dysarthric Voice Conversion

A Preliminary Study of a Two-Stage Paradigm for Preserving Speaker Identity in Dysarthric Voice Conversion

Contact Info

Product

Resources

About