Speaker adaptation for Wav2vec2 based dysarthric ASR

Baskar, Murali Karthick; Herzig, Tim; Nguyen, Diana; Díez, Mireia; Polzehl, Tim; Burget, Lukáš; Černocký, Jan "Honza''

doi:10.48550/arxiv.2204.00770

Cited by 1 publication

(1 citation statement)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Different studies have focused on the possible utilization of self-supervised learning [21] (SSL) approaches in the presence of atypical speech to address the main challenge of speech data scarcity [22][23][24][25]. For example, Wang et al [26] explored the advantages of pre-trained mono and cross-lingual speech representations for the spoken language understanding of Dutch dysarthric speech, and Hu et al [27] investigated a series of approaches to integrate domain-adapted SSL pre-trained models into time delay neural networks and conformer ASR systems for elderly and impaired speech recognition by working on the UA-Speech and the Dementia Pitt corpora.…”

Section: Related Workmentioning

confidence: 99%

A Voice User Interface on the Edge for People with Speech Impairments

Mulfari,

Villari

2024

Electronics

View full text Add to dashboard Cite

Nowadays, fine-tuning has emerged as a powerful technique in machine learning, enabling models to adapt to a specific domain by leveraging pre-trained knowledge. One such application domain is automatic speech recognition (ASR), where fine-tuning plays a crucial role in addressing data scarcity, especially for languages with limited resources. In this study, we applied fine-tuning in the context of atypical speech recognition, focusing on Italian speakers with speech impairments, e.g., dysarthria. Our objective was to build a speaker-dependent voice user interface (VUI) tailored to their unique needs. To achieve this, we harnessed a pre-trained OpenAI’s Whisper model, which has been exposed to vast amounts of general speech data. However, to adapt it specifically for disordered speech, we fine-tuned it using our private corpus including 65 K voice recordings contributed by 208 speech-impaired individuals globally. We exploited three variants of the Whisper model (small, base, tiny), and by evaluating their relative performance, we aimed to identify the most accurate configuration for handling disordered speech patterns. Furthermore, our study dealt with the local deployment of the trained models on edge computing nodes, with the aim to realize custom VUIs for persons with impaired speech.

show abstract