Automatic Assessment of Speech Impairment in Cantonese-Speaking People with Aphasia

Qin, Ying; Lee, Tan; Kong, Anthony Pak‐Hin

doi:10.1109/jstsp.2019.2956371

Cited by 23 publications

(17 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, the biggest challenge in the field nowadays is to improve the performance of the continuous recognition of aphasic speech in large vocabularies. To the best of our knowledge, the published works in the task of aphasic continuous speech recognition of large vocabularies only consider English [23,38,39] and Cantonese [40] to date. In this sense, the performance and results for these systems widely oscillate depending on the severity level of aphasia, ranging WER from 33 on mildest cases to more than 60 on very severe cases.…”

Section: Related Work In Aphasic Speech Recognitionmentioning

confidence: 99%

Improving Aphasic Speech Recognition by Using Novel Semi-Supervised Learning Methods on AphasiaBank for English and Spanish

2021

View full text Add to dashboard Cite

Automatic speech recognition in patients with aphasia is a challenging task for which studies have been published in a few languages. Reasonably, the systems reported in the literature within this field show significantly lower performance than those focused on transcribing non-pathological clean speech. It is mainly due to the difficulty of recognizing a more unintelligible voice, as well as due to the scarcity of annotated aphasic data. This work is mainly focused on applying novel semi-supervised learning methods to the AphasiaBank dataset in order to deal with these two major issues, reporting improvements for the English language and providing the first benchmark for the Spanish language for which less than one hour of transcribed aphasic speech was used for training. In addition, the influence of reinforcing the training and decoding processes with out-of-domain acoustic and text data is described by using different strategies and configurations to fine-tune the hyperparameters and the final recognition systems. The interesting results obtained encourage extending this technological approach to other languages and scenarios where the scarcity of annotated data to train recognition models is a challenging reality.

show abstract

Section: Related Work In Aphasic Speech Recognitionmentioning

confidence: 99%

Improving Aphasic Speech Recognition by Using Novel Semi-Supervised Learning Methods on AphasiaBank for English and Spanish

2021

View full text Add to dashboard Cite

show abstract

“…GRU is a simplified architecture with an efficiency degree that is comparable to LSTM. These two approaches have been adopted for building automatic speech assessment systems [10,[16][17][18][19], e.g., the work done by Korzekwa et al on dysarthric speech [16].…”

Section: Automatic Assessment Approachesmentioning

confidence: 99%

“…Mel-frequency cepstral coefficients (MFCCs) are commonly used in speech assessment systems for acoustic modeling [10,[17][18][19]24] and feature extraction [25,26]. While deep learning models recently attract intense attentions, Mel Spectrogram is also getting increasingly popular [10,12,16].…”

Section: Speech Representationmentioning

confidence: 99%

Classifying Speech Intelligibility Levels of Children in Two Continuous Speech Styles

Lin

Tseng

2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Speech difficulties of children may result from pathological problems. Oral language is normally assessed by expertdirected impressionistic judgments on varying speech types. This paper attempts to construct automatic systems that help detect children with severe speech problems at an early stage. Two continuous speech types, repetitive and storytelling speech, produced by Chinese-speaking hearing and hearing-impaired children are applied to Long Short-Term Memory (LSTM) and Universal Transformer (UT) models. Three approaches to extracting acoustic features are adopted: MFCCs, Mel Spectrogram, and acoustic-phonetic features. Results of leave-one-out cross-validation and models trained by augmented data show that MFCCs are more useful than Mel Spectrogram and acoustic-phonetic features. Respective LSTM and UT models have their own advantages in different settings. Eventually, our model trained on repetitive speech is able to achieve an F1-score of 0.74 for testing on storytelling speech.

show abstract

“…The experiments are focused mostly on specific aspects or reduced populations. Some works focus on the speech intelligibility of people with aphasia [17]- [19]. Other experiments are reported with patients who had their larynx removed due to cancer and with children with cleft lip and palate [20].…”

Section: A Speech Technologies For Pronunciation Trainingmentioning

confidence: 99%

Evaluating the Impact of an Autonomous Playing Mode in a Learning Game to Train Oral Skills of Users With Down Syndrome

et al. 2021

View full text Add to dashboard Cite

The use of ICT tools is broadly extended among people with intellectual disabilities and also, to a lesser degree, the use of learning tools including learning games. Although the use of learning games is widely accepted due to its high engagement capacity, there are few studies that analyze its usability for people with intellectual disabilities. This work presents an evaluation of the impact of adding an autonomous playing mode on the usability of a learning game designed to aid the training of oral skills in people with Down syndrome. A study in which the effectiveness, efficiency and user satisfaction of a learning game are compared when the system is used with different degrees of teacher supervision is carried out. A learning game originally designed to train oral competencies for people with Down syndrome in a teacher supervised scenario is adapted to allow its autonomous use, by including a module that provides the automatic assessment of oral productions . The use of the tool is thus compared in three different scenarios: a supervised environment, autonomous use, and laboratory use with multiple users working in parallel. The different usability evaluation instruments used reveal that, although there are no differences in the degree of engagement, there may be important differences regarding session performance: the quality of the audios is lower in the laboratory sessions and the number of errors increases in the autonomous sessions. We conclude that, although the autonomous use of learning games by users with intellectual disabilities is possible, and this can lead to considerable savings in human resources, if the feedback provided by the game is not comparable with that provided by the teacher, performance may drop considerably although the degree of engagement is maintained.

show abstract

Automatic Assessment of Speech Impairment in Cantonese-Speaking People with Aphasia

Cited by 23 publications

References 38 publications

Improving Aphasic Speech Recognition by Using Novel Semi-Supervised Learning Methods on AphasiaBank for English and Spanish

Improving Aphasic Speech Recognition by Using Novel Semi-Supervised Learning Methods on AphasiaBank for English and Spanish

Classifying Speech Intelligibility Levels of Children in Two Continuous Speech Styles

Evaluating the Impact of an Autonomous Playing Mode in a Learning Game to Train Oral Skills of Users With Down Syndrome

Contact Info

Product

Resources

About