Exploiting ultrasound tongue imaging for the automatic detection of speech articulation errors

Ribeiro, Manuel Sam; Cleland, Joanne; Eshky, Aciel; Richmond, Korin; Renals, Steve

doi:10.1016/j.specom.2021.02.001

Cited by 9 publications

(2 citation statements)

References 45 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Another approach involves machine learning to classify images. An example of this includes recent work to determine the correctness of articulatory gestures in children with speech disorders automatically, 36 an approach which can also be used for outcome measurement. Further work using machine learning to classify various speech, swallow and laryngeal functions is likely in the future.…”

Section: Broader Considerationsmentioning

confidence: 99%

An initial framework for use of ultrasound by speech and language therapists in the UK: Scope of practice, education and governance

2022

Self Cite

View full text Add to dashboard Cite

Background: There is growing evidence to support the use of ultrasound as a tool for the assessment and treatment of speech, voice and swallowing disorders across the Speech and Language Therapy profession. Research has shown that development of training competencies, engagement with employers and the professional body are vital to progressing ultrasound into practice. Methods: We present a framework to support translation of ultrasound into Speech and Language Therapy. The framework comprises three elements: (1) scope of practice, (2) education and competency and (3) governance. These elements align to provide a foundation for sustainable and high-quality ultrasound application across the profession. Results: Scope of practice includes the tissues to be imaged, the clinical and sonographic differentials and subsequent clinical decision-making. Defining this provides transformational clarity to Speech and Language Therapists, other imaging professionals and those designing care pathways. Education and competency are explicitly aligned with the scope of practice and include requisite training content and mechanisms for supervision/support from an appropriately trained individual in this area. Governance elements include legal, professional and insurance considerations. Quality assurance recommendations include data protection, storage of images, testing of ultrasound devices as well as continuous professional development and access to a second opinion. Conclusion: The framework provides an adaptable model for supporting expansion of ultrasound across a range of Speech and Language Therapy specialities. By taking an integrated approach, this multifaceted solution provides the foundation for those with speech, voice and swallowing disorders to benefit from advances in imaging-informed healthcare.

show abstract

Section: Broader Considerationsmentioning

confidence: 99%

An initial framework for use of ultrasound by speech and language therapists in the UK: Scope of practice, education and governance

2022

Self Cite

View full text Add to dashboard Cite

show abstract

“…To complement this broad range of applications, there is a growing interest in automatically processing and analysing ultrasound, for example, by extracting tongue contours (Fabre et al, 2015;Xu et al, 2016), animating tongue models (Fabre et al, 2017;Chen et al, 2018), classifying speech articulation errors (Ribeiro et al, 2019(Ribeiro et al, , 2021a, and most relevant to our work, synchronising it with simultaneously-recorded audio (Eshky et al, 2019).…”

Section: Ultrasound Tongue Imagingmentioning

confidence: 99%

Automatic audiovisual synchronisation for ultrasound tongue imaging

Eshky,

Cleland,

Ribeiro

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Ultrasound tongue imaging is used to visualise the intra-oral articulators during speech production. It is utilised in a range of applications, including speech and language therapy and phonetics research. Ultrasound and speech audio are recorded simultaneously, and in order to correctly use this data, the two modalities should be correctly synchronised. Synchronisation is achieved using specialised hardware at recording time, but this approach can fail in practice resulting in data of limited usability. In this paper, we address the problem of automatically synchronising ultrasound and audio after data collection. We first investigate the tolerance of expert ultrasound users to synchronisation errors in order to find the thresholds for error detection. We use these thresholds to define accuracy scoring boundaries for evaluating our system. We then describe our approach for automatic synchronisation, which is driven by a self-supervised neural network, exploiting the correlation between the two signals to synchronise them. We train our model on data from multiple domains with different speaker characteristics, different equipment, and different recording environments, and achieve an accuracy >92.4% on held-out in-domain data. Finally, we introduce a novel resource, the Cleft dataset, which we gathered with a new clinical subgroup and for which hardware synchronisation proved unreliable. We apply our model to this out-of-domain data, and evaluate its performance subjectively with expert users. Results show that users prefer our model's output over the original hardware output 79.3% of the time. Our results demonstrate the strength of our approach and its ability to generalise to data from new domains.

show abstract