Intelligibility of Disordered Speech: Global and Detailed Scores

Abstract

Measuring the intelligibility of disordered speech is a common practice in both clinical and research contexts. Over the years various methods have been proposed and studied, including methods relying on subjective ratings by human judges, and objective methods based on speech technology. Many of these methods measure speech intelligibility at the speaker or utterance level. While this may be satisfactory for some purposes, more detailed evaluations might be required in other cases such as diagnosis and measuring or comparing the outcomes of different types of therapy (by humans or computer programs). In the current paper we investigate intelligibility ratings at three different levels of granularity: utterance, word, and subword level. In a web experiment 50 speech fragments produced by seven dysarthric speakers were rated by 36 listeners in three ways: a score per utterance on a Visual Analogue and a Likert scale, and an orthographic transcription. The latter was used to obtain word and subword (grapheme and phoneme) level ratings using automatic alignment and conversion methods. The implemented phoneme scoring method proved feasible, reliable, and provided a more sensitive and informative measure of intelligibility. Possible implications for clinical practice and research are discussed.

Towards a Comprehensive Assessment of Speech Intelligibility for Pathological Speech

“…All the methods described above rely on perceptual judgments. It is common practice to collect different measures from multiple judges and check their reliability before the usage for future research purposes [5,6,11], since these measures can be influenced by, for example, the type of judge or listener. As pointed out in [18], measures collected from inexperienced ('lay' or 'naive') listeners showed larger variances than those collected from well-trained expert judges such as speechlanguage therapists [18].…”

Acoustic correlates of speech intelligibility: the usability of the eGeMAPS feature set for atypical speech

“…Besides, reliability measures are also needed. Research has shown that these operations can help obtain reliable ratings [21], and in fact, have been widely used in research and clinical practice. However, these operations are generally time-consuming and costly and the need for multiple raters makes this practice even more laborious and expensive.…”

“…An important question in this connection is which intelligibility scores should be taken as the point of reference. As explained above, different types of human ratings have been employed in literature, and these often vary with respect to their degree of detail [21]. In addition, not all sorts of human ratings are available for all speech databases.…”

Articulatory Features for ASR of Pathological Speech

“…According to the outcomes of the efficacy tests presented in [3], user satisfaction towards such a system appears to be quite high. However, most of these systems are not yet capable of automatically detecting problems at the level of individual speech sounds, which are known to have an impact on speech intelligibility [4][5][6][7][8]. Our goal is to develop robust automatic speech recognition (ASR) systems for pathological speech and incorporate the ASR technology to detect these problems.…”