The development and validation of the Closed-set Mandarin Sentence (CMS) test

Abstract

Matrix-styled sentence tests offer a closed-set paradigm that may be useful when evaluating speech intelligibility. Ideally, sentence test materials should reflect the distribution of phonemes within the target language. We developed and validated the Closed-set Mandarin Sentence (CMS) test to assess Mandarin speech intelligibility in noise. CMS test materials were selected to be familiar words and to represent the natural distribution of vowels, consonants, and lexical tones found in Mandarin Chinese. Ten key words in each of five categories (Name, Verb, Number, Color, and Fruit) were produced by a native Mandarin talker, resulting in a total of 50 words that could be combined to produce 100,000 unique sentences. Normative data were collected in 10 normal-hearing, adult Mandarin-speaking Chinese listeners using a closed-set test paradigm. Two test runs were conducted for each subject, and 20 sentences per run were randomly generated while ensuring that each word was presented only twice in each run. First, the level of the words in each category were adjusted to produce equal intelligibility in noise. Test-retest reliability for word-in-sentence recognition was excellent according to Cronbach’s alpha (0.952). After the category level adjustments, speech reception thresholds (SRTs) for sentences in noise, defined as the signal-to-noise ratio (SNR) that produced 50% correct whole sentence recognition, were adaptively measured by adjusting the SNR according to the correctness of response. The mean SRT was −7.9 (SE=0.41) and −8.1 (SE=0.34) dB for runs 1 and 2, respectively. The mean standard deviation across runs was 0.93 dB, and paired t-tests showed no significant difference between runs 1 and 2 (p=0.74) despite random sentences being generated for each run and each subject. The results suggest that the CMS provides large stimulus set with which to repeatedly and reliably measure Mandarin-speaking listeners’ speech understanding in noise using a closed-set paradigm.

Masking Effects in the Perception of Multiple Simultaneous Talkers in Normal-Hearing and Cochlear Implant Listeners

“…The Closed-set Mandarin Speech (CMS; Tao et al., 2017) test materials were used to test speech understanding in the presence of one or more competing talkers. The CMS test materials consist of familiar words selected to represent the natural distribution of vowels, consonants, and lexical tones found in Mandarin Chinese.…”

“…SRTs, defined as the SNR that produced 50% correct word recognition, were adaptively measured using a modified coordinate response matrix (CRM) test (Brungart et al, 2001;Tao et al, 2017Tao et al, , 2018. Similar to CRM tests, two target keywords (randomly selected from the Number and Color categories) were embedded in a five-word carrier sentence uttered by the Mandarinspeaking male target talker.…”

Tinnitus impairs segregation of competing speech in normal-hearing listeners

“…The SRT for each test run was calculated by averaging the TMR across the last 6 reversals. The Closed-set Mandarin Speech (CMS) corpus 51,52 was used to measure segregation of competing speech. The CMS corpus consists of matrix-styled test materials which can be used to randomly create five-word sentences with same grammatical structure: name, verb, number, color, and object, each of which contain 10 words.…”

Cochlear Implant Facilitates the Use of Talker Sex and Spatial Cues to Segregate Competing Speech in Unilaterally Deaf Listeners

“…Recognition of target speech in the presence of masker speech was measured using the matrix-style test materials from the Closed-Set Mandarin Speech corpus (Tao et al 2017), which has been used in several studies by Fu and colleagues (Tao et al 2018; Liu et al 2019; Chen et al 2020; Liu et al 2020). The stimuli consisted of 5 categories (Name, Verb, Number, Color, and Object), each with 10 words.…”