Reliability studies in broad and narrow phonetic transcription

Abstract

A 16-category framework is proposed to review the sources of variance in studies of phonetic transcription reliability. The same framework is used to analyse transcription agreement data collected in the course of a project in child phonology, including 22 reliability estimates from five consensus transcription teams who transcribed eight subject groups. Detailed agreement data at the level of consonants, vowels and diphthongs, feature classes, and diacritics are presented for each of the 16 categories, including such traditional measurement variables as sampling mode (continuous speech; articulation tests), agreement type (intrajudge; inter-judge), and level of transcription (broad; narrow). Tabular and plotted data are deliberately presented at the lowest feasible levels for readers interested in specific questions at these levels. A total of 16 generalizations about transcription reliability are derived from descriptive and inferential statistical findings. The primary conclusion is that for certain clinical and research tasks in communicative disorders, broad phonetic transcription appears to be reliable, whereas narrow transcription may be unreliable.A historical overview of transcription studies in communicative disorders in America suggests a division into two periods, beginning with Henderson's (1938) call for programmatic research. From the 1930s through approximately 1970, research reflected the emerging discipline's attempt to develop effective ways to rate, score, and transcribe deviant speech. Impelled by efficiency criteria the assessment paradigm evolved to a form of nominal scoring of responses evoked primarily by articulation test stimuli (e.g. Burkowsky). A second period, beginning in the early 1970s and putatively concluding with the decade of the 1980s, witnessed renewed interest in phonetic transcription as input for linguistic description procedures developed for young normally developing children and children with intelligibility deficits (e.g.

Speech characteristics of 8-year-old children: Findings from a prospective population study

“…In many cases, this is clear cut, but where it is not, discrepancies between one transcriber and another can occur. Narrow transcription is required to identify and mark such distortions and while the importance and value of narrow transcription is recognized (Ball et al, 2009), reliability between transcribers is often low (Shriberg & Lof, 1991). Consequently, due to the large number of children, and the need for reliability in the present study, a pragmatic approach was used.…”

Development of Grammatical Accuracy in English-Speaking Children With Cochlear Implants: A Longitudinal Study

“…The reliability of broad transcription for children with CIs was reported by Spencer and Oleson (2008). The average point-to-point reliability of phoneme accuracy was 79% (SD = 9%), which is considered to be within the acceptable range (i.e., 60%-90%; Shriberg & Lof, 1991) for phonetic transcription. To check the reliability of orthographic transcription for children with typical hearing, we randomly sampled the stories from 30% of children with typical hearing in each age group (n = 9).…”

Polysyllable productions in preschool children with speech sound disorders: Error categories and the Framework of Polysyllable Maturity

“…In cases of mismatches, the decision of the first author was upheld. In a study of the sources of variance which may affect reliability in phonetic transcription, Shriberg and Lof (1991) described "acceptable agreement" as >85% (p. 255).…”