PRAAT scripts to measure speed fluency and breakdown fluency in speech automatically

Abstract

Fluency in terms of speed of speech and (lack of) hesitations such as silent and filled pauses ('uhm's) is part of oral proficiency. Language assessment rubrics therefore include aspects of fluency. Measuring fluency, however, is highly time-consuming because of the manual labour involved. The current paper aims to automatically measure aspects of L2 fluency, including filled pauses, in both Dutch and English. A revised existing script and a new script for filled pauses are tested on accuracy. We also gauged whether the outcomes of the new script could be used for language assessment purposes by relating the outcomes to human judgements. Without further investigations, the current script should not (yet) be used for the purpose of assessing fluency automatically in (high-stakes) oral proficiency assessment. However, the performance of the scripts for measuring aspects of fluency globally and quickly are promising, especially given their stability in accuracy on new corpora.

Assessment of fluency in the Test of English for Educational Purposes

“…Observing a ceiling effect for speed fluency is also an important finding to be considered by rating scale/descriptor developers. The findings of the study are important as they strengthen the existing evidence that enables researchers to develop automatic scoring of fluency for the purpose of language assessment (Davis & Papageorgiou, 2021; De Jong et al, 2021; Isaacs, 2018).…”

“…The attention given to fluency in SLA can be attributed to the increasing importance of fluency in oral language ability (Suzuki & Kormos, 2020), the relationship between fluency and automaticity (DeKeyser, 2007;Tavakoli, 2019), and the key role fluency plays as an indicator of global language ability (Révész et al, 2016). Interest in examining fluency has recently increased as research in artificial intelligence suggests fluency is an aspect of performance that lends itself well to automated assessment of oral language ability (Davis & Papageorgiou, 2021;De Jong et al, 2021). From an SLA perspective, fluency is particularly important as a window that allows researchers to investigate a set of cognitive processes underlying speech production (Segalowitz, 2010;Skehan, 2009;Suzuki, 2021).…”

Automated extraction of speech and turn-taking parameters in autism allows for diagnostic classification using a multivariable prediction model

“…We calculated pitch and intensity synchrony with rMEA's cross-correlation function to calculate windowed cross-lagged correlations (WCLC) using the same window length of 16 s, step size of 8 s and lag of 2 s as Ochi et al (18). We used the uhm-o-meter (30,31) to extract turns from conversations, with a turn defined as all speaking instances of one interactant until the end of the speaking instance preceding the next speaking instance of someone else (see Figure 2). For each turn, we calculated turn-taking gap, average pitch, average intensity and number of syllables to calculate articulation rate.…”

Zooming into the L2 Speech Fluency Markers of Anxious and Non-Anxious Advanced L2 Learners – an Extreme Case Sampling Report

“…The transcription conventions applied after Peltonen and Lintunen (2016) were as follows: (. )micro pauses .25 and shorter, :elongations or drawls of sounds (e.g., a:nd), *pt*lip smack, *hah*laughter, (0.43)timed pause, *h*audible breaths, {*h*_0.83}timed audible nonlexicalized filled pauses, including audible breaths.. Silent pauses longer than .25 were annotated in PRAAT (Boersma & Weenink, 2007) with the support of PRAAT script (De Jong et al, 2021), and temporal measures calculated with Lennes's (2002) script. The numbers of filled pauses and syllables were calculated manually.…”