Automated assessment of second language comprehensibility: Review, training, validation, and generalization studies

Saito, Kazuya; Macmillan, Konstantinos; Kachlicka, Magdalena; Kunihara, Takuya; Minematsu, Nobuaki

doi:10.1017/s0272263122000080

Cited by 6 publications

(5 citation statements)

References 65 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This result could be due to the EFL teachers' similar contextual conditions, such as physical and social concerning curriculum, teaching practices, and aids and teaching approaches, namely the communicative teaching approach. This result is not line with that by Saito et al (2022) concluded that a variety of nonlinguistic elements such as speaker, age, gender, and microphone may affect automated speech recognition.…”

Section: Discussioncontrasting

confidence: 99%

“…Troyan et al (2013) implemented a practice-based approach centered around three key practices: comprehensive use of the target language during instruction, questioning to develop and gauge student understanding, teaching grammar inductively in meaningful contexts and co-constructing understanding. Saito et al (2022) examined the possibility of establishing quick and reliable automated comprehensibility assessments in which they gathered many spectrograms to build an acoustic model for each speech class. The spectrogram transmitted a variety of nonlinguistic elements "such as speaker, age, gender, and microphone" (p. 8).…”

Section: Literature Reviewmentioning

confidence: 99%

See 1 more Smart Citation

Teaching practices for English as foreign language at the tertiary level: A comprehensibility perspective

2023

View full text Add to dashboard Cite

Comprehension is one of the most crucial factors contributing to acquiring a new language; therefore, teachers must facilitate language comprehensibility using the best teaching practices to help learners understand the target language. This study aimed to identify tertiary EFL teachers’ practices for teaching language comprehensibility to assist in highlighting the comprehensibility practices tertiary EFL teachers employ to ensure that students understand, interact with, and use the English language. To identify the extent that the teachers employ language comprehensibility practices in an EFL context, the descriptive-correlational approach was employed. A closed-item questionnaire was administered to a sample of 65 teachers in Najran University, Saudi Arabia in the academic year 2021–2022. The Statistical Package for Social Sciences (SPSS v. 23) was used to analyze the data. The results showed that tertiary EFL teachers taught language comprehensibility very skillfully. Also, no significant difference in teaching language comprehensibility was found concerning the gender variable. There were, however, differences in the means of the sample’s responses towards methods of teaching language comprehensibility according to years of experience, particularly for those with the most years of experience. Considering the results, the study suggested paying more attention to integrating language comprehensibility practices in EFL contexts.

show abstract

Section: Discussioncontrasting

confidence: 99%

Section: Literature Reviewmentioning

confidence: 99%

Teaching practices for English as foreign language at the tertiary level: A comprehensibility perspective

2023

View full text Add to dashboard Cite

show abstract

“…It is believed that native speakers have a negative perception of fluency with longer pauses (longer than 250 ms), and this threshold can best distinguish pauses (de Jong and Bosker 2013). The commonly used threshold of 250 ms is also selected in accordance with previous empirical research (e.g., Bao 2023;Saito et al 2022;Suzuki and Kormos 2023). Second, the silence threshold was set to −20 dB, which is supposed to avoid noise interference and fit into the classroom setting (Saito et al 2022).…”

Section: Measures Of Fluencymentioning

confidence: 99%

“…The commonly used threshold of 250 ms is also selected in accordance with previous empirical research (e.g., Bao 2023;Saito et al 2022;Suzuki and Kormos 2023). Second, the silence threshold was set to −20 dB, which is supposed to avoid noise interference and fit into the classroom setting (Saito et al 2022). Third, manual re-checking was carried out after automatic identification of pauses and syllables.…”

Section: Measures Of Fluencymentioning

confidence: 99%

The Effects of Task Repetition Schedules on L2 Fluency Enhancement

Zhang,

Yi,

Zhou

2023

Languages

View full text Add to dashboard Cite

This article explores the effects of different task repetition schedules on English learners’ oral fluency in terms of speed, breakdown, and repair indices using a PRAAT script to enhance objectivity and consistency in the assessment of learner performances. A total of 90 freshmen participated in the experiment and were divided into three groups: the blocked repetition group, the interleaved repetition group, and a control group. This study adopted a pre- and post-test design. After the learners repetitively practiced the problem-solving tasks for three weeks, their improvement was measured by a new task of the same type. The analyses of speed, breakdown, and repair in learners’ oral performance reported that the experimental groups outperformed the control group in fluency measurement. Specifically, the interleaved repetition group was advantageous compared with the blocked repetition group, except for the silent pause numbers. The findings not only support the interleaving effects and enrich the line of task repetition research but also have pedagogical implications in that introducing interleaved practice in real classrooms is beneficial to L2 speaking enhancement.

show abstract

“…Therefore, this paper proposes a new way of assessing E-TPS by using Praat as a device, with human interpretation as a rater. Such a combination is expected to reorient the pedagogical goal in pronunciation teaching from the traditional focus on accent reduction, to investigating global measures of prosody and fluency, reliable automated comprehensibility assessments, and the effect of mobile-based aural comprehension lessons (Isaacs, 2018;Kallio et al, 2023;Saito et al, 2022;Woldetsadik et al, 2022).…”

Section: Introductionmentioning

confidence: 99%

Evaluating English Teachers’ Pronunciation Skills (E-TPS) Using Praat Technology and Human Raters

Ihsan,

Munir

2023

ITJ

View full text Add to dashboard Cite

When the goals in teaching pronunciation are that students attain “near-native” and “socially acceptable” pronunciation, and are assessed with a valid and reliable instrument using the Praat device and human interpretation raters, a new way to assess pronunciation skills is needed. This article proposes such a new way. Through suprasegmental identity (rhythm), English teachers’ pronunciation skill (E-TPS) will be correctly assessed. Using Praat as a device and human raters for interpretation to assess E-TPS stands on several premises. First, the measures of pronunciation constructs should be valid and reliable. Second, the role of raters in English pronunciation assessment presents unique challenges in drawing valid inferences from performance, in scoring assignments, in making decisions for which a pronunciation assessment was intended, to the pedagogical and social consequences beyond, and to connect concerns in giving judgments. Third, the raters should have familiarity with both accent and content. Fourth, a suprasegmental (stress, rhythm, and intonation) approach is an important complement to segmental teaching. Fifth, rhythm plays a big role and is defined as a “continuum” of functions and effects in pronunciation assessment. Sixth, the trend among the technology-minded for testing using automated assessment of pronunciation has as its goal accuracy in imitating native speakers, as is reflected in the use of Praat as a device. Therefore, the blending approach (Praat and raters) represents the future in assessing E-TPS.

show abstract

Automated assessment of second language comprehensibility: Review, training, validation, and generalization studies

Cited by 6 publications

References 65 publications

Teaching practices for English as foreign language at the tertiary level: A comprehensibility perspective

Teaching practices for English as foreign language at the tertiary level: A comprehensibility perspective

The Effects of Task Repetition Schedules on L2 Fluency Enhancement

Evaluating English Teachers’ Pronunciation Skills (E-TPS) Using Praat Technology and Human Raters

Contact Info

Product

Resources

About