Effects of benchmarking and peer‐assessment on French learners' self‐assessments of accentedness, comprehensibility, and fluency

Tsunemoto, Aki; Trofimovich, Pavel; Blanchet, Josée; Bertrand, Juliane; Kennedy, Sara

doi:10.1111/flan.12571

Cited by 13 publications

(10 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As previously mentioned, it is challenging for L2 learners to accurately self-assess their pronunciation (Li, 2018;Strachan, Kennedy & Trofimovich, 2019;Trofimovich et al, 2016). Yet, as noted by Tsunemoto et al (2022), "self-assessment is considered central to learners' autonomy" (p. 135) and "accurate self-assessment in pronunciation is particularly important because teachers often lack time, resources, and training to provide pronunciation-specific instruction" (p. 136) The use of ASR in the course where the current data was collected aimed to serve this exact purpose: to provide EFL learners the pronunciation practice that is lacking in their courses and, thereby, promote self-regulation and autonomy. Thus, learners' trust in the ability of ASR to recognize their speech in the same way a native speaker would is an important factor in encouraging autonomous practice.…”

Section: Discussionmentioning

confidence: 99%

“…Importantly, research indicates that explicit pronunciation instruction can lead to improvements in intelligibility (and comprehensibility) (Derwing, Munro & Wiebe, 1997) and with stronger effects when learners' pronunciation awareness is raised (Kennedy & Trofimovich, 2010). However, research also reveals that L2 learners' self-assessment of their speech might not be very accurate (Li, 2018;Tsunemoto, Trofimovich, Blanchet, Bertrand & Kennedy, 2022). In particular, L2 learners at the low end of the comprehensibility (and accentedness) scale were found to mostly overestimate their performance, with particularly strong effects for native Chinese learners of English (Trofimovich, Isaacs, Kennedy, Saito & Crowther, 2016).…”

Section: L2 Speech Intelligibilitymentioning

confidence: 99%

See 1 more Smart Citation

Assessment of L2 intelligibility: Comparing L1 listeners and automatic speech recognition

Inceoglu

Chen²,

Lim³

2022

ReCALL

View full text Add to dashboard Cite

An increasing number of studies are exploring the benefits of automatic speech recognition (ASR)–based dictation programs for second language (L2) pronunciation learning (e.g. Chen, Inceoglu & Lim, 2020; Liakin, Cardoso & Liakina, 2015; McCrocklin, 2019), but how ASR recognizes accented speech and the nature of the feedback it provides to language learners is still largely under-researched. The current study explores whether the intelligibility of L2 speakers differs when assessed by native (L1) listeners versus ASR technology, and reports on the types of intelligibility issues encountered by the two groups. Twelve L1 listeners of English transcribed 48 isolated words targeting the /ɪ-i/ and /æ-ε/ contrasts and 24 short sentences that four Taiwanese intermediate learners of English had produced using Google’s ASR dictation system. Overall, the results revealed lower intelligibility scores for the word task (ASR: 40.81%, L1 listeners: 38.62%) than the sentence task (ASR: 75.52%, L1 listeners: 83.88%), and highlighted strong similarities in the error types – and their proportions – identified by ASR and the L1 listeners. However, despite similar recognition scores, correlations indicated that the ASR recognition of the L2 speakers’ oral productions mirrored the L1 listeners’ judgments of intelligibility in the word and sentence tasks for only one speaker, with significant positive correlations for one additional speaker in each task. This suggests that the extent to which ASR approaches L1 listeners at recognizing accented speech may depend on individual speakers and the type of oral speech.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: L2 Speech Intelligibilitymentioning

confidence: 99%

Assessment of L2 intelligibility: Comparing L1 listeners and automatic speech recognition

Inceoglu

Chen²,

Lim³

2022

ReCALL

View full text Add to dashboard Cite

show abstract

“…Saito et al, 2020). More extended self-assessment practice or performing peer assessment benchmarking training over time (as in Tsunemoto et al, 2021) may be needed for learners to calibrate their selfassessments and to raise L2 learners' awareness of their own comprehensibility.…”

Section: Discussionmentioning

confidence: 99%

“…However, little research to date has investigated the effectiveness of these methods. Tsunemoto et al (2021) implemented a treatment based on benchmarking (asking learners to discuss speech evaluation criteria) and peer assessment (evaluating the speaking performance of peers) to raise L2 learners' awareness of their own comprehensibility. They found the treatment to effectively reduce the misalignment between selfassessments of comprehensibility and NLs' ratings, suggesting L2 learners' increased awareness of their own pronunciation.…”

Section: L2 Speech Self-assessmentmentioning

confidence: 99%

L2 Learners’ Self-Assessment of Comprehensibility and Accentedness: Over/Under-Estimation, Effects of Rating Peers, and Attention to Speech Features

2022

View full text Add to dashboard Cite

Comprehensibility and accentedness of L2 speech are often assessed through native listeners' ratings, but there is little research on learners' speech self-assessments. This study investigates the extent to which learners' self-assessment matched native listeners' evaluations and whether their self-assessments were influenced by having previously assessed the performance of peers. In addition, we asked learners to identify the speech features they related to comprehensibility and accentedness when self-assessing their speech. Advanced L2 English learners (N=56) performed a picture-description oral narrative task, which they then self-assessed for comprehensibility and accentedness under two conditions: having previously evaluated 20 speech samples of peers (N=24) or not (N=32). Native English listeners (N=14) assessed the 56 learners' narratives for the same dimensions. Results indicated that learners self-assessed their speech inaccurately for comprehensibility and accentedness, by either overestimating or underestimating their own speech, in accordance with previous research on comprehensibility (Trofimovich et al., 2016). Nevertheless, previous rating experience did not lead to more accurate self-assessments of comprehensibility and accentedness, suggesting that extended practice in the assessment of comprehensibility may be necessary for learners to be able to calibrate their speech selfassessments. When assessing their speech for comprehensibility, learners reported paying attention to pronunciation and the story plot rather than grammar or fluency, whereas for accentedness they focused on segmentals and overall accent rather than suprasegmentals.

show abstract

“…Besides, the majority of the studies examined the students' perceptions after the instruction; few tracked the student engagement throughout the AaL activities over time or probed into the assessment process (Stančić, 2021 ). Limitations in sample size (To and Panadero, 2019 ; Tsunemoto et al, 2021 ) or uni-dimensional qualitative approach also restricted the generalizability of findings to other populations and contexts.…”

Section: Literature Reviewmentioning

confidence: 99%

Assessment as Learning: How Does Peer Assessment Function in Students' Learning?

2022

View full text Add to dashboard Cite

Peer assessment (PA) is employed as one fundamental practice of classroom-based assessment in terms of its learning-oriented and formative nature. The exercise of peer assessment has multiple and additional benefits for student learning. However, research into the learning processes in peer assessment is scarce both in theory and in practice, making it difficult to evaluate and pinpoint its value as a tool in assessment as learning (AaL). This study focuses both on the learning process and outcome through assessment activities. We set out with three goals in mind: (1) to examine students' assessment performance in context, (2) to evaluate its impact on student progress, and (3) to illuminate teachers on organizing assessment activities. Three specific research questions are answered in this study: (1) How do student raters perform in the process of PA in an advanced English class? (2) To what extent do assessment activities influence the students' speaking ability? (3) What are students' perceptions of PA regarding its benefits and caveats? A total of 29 undergraduate students participated in two assessment activities on argumentative speaking. Many-Facet Rasch Model analysis was conducted to measure the rater effects both at the group level and the individual level. Bias/interaction analyses were performed to diagnose rater behavior in different contexts including the rating session, speaking session, and peer assessment vs. self-assessment. Questionnaire and semi-structured interview data were also collected to explore factors and strategies that could interfere with PA as AaL. Results show that students exhibited stable rating behavior and made progress in argumentative speaking in all dimensions, including delivery, organization, and language use. They are more stringent with themselves than with peers although there is one rare case with a bias against peers. Participants acknowledged the benefits of PA but also shared reasonable concerns in practice. This study validated the feasibility and the effectiveness of PA for student learning. Discussion on findings and guidelines for effective implementation of PA as AaL are provided.

show abstract

Effects of benchmarking and peer‐assessment on French learners' self‐assessments of accentedness, comprehensibility, and fluency

Cited by 13 publications

References 44 publications

Assessment of L2 intelligibility: Comparing L1 listeners and automatic speech recognition

Assessment of L2 intelligibility: Comparing L1 listeners and automatic speech recognition

L2 Learners’ Self-Assessment of Comprehensibility and Accentedness: Over/Under-Estimation, Effects of Rating Peers, and Attention to Speech Features

Assessment as Learning: How Does Peer Assessment Function in Students' Learning?

Contact Info

Product

Resources

About