Variations in Rating Scale Functioning in Assessing Speech Act Production in L2 Chinese

A comparative study of the effect of teacher’s group and individual feedback on Iranian EFL learners’ learning of speech acts in apology letter writing

“…The function for which a rating scale is used is also shown to be influential in Li, Taguchi, and Xiao (2019) who investigated the effect different functions, based on which a production is assigned a score, have in assessing speech act production. In their inquiry, they mainly focused on communicative function, situational appropriateness, and grammaticality and at the end, they found that rating scale functioning varied across different raters, different speech acts, and different proficiency levels.…”

Contrastive Pragmatics and Second Language (L2) Pragmatics: Approaches to Assessing L2 Speech Act Production

“…The issue of learner agency directs us to consider the role of native speakers in assessing speech acts, a challenge that both approaches face. As previously discussed, adopting the assumption of uniform native speaker norms when assessing L2 learners' performance is problematic, because native speakers vary in their perceptions of politeness and appropriateness (Taguchi, 2011b;Li et al, 2019). Hence, we need to explore alternative baseline models for comparative analysis, and to investigate the role of non-native raters.…”

“…For example, Taguchi (2011b) showed that native speaker raters coming from different cultural backgrounds (i.e., African American, Asian American, and Australian) brought with them quite different perceptions regarding appropriateness, politeness, and formality of speech acts. Li et al (2019) further reported that even native speakers with highly comparable cultural, educational, and professional backgrounds showed considerable variations in interpreting the same set of rating criteria. These findings echo recent critical reflections on the assumption of a uniform native speaker norm in the context of multilingualism and transculturalism, a point made earlier in discussing the disadvantages associated with the contrastive linguistics approach.…”

“…These findings echo recent critical reflections on the assumption of a uniform native speaker norm in the context of multilingualism and transculturalism, a point made earlier in discussing the disadvantages associated with the contrastive linguistics approach. The findings of Taguchi (2011b) and Li et al (2019) also point to the importance of implementing appropriate rater training programs as part of the assessment process. While the field of L2 pragmatics assessment has yet to investigate the effects of rater training on rating processes and rating outcomes, one can be informed by relevant research on performance-based language assessment at large.…”

“…As researchers continue to adopt and adapt Hudson et al's rating criteria, a major development has been to add a dimension of grammatical accuracy into speech act assessment (e.g. Sasaki, 1998;Taguchi, 2007Taguchi, , 2012Grabowski, 10.1163/26660393-bja10014 | Contrastive PragmaticS (2020) 1-23 2013; Li, 2014;Chen and Liu, 2016;Li, Taguchi, and Xiao, 2019;Xiao, Taguchi, and Li, 2019). While Hudson et al (1995) made it clear that the grammaticality of speech act production was not part of their consideration (p.165), researchers have argued that grammar and pragmatics are inevitably interconnected (Kasper and Rose, 2002;Bardovi-Harlig, 2003).…”