Pragmatic Rating of L2 Refusal: Criteria of Native and Non-native English Teachers

Abstract

Many studies have shed light on rater criteria for assessing the performance of language skills (e.g., Eckes, 2005). However, the interface between rater assessment and interlanguage pragmatics (ILP)

Politeness and Speech acts of Refusal and Complaint among Jordanian Undergraduate Students

“…The instruments were used to measure the students' ability to implement refusal and complaint strategies fluently and properly in various situations. The DCT for refusals consisted of 14 situations and was adopted from Alemi and Tajeddin (2013) and Ren (2012). The first six situations, obtained from Alemi and Tajeddin (2013), focused on different contexts (e.g., educational, workplace, and daily life).…”

Rater Behaviour When Judging Language Learners’ Pragmatic Appropriateness in Extended Discourse

“…The challenges of determining the construct representation, developing practical, valid, and reliable tests (e.g., Roever, 2008Roever, , 2011Roever, Fraser, & Elder, 2014;Walters, 2007Walters, , 2009, and performing classroom assessment (Ishihara, 2013;Ishihara & Cohen, 2010) have been recently addressed in the literature, but one aspect that has received little attention is rater behaviour (for rare exceptions, see Alemi & Tajeddin, 2013;Brown & Ahn, 2011;Liu, 2007;Liu & Xie, 2014;Roever, 2008;Taguchi, 2006Taguchi, , 2011Tajeddin & Alemi, 2014;Walters, 2007;Youn, 2007).…”

“…Studies examining how raters approach the judgment of pragmatic performance cited such effects as different degrees of severity between raters and item types (Youn, 2007), differences in ratings between NS and non-nativespeaking (NNS) judges (Alemi & Tajeddin, 2013;Liu, 2007;Walters, 2007), familiarity of the raters with the learners' culture, and differential assignment of weightings to certain aspects of performance, such as grammar (Liu & Xie, 2014).…”

“…Raters also weighed the criteria differently. Alemi and Tajeddin (2013) investigated the different rating criteria that NS and NNS ESL teachers apply, and found that NSs used 11 criteria, while NNS used only 6. NSs tended to use more specific criteria, such as "reasoning/explanation," while NNSs most commonly used the general "appropriateness" criterion.…”

“…Like Alemi and Tajeddin (2013) and Walters (2007), we were interested in how raters approach the process of rating pragmatic appropriateness without specific training or guidelines. Thus, like Walters, we used a very general holistic rating scale without scale descriptors.…”