Scoring performance tests

Oral Proficiency Interview (OPI) for the Assessment of Speaking

“…Therefore, it can be argued that the tester plays a crucial role in the performance of the test-taker since the conversation is tightly controlled, delivered and co-constructed by the tester (Hırçın Çoban, 2017; Kasper & Ross, 2007). Put differently, rater effects that include consistency or inconsistency in a rater's elicitation techniques and scores, rater severity or leniency, illusory halo or horns and rater bias may lead to contamination of the assessment (Ahmadi & Sadeghi, 2016;Brooks, 2009;Brown & Abeywickrama, 2018;Chalhoub-Deville & Fulcher, 2003;Cho, 2004;Davis, 2022;Galaczi & Lim, 2022;Hırçın Çoban, 2017;Malone, 2003;O'Loughlin, 2002;Pill & Smart, 2021;Ross, 2012;Wigglesworth & Frost, 2017). As has been noted by Malone and Montee (2010), though OPIs are implemented by trained and certified testers, "…human error is inevitable and can be limited but not eliminated" (p. 978).…”

“…In order to score any oral and/or written performance, a well-designed rating scale (or rubric) is essential to ensure reliability (Brown & Abeywickrama, 2018;Fulcher, 2012;Galaczi & Lim, 2022;Green, 2021;Namaziandost & Ahmadi, 2019;Wigglesworth & Frost, 2017). Either holistic or analytic scales are commonly used for scoring the speaking performance of the test-takers.…”

“…Either holistic or analytic scales are commonly used for scoring the speaking performance of the test-takers. In a holistic scale, a single number is awarded to a specific performance (Brown & Abeywickrama, 2018;Galaczi & Lim, 2022;Gültekin, 2022;Kuiken & Vedder, 2021;Louma, 2004) and that number is meant to signify the overall speaking ability of the test-taker (Fulcher & Davidson, 2007). It should be noted that though holistic scales are user-friendly, practical and fast to use, they do not provide the test-takers with any kind of feedback (Hamp-Lyons, 1991;Kuiken & Vedder, 2021).…”

“…In contrast, in an analytic scale, multiple scores across a range of categories or criteria (such as fluency, vocabulary use, pronunciation, interaction, content, etc.) are given to a single spoken performance and these scores may be added together to obtain an overall score (Brown & Abeywickrama, 2018;Galaczi & Lim, 2022;Green, 2021;Louma, 2004;Richards, 2015). Therefore, test-takers have the chance to get specific diagnostic feedback as to their strengths and weaknesses (Kuiken & Vedder, 2021).…”

“…A well-designed rating scale may be regarded as the first step for a reliable assessment of speaking; however, the same scale may be interpreted differently by different raters since it is the rater that mediates between the rating scale and the performance of the specific test-taker (Davis, 2022;Galaczi & Lim, 2022;Pill & Smart, 2021). Therefore, the necessity of training the interlocutors and raters to attain high level of standardization (of the experiences of all the test-takers as much as possible) has been frequently underscored in the relevant literature (Brown, 2012;Davis, 2022;Fulcher & Davidson, 2007;Green, 2021;Hamp-Lyons, 1991;Hırçın Çoban, 2017;Kuiken & Vedder, 2021;Louma, 2004;Malone, 2000;Malone & Montee, 2010;Pill & Smart, 2021;Ross, 2012;Surface & Dierdorff, 2003;Wigglesworth & Frost, 2017).…”