Investigating the effect of classroom-based feedback on speaking assessment: a multifaceted Rasch analysis

Abstract

Due to subjectivity in oral assessment, much concentration has been put on obtaining a satisfactory measure of consistency among raters. However, the process for obtaining more consistency might not result in valid decisions. One matter that is at the core of both reliability and validity in oral assessment is rater training. Recently, multifaceted Rasch measurement (MFRM) has been adopted to address the problem of rater bias and inconsistency in scoring; however, no research has incorporated the facets of test takers’ ability, raters’ severity, task difficulty, group expertise, scale criterion category, and test version together in a piece of research along with their two-sided impacts. Moreover, little research has investigated how long rater training effects last. Consequently, this study explored the influence of the training program and feedback by having 20 raters score the oral production produced by 300 test-takers in three phases. The results indicated that training can lead to more degrees of interrater reliability and diminished measures of severity/leniency, and biasedness. However, it will not lead the raters into total unanimity, except for making them more self-consistent. Even though rater training might result in higher internal consistency among raters, it cannot simply eradicate individual differences related to their characteristics. That is, experienced raters, due to their idiosyncratic characteristics, did not benefit as much as inexperienced ones. This study also showed that the outcome of training might not endure in long term after training; thus, it requires ongoing training throughout the rating period letting raters regain consistency.

The interplay among self-assessment, using reflection for assessment, classroom enjoyment, and immunity: into prospects of effective language learning

“…Additionally, students' self-concept, self-identity, and sociocultural background are critical in the depth of students' attitudes toward CSA (Bourke & Mentis, 2007, 2013. As the previous studies pinpointed, investment in boosting CSA strategies among EFL learners influenced their academic well-being (Bijani et al, 2022;Heydarnejad, Ismail, et al, 2022;Punpromthada et al, 2022). According to Al-Mamoory and Abathar Witwit (2021), the more learners practice CSA, the better they can decide reflectively.…”

The Effect of Affection Control on University Learners’ Self-Evaluation and Theoretical Floatability: A Structural Equation Modeling

“…Shafiee Rad and Jafarpour (2022) affirmed that EFL students’ affection control and L2 grit impact their flexibility. Siemer (2001) and Bijani et al (2022) demonstrated the importance of self-assessment in EFL teachers’ self-efficacy. Peistaraite and Clark (2020) discovered that emotion planning promotes self-organized learning in classical musicians.…”

“…From a different angle, Essau (2021) defined emotions as personally constructed states of being that can result from conscious evaluations of personal accomplishments in achieving objectives or upholding standards or beliefs during interactions within social-historical contexts. The process-oriented model of ER typically defines the techniques used in affection control (Aunola et al, 2015; Bijani et al, 2022). This paradigm categorizes affection control into five groups: context selection, situation alteration, attentional focus, cognitive shift, and response modulation.…”