The CEFR Rating Scale Functioning: An Empirical Study on Self- and Peer Assessments

Abstract

One of the criticisms on the Common European Framework of Reference (CEFR) rating scales pertains to the lack of reference to the performance of learners in the construction process of the scales. Therefore, this study attempted to delve into rating scale functioning by English as a Second Language (ESL) learners during self-assessment and peer assessment of their oral proficiency practice. Two objectives guided the study: 1) to gauge the overall rating scale functioning and 2) to measure each criterion scaling structure. Three self- and peer assessments’ cycles were conducted in three months. In each cycle, eleven learners recorded their own speech, uploaded their video clips to a private YouTube channel and assessed their own videos as well as selected peers based on five CEFR oral assessment criteria with six levels of ratings (A1-C2). Findings revealed that four of the CEFR levels were utilised (B1-C2). Categories A1 and A2 (basic user level) however, were not observed during the practice. Analysis from the Many-Facet Rasch Measurement (MFRM) indicated that utilised categories seemed to function usefully since each category observed was advancing by more than 1.4 logits. Category B2 dominated four criteria of ratings awarded while B1 dominated the rating distribution for fluency. The implications of this study will be discussed in relation to rating scale development, specifically on matching learners’ proficiency to the psychometrically developed rating criteria as well as illustrating assessment as learning approach in the ESL classroom where learners become the key assessors for their own performance.

Common European Framework of Reference (CEFR): A Review on its Implementation in ESL/EFL Classrooms

“…But the majority of Malaysian ESL students fall short of attaining a level of proficiency that is competitive (Azman, 2017), particularly in communication and writing abilities.The English Language Education Roadmap in Malaysia, The Roadmap 2015-2025, 1-430, describes how a Cambridge Baseline Study was started in 2013 to gather data on Malaysian English language learners as well as the language proficiency of teachers from primary to university level. Preschool through post-secondary student samples, as well as a sample of English teachers, had their competency evaluated in accordance with the CEFR standards (Idris & Raof, 2017). English Language Education Roadmap in Malaysia, The Roadmap 2015-2025, 1-430).…”

“…It was discovered that the current English Education system is inadequate to create human resources that are fluent and marketable for the prospect of an international employment. because of the Classical Paradigm, which focused on grammar and writing abilities passed down through ESL teaching and learning (Idris & Raof, 2017).…”

“…The Malaysian Ministry of Education has decided to implement the CEFR in the nation's English language teaching and learning. By implementing an educational programme reform in the Malaysian English as a Second Language (ESL) syllabus, instruction, and evaluation, the Malaysian Education Blueprint 2013-2025 moves the Malaysian educational framework forward.The English Language Standard and Quality Council (ELSQC) was established, and the Roadmap 2015-2025 was commissioned, specifically for this reason (Idris & Raof, 2017). For each stage of schooling, a minimum CEFR level was defined as a goal in the Roadmap and is shown in Figure 2 (Idris & Raof, 2017).…”

Dear Second Language Learners (L2): The Complete Guide of Primary Trait Scoring Rubric for Self and Peer Assessment (SAPA)

“…Even with the introduction of school-based assessment (SBA) in 2015, students are still assessed by teachers. This has led to a situation where students are rarely asked to evaluate themselves or their classmates, as the system is skeptical of trusting students with the position of evaluator (Idris & Abdul Raof, 2017).…”

“…To ensure the implementation of SAPA in the classroom, an assessment method that is an effective rubric is needed to support students, especially second language (L2) learners. This is evidenced by the fact that most SAPA studies have used rubrics to test students' language proficiency (Idris & Abdul Raof, 2017), assess the validity of a score (Deygers & Van Gorp, 2015), identify assessment errors (Ahmadi Shirazi, 2019;Engelhard & Wind, 2019), and determine the effectiveness of teacher and learner assessment activities (Mohavedi & Kiasi, 2021). The purpose of this paper is to illustrate the potential of the primary trait rubric, which can be used in SAPA activities to assess students' performance.…”