Establishing Rating Scales to Assess Writing Proficiency Development in Young Learners

Abstract

Writing assessment scales were developed to include functional aspects of writing proficiency in contemporary Norwegian teaching toolkits for Grades 1 to 3. This study aims to describe the process of developing empirically based, assessor-oriented writing proficiency scales and of investigating the quality of the scales. We focus on psychometric qualities, professional users’ perceptions of their quality, and the teachers’ use of the scales. Overall, the first piloted version of the scales showed indications of well-functioning scales. The results from this investigation show that it is possible to develop scales for the assessment of young children’s writing proficiency that capture the intended construct and provide a basis for reliable assessment. The investigation also found that users of the assessment tool found that it functioned well.
 Keywords: writing assessment, writing development, rating scales, reliability
 Upprätta bedömningsskalor för att utvärdera utvecklingen av skrivförmågan hos unga studenter
 Sammanfattning
 Denna artikel berör arbetet med att utveckla bedömningsskalor för bedömning av funktionell skrivförmåga i årskurs 1–3. Syftet med undersökningen var att dels beskriva arbetet med att skapa empiriskt baserade bedömar-orienterade skalor, dels att undersöka kvaliteten på dessa skalor. Det senare gjordes genom att studera skalorna psykometriska kvalitet, den kvalitet de uppfattades att ha samt lärares användning av skalorna. Sammantaget visade undersökningarna att skalorna fungerade väl, redan vid första utkast. Vidare indikerade resultaten att det var möjligt att konstruera skalor för bedömning av små barns skrivande som kunder generera reliabel bedömning samtidigt de fångade väsentliga aspekter av funktionellt skrivande. Undersökningarna indikerade också att användare av bedömningsverktygen generellt uppfattade dem som välfungerande.
 Nyckelord: skrivbedömning, skrivutveckling, bedömningsskalor, reliabilitet

Learning loss during the COVID-19 pandemic and the impact of emergency remote instruction on first grade students’ writing: A natural experiment.

“…Each trait was scored using a separate 1 to 5 rating scale. The rating scaled were developed and validated for use in Grades 1–3 by Skar, Jølle, and Aasen (2020), and also used in Skar et al (2021), who reported that, on average, girls outperformed boys, and students in higher grades outperformed students in lower grades. Higher scores represented better quality for the trait assessed.…”

The Long-Term Effects of the COVID-19 Pandemic on Children’s Writing: a Follow-up Replication Study

“…As an example, consider audience awareness which targeted the extent to which a text not only communicated with the reader, but also if the young author had managed to decontextualize his/her writing enough for a reader to make meaning of the content even if the reader was unable to clarify content in interviews with the author (Skar, Aasen et al, 2022 ). Prior to this investigation, the rating scales had been validated (Skar, Jølle, et al, 2020 ) and used to assess thousands of texts (Ska, Lei, et al 2022 ). Please refer to Appendix A in Skar, Kvistad, et al ( 2022 ) for descriptors for all eight scales.…”

Functional Writing in the Primary Years: Protocol for a Mixed-Methods Writing Intervention Study

“…Unfortunately, there is currently a lack of knowledge concerning these areas. To meet these challenges and to provide the sector and teacher education with relevant knowledge, the FUS project's primary objective is to investigate the consequences of an early start with functional writing on young students' writing proficiency (see Skar et al, 2020 for construct definitions), teachers' professional learning, and learning activities.…”

“…To measure text quality, a new assessment tool (Skar et al, 2020), which in a validating phase produced reliability well over .85, was developed. All texts will be rated on eight rating scales (Audience Awareness, Vocabulary, Organization of Content, Language Use, Punctuation, Spelling, Handwriting, and Relevance), and all assessments will be processed using contemporary statistical techniques in writing assessment (i.e., many-facet Rasch measurement; Skar, 2017).…”

“…TLAs include but are not limited to reading material, listening to podcasts, writing reflective notes, etc. Teachers will also use a newly developed assessment tool that highlights eight aspects of writing quality (Skar et al, 2020), which is presumed to broaden their understandings and give nuance to the text quality in texts written by young children. • Network sessions (NS): meetings with facilitators from the project or other project schools to share experiences, etc.…”