Investigation of Writing Assessment Literacy of Ukrainian University Teachers

Abstract

The rating process of students’ writing has been a long-standing concern in L2 large-scale standardized and classroom-based assessment. Several studies have tried to identify how the raters make scoring decisions and assign scores to ensure validity of writing assessment. The current paper addresses the issue of writing assessment practices of Ukrainian university teachers, how they approach rating scales and criteria with an attempt to understand culturally specific challenges of teachers’ writing assessment in Ukraine. To investigate the issue, this study employs the analysis of the survey results obtained from 104 university teachers of English. The survey consisted of 13 questions that provided insight into current practices in assessment of writing, such as frequency of assessment, use of rating scales, rater’s profile, criteria of assessment, feedback and rewriting, training in assessment of writing. The survey responses show that assessment in Ukraine is not regulated by common standard, and thus the approach to students’ writing assessment is often intuitive. A frequent practice is that teachers tend to rely on errors – as observable features of the text – to justify their rating decisions, Consequently, by shifting focus onto the surface features of writing, grammar mistakes in particular, the teachers underrate such criteria as “register”, “compliance with textual features” and “layout”. Additionally, the data reveal contradictory findings about writing assessment literacy of the teachers questioned. Even though most teachers claim they apply scales while rating, many confess they cannot tell the difference between holistic and analytic scales. Moreover, the results indicate that feedback is not yet a meaningful interaction between a Ukrainian teacher and a learner. Therefore, the results of the study demonstrate the need for the improvement in writing assessment practices, which could be achieved through providing training and reorientation to help Ukrainian teachers develop common understanding and interpretation of task requirements and scale features.

Implications of training university teachers in developing local writing rating scales

“…Embarking on this investigation, we presumed that the factors affecting the rating scale development process might involve certain gaps in the linguistic and assessment competence of the assessors, which had been previously identified in the survey of teacher writing assessment literacy (Kvasova et al, 2019) Therefore, we expected that consideration of these issues would contribute to an understanding of best practice in scale design implemented by practicing teachers. The insights gained will be drawn on in the development of procedures and materials for training teachers in assessing writing in Ukrainian universities, which could be useful in other contexts, too.…”

“…Based on the results of a survey of typical assessment practices of 104 Ukrainian university teachers (Kvasova et al, 2019), we can state that the teachers are continually engaged in assessing students' writing. Some of them are reasonably familiar with the RSs used in commercial examination systems.…”

“…While doctoral research studies into teaching L2 speaking and writing are not scarce in the country, the assessment issues have not been broadly addressed in practice. In respect of the assessment of writing in particular, teachers experience major difficulties, among which the absence of detailed guidelines on rating scale (RS) design and/or use seem to be critical causes (Kvasova et al, 2019). To bridge this gap, an Erasmus+ mobility project (2017)(2018)(2019), with the team of UALTA trainers in LTA involved, aimed to explore the process and outcomes of university teachers' developing and applying an analytic RS (Kvasova et al, 2021).…”