The Effects of Student-Fashioning and Teacher-Pleasing in the Assessment of First-Year Writing Reflective Essays

Abstract

The use of reflective essays has become a key artifact of outcome-based writing assessment in the field of writing studies (White, 2005). However, scoring reflective essays may be influenced by textual features irrelevant to most outcomes and assessment rubrics. Two problematic features are teacher-pleasing, which Yancey (1996) called the "schmooze factor," and student-fashioning, which Miura (2018) related to identity formation. In this article, we present two mixed methods studies to examine the effects of these particular textual features on the direct assessment of first-year writing (FYW) reflective essays. In the first pilot study, we identified four textual features relevant to teacherpleasing and student-fashioning. In the second quasi-experimental study, we created a sample of FYW essays with and without these features. Two assessment teams then scored the essays in order to determine whether these features had statistically significant effects on assessment scores. The empirical results of these linked studies indicate these features did not have significant effects in a direct assessment of FYW reflective essays. However, in the discussion and conclusion, we argue our mixed methods approach offers a feasible and efficient set of research methods to examine specific textual features in the direct assessments of student writing.

Prompting Reflection: Using Corpus Linguistic Methods in the Local Assessment of Reflective Writing

“…In addition to studying how identity markers affect scoring (Pruchnic et al, 2021), writing assessment researchers have developed various methods to assess reflection. In one line of research, researchers have attempted to constrain reflections to a specific task.…”