An Empirical Study on the Impact of an Automated Writing Assessment on Chinese College Students’ English Writing Proficiency

Abstract

Automated writing evaluation (AWE) is an online essay scoring system which can provide feedback and revising advice to teachers and students. In this paper, an empirical study was carried out to explore the impact of Writing Roadmap2.0 (WRM2.0)-an automated writing assessment system on the English writing proficiency, which is reflected in three dimensions-the language form, the contextual structure and the writing quality of non-English major freshmen in China. In this study, 100 participants were divided into the experimental class (EC) and the controlled one (CC) at random, with 50 ones in each class. Both qualitative method and quantitative method were adopted for data collection and analysis, including the pre-and post-tests on WRM2.0, teacher-assessed writing task and interviews. The results revealed that while there was no significant difference in the writing proficiency in pre-test on WRM2.0 between EC and CC, the former outperformed the latter in both post-test on WRM2.0 and teacher-assessed writing task in the final exam in two dimensions: the language form and the writing quality. With regard to the aspect of contextual structure, EC benefited a little on WRM2.0. Generally speaking, this empirical study observed positive impact of WRM2.0 on writing proficiency of L2 students in China. It is expected that the findings will provide references for the further integration of AWE with writing teaching and learning in the EFL classroom.

College Students' Perceptions of an Automated Writing Evaluation as a Supplementary Feedback Tool in a Writing Class

“…Secondly, our findings highlighted the need and usefulness of grammar, vocabulary, punctuation, and spelling feedback to improve the students' drafts. This study resonances with previous study indicating the usefulness of AWE program to help the students reduce the grammatical errors (Huang & Renandya, 2018;Kellogg et al, 2010;Parra & Calero, 2019), help students improve their vocabulary (He, 2016;Huang & Renandya, 2018), and mechanics errors (Huang & Renandya, 2018;Kellogg et al, 2010;S. Wang & Li, 2019) on the students' drafts.…”

“…The feedback offered by AWE is varied in terms of linguistic accuracies, such as grammar, mechanics, or vocabulary choice feedback (Zhang, 2016). Due to its potential in giving feedback, there has been a growing interest in using the program, especially to improve the students' writing quality (Wang & Li, 2019), by also reducing the grammatical errors and mechanics or style issues (Parra & Calero, 2019).…”

EFL students’ perception on the use of Grammarly and teacher feedback

“…Germane to studies on AWE programs, the previous studies focused mainly on the outcomes or scores of the students' writing (Qassemzadeh & Soleimani, 2016;Karyuatry et al, 2018;Yulianti & Reni, 2018;Wang & Li, 2019), the comparison between AWE feedback and other types of feedback Liu & Kunnan (2016), and the validity of AWE as a scoring system (Wang & Brown, 2007;Chapelle et al, 2015). In addition, the extant studies were also mainly conducted particularly at intermediate-above level Chen & Cheng (2008) or at low-proficiency level Huang & Renandya (2018).…”

Exploring the Effects of Automated Written Corrective Feedback on EFL Students’ Writing Quality: A Mixed-Methods Study

“…Franzke et al (2005) examined the effect of Summary Street feedback on students' writing quality, and they found that students receiving the feedback performed significantly better than students receiving no feedback in text quality, content, organization, and stylistic quality. Second, when it comes to comparing AWCF with teacher WCF, studies also have revealed the superiority of the former over the latter (e.g., S. Wang & Li, 2019; Y. J. Wang et al, 2013;Warden, 2000).…”