Can ChatGPT Grade Non-Native Academic English Writing?

Abstract

This chapter reports the results of a pilot study investigating the potential of ChatGPT to assess (or grade) texts produced by international graduate students whose mother tongue is other than English (IGS). The chapter overviews some suggested applications of ChatGPT to academic writing (AW) and particularly, for English as an additional language (EAL) contexts. Most works up to date consider ChatGPT applications for literacy and writing skills development. The authors compare evaluations of texts by ChatGPT vis-à-vis human experts. The texts employed for the study are short literature reviews written by IGS in Canadian universities. The results demonstrate some potential of ChatGPT for assessment. However, while ChatGPT provides mostly relevant scores and comments related to the assessment rubrics provided, the comments are not diversified and individualized enough. ChatGPT queries for assessment should be modified and the tool trained in diversifying comments to achieve better results.