ChatGPT Analysis of Strengths and Weaknesses in English Writing and Their Implications

Abstract

The emergence of the ChatGPT language model has sparked significant interest in educational circles, thanks to its capabilities in generating text, understanding context, and engaging in logical reasoning and conversation. This study explores the impact of ChatGPT on English writing, integrating statistical language models and TF-IDF word frequency analysis to develop a method for identifying and scoring deviations in English essays. Our findings demonstrate the model’s high accuracy in detecting digressions across three categories (88%, 73.33%, and 60%), and its scoring closely aligns with human assessments, with a mean error of just 5.17 and a Pearson correlation coefficient of 0.8024. This evidence supports the model’s reliability and significant potential for application in English writing evaluation.