Automatic Essay Evaluation Technologies in Chinese Writing—A Systematic Literature Review

Abstract

Automatic essay evaluation, an essential application of natural language processing (NLP) technology in education, has been increasingly employed in writing instruction and language proficiency assessment. Because automatic Chinese Essay Evaluation (ACEE) has made some breakthroughs due to the rapid development of upstream Chinese NLP technology, many evaluation tools have been applied in teaching practice and high-risk evaluation processes. However, the development of ACEE is still in its early stages, with many technical bottlenecks and challenges. This paper systematically explores the current research status of corpus construction, feature engineering, and scoring models in ACEE through literature to provide a technical perspective for stakeholders in the ACEE research field. Literature research has shown that constructing the ACEE public corpus is insufficient and lacks an effective platform to promote the development of ACEE research. Various shallow and deep features can be extracted using statistical and NLP techniques in ACEE. However, there are still substantial limitations in extracting grammatical errors and features related to syntax and traditional Chinese Literary style. For the construction of scoring models, existing studies have shown that traditional machine learning and deep learning methods each have advantages in different corpora and feature selections. The deep learning model, which exhibits strong adaptability and multi-task joint learning potential, has broader development space regarding model scalability.

Information Communication Technology (ICT) and Education

“…Several studies were so-called systematic literature reviews, including focussing on ICT skills as part of 21st century education [16], learning with plants [17], and automatic Chinese Essay Evaluation [9] using Natural Language Processing (NLP) techniques. Given the worldwide distribution of telecommunications devices and an apparently unstoppable drift toward integrating ICT throughout workplaces and everyday life, it seems indisputable that everyone needs to develop skills to happily and effectively use these affordances.…”

“…In the third SLR, Yang et al [9] made a case that with the advancements of NLP, an increasing number of applications and organisations are implementing automatic essay evaluation (AEE). This is particularly beneficial for learners as it might provide automatic feedback and reflection on written work.…”

“…While for the English language there are already a wealth of NLP and AEE tools available, particularly with generative AI tools like ChatGPT, for other languages this is limited due to a lack of multiple publicly available corpora and tools. In their SLR focussed on automatic Chinese Essay Evaluation (ACEE), Yang et al [9] showcased that rapid progress has been made in the Chinese language context. They found 29 articles and concluded that there are still substantial limitations in extracting grammatical errors and features related to syntax and traditional Chinese literary style.…”

“…We arranged our Special Issue based upon six topics, namely artificial intelligence in learning [7][8][9], applied robotics in education [10], automated approaches to create smart and inclusive learning environments for blended and online learning [11], emerging technologies in education [2,[11][12][13][14], learning analytics, and big data in learning environments [6], and learning strategies for smart learning environments [15,16]. In terms of the methodologies used, beyond more common survey approaches [2,13,15], there were several innovative approaches, such as a pluralistic walk-through of learning analytics applications for practitioners [6], playing poker with robots [10], and log-data from interactive knowledge representations [12].…”

“…In terms of the methodologies used, beyond more common survey approaches [2,13,15], there were several innovative approaches, such as a pluralistic walk-through of learning analytics applications for practitioners [6], playing poker with robots [10], and log-data from interactive knowledge representations [12]. After we describe the three systematic literature reviews [9,16,17], we will briefly describe the main insights of these studies based upon these topics, after which we will provide some further recommendations for future research.…”