Machine Learning-based Automated Essay Scoring System for Chinese Proficiency Test (HSK)

Xiao, Rui; Guo, Wenbin; Zhang, Yunchun; Ma, Xiaoyan; Jiang, Jiaqi

doi:10.1145/3443279.3443299

Cited by 7 publications

(5 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Fine-tuning these models, as demonstrated by Wang, et al 45 and Yang, et al 46 , has led to the creation of more refined and accurate scoring systems. The convergence of AI and education is a crucial juncture that foreshadows the advent of personalized and inclusive learning experiences [47][48][49] . LLMs are poised to offer promising results in terms of accuracy, adaptability, and contextual comprehension.…”

Section: Llm Based Approachesmentioning

confidence: 99%

Beyond Traditional Assessment: Exploring the Impact of Large Language Models on Grading Practices

Fagbohun,

Iduwe,

Abdullahi

et al. 2024

JAIMLD

View full text Add to dashboard Cite

In this study, we examine the transformative role of large language models (LLMs) in redefining educational assessments. Traditional grading systems, characterized by their uniform and often manual approaches, face significant challenges in terms of scalability, consistency, and personalized feedback. The advent of LLMs heralds a new era of assessment, offering nuanced, scalable, and efficient solutions. This study explores the integration of LLMs in grading practices and examines their potential to revolutionize the assessment landscape.We begin by analyzing the limitations of traditional grading methods and emphasizing the need for more sophisticated and adaptable solutions. This paper then introduces the concept of LLMs, outlining their advanced capabilities in natural language processing and machine learning, which are pivotal in understanding and evaluating student responses. We delve into the mechanisms by which LLMs process, analyze, and grade a wide range of responses, from short answers to complex essays, highlighting their ability to provide detailed feedback and insights beyond mere correctness.The core of the discussion revolves around real-world applications and case studies in which LLMs have been implemented in educational assessments. These include automated grading systems and adaptive testing platforms, showing the effectiveness of LLMs in handling diverse and intricate responses. The outcomes of these implementations were analyzed, demonstrating LLMs potential in enhancing the accuracy, fairness, and efficiency of grading practices. However, the integration of LLMs into grading systems is challenging. This study critically examines issues such as potential biases in AI models, data privacy concerns, and the need to maintain ethical standards in automated grading. We propose strategies to mitigate these challenges by emphasizing the importance of human oversight and continuous model refinement.This study offers a forward-looking perspective on the future of grading practices that use LLMs. We envision a paradigm shift towards more personalized, fair, and efficient assessment methods, facilitated by the ongoing advancements in LLM technologies. The integration of LLMs into grading systems promises a more adaptive and insightful approach to educational assessment, aligning with the broader goals of personalized learning and educational equity.

show abstract

Section: Llm Based Approachesmentioning

confidence: 99%

Beyond Traditional Assessment: Exploring the Impact of Large Language Models on Grading Practices

Fagbohun,

Iduwe,

Abdullahi

et al. 2024

JAIMLD

View full text Add to dashboard Cite

show abstract

“…The corpus serves as the foundation for AEE research, so we depicted the usage of corpora in the ACEE research literature in Figure 4. Some scholars utilized large-scale exam corpora such as HSK (Hanyu Shuiping Kaoshi, the Chinese Language Proficiency Test) [18,[33][34][35][36] and MHK (Minzu Hanyu Kaoshi, Chinese Language Proficiency Test for Ethnic Minorities in China) [19,[37][38][39][40][41]. The HSK and MHK corpora contain essays written by NNS for Chinese proficiency assessments.…”

Section: Corpus Construction and Usage (Rq1)mentioning

confidence: 99%

“…It is commonly used in AEE for scoring tasks with small corpora. Besides these models, Bayesian regression [36], Bayesian linear ridge regression [62], and gradient boosting decision tree (GBDT) regression models help predict continuous numerical scores. The most commonly used scoring model is the ACEE's multiple linear regression (MLR) [18,19,33,65].…”

Section: Traditional Machine Learning Approachesmentioning

confidence: 99%

See 1 more Smart Citation

Automatic Essay Evaluation Technologies in Chinese Writing—A Systematic Literature Review

Yang,

He,

et al. 2023

Applied Sciences

View full text Add to dashboard Cite

Automatic essay evaluation, an essential application of natural language processing (NLP) technology in education, has been increasingly employed in writing instruction and language proficiency assessment. Because automatic Chinese Essay Evaluation (ACEE) has made some breakthroughs due to the rapid development of upstream Chinese NLP technology, many evaluation tools have been applied in teaching practice and high-risk evaluation processes. However, the development of ACEE is still in its early stages, with many technical bottlenecks and challenges. This paper systematically explores the current research status of corpus construction, feature engineering, and scoring models in ACEE through literature to provide a technical perspective for stakeholders in the ACEE research field. Literature research has shown that constructing the ACEE public corpus is insufficient and lacks an effective platform to promote the development of ACEE research. Various shallow and deep features can be extracted using statistical and NLP techniques in ACEE. However, there are still substantial limitations in extracting grammatical errors and features related to syntax and traditional Chinese Literary style. For the construction of scoring models, existing studies have shown that traditional machine learning and deep learning methods each have advantages in different corpora and feature selections. The deep learning model, which exhibits strong adaptability and multi-task joint learning potential, has broader development space regarding model scalability.

show abstract

“…Amidst the increasing importance of English as a global language in today's era of economic globalization and cultural diversity [1], understanding and correctly employing verb inflection, a fundamental aspect of English grammar, pose substantial challenges for English learners [2]. Despite advancements in large-scale data processing with algorithms like deep random forest [3], effectively classifying complex verb forms remains an issue [4]. Although approach improved model generalization and temporal performance by incorporating thematic and keyword features for verb inflection classification, overlooking word frequency information during feature extraction persists as a limitation [5].…”

Section: Introductionmentioning

confidence: 99%

Study on English verb classification based on K-means algorithm and support vector machine algorithm

Teng

2024

International Conference on Communication, Information, and Digital Technologies (CIDT2024)

View full text Add to dashboard Cite

Facing the challenge of English verb inflection classification, a technique combining K-means clustering algorithm with Support Vector Machine (SVM) is proposed to enhance the efficiency and accuracy of classification. This study establishes a semantic structure model of English verb inflection and accurately describes its semantic attributes, aiming to capture the state features. Quantification of similarity between these state feature quantities using cosine similarity evaluation method is performed. Experimental results demonstrate a high level of accuracy in calculating the similarity of state feature quantities, effectively classifying both regular and irregular changes in verb inflection. By setting the positive slack factor and error penalty parameters to 3000 and 9, respectively, further optimization of classification results is achieved, indicating potential application in the field. To ensure scientific rigor and authority, extensive citations from relevant literature are provided, elucidating differences from existing technologies and their advantages. Strict adherence to APA Seventh Edition citation and formatting standards solidifies the paper's academic contribution with a robust literature foundation

show abstract

Machine Learning-based Automated Essay Scoring System for Chinese Proficiency Test (HSK)

Cited by 7 publications

References 9 publications

Beyond Traditional Assessment: Exploring the Impact of Large Language Models on Grading Practices

Beyond Traditional Assessment: Exploring the Impact of Large Language Models on Grading Practices

Automatic Essay Evaluation Technologies in Chinese Writing—A Systematic Literature Review

Study on English verb classification based on K-means algorithm and support vector machine algorithm

Contact Info

Product

Resources

About