Similarity measures in automated essay scoring systems: A ten-year review

Ramnarain-Seetohul, Vidasha; Bassoo, Vandana; Rosunally, Yasmine

doi:10.1007/s10639-021-10838-z

Cited by 8 publications

(5 citation statements)

References 94 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The system can provide immediate feedback on their written responses, allowing them to identify areas for improvement and adjust their study strategies accordingly. Likewise, a number of previous studies (e.g., He et al, 2022;Ramnarain-Seetohul et al, 2022) have established that AES systems effectively provide students with immediate feedback on their written responses. This feedback mechanism assists learners in recognizing areas for improvement, guiding their study strategies, and enhancing their writing skills (Phoophuangpairoj & Pipattarasakul, 2022).…”

Section: Discussionmentioning

confidence: 90%

“…Similarly, several empirical investigations have underscored the time-saving potential of AES systems. For example, the research by Hussein et al (2019), Phoophuangpairoj andPipattarasakul (2022), andRamnarain-Seetohul et al (2022) has shown that automated systems significantly reduce the time required for scoring and reporting, compared to manual grading methods. This efficiency is Contemporary Educational Technology, 2023 Contemporary Educational Technology, 15(4), ep475…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Exploring effective methods for automated essay scoring of non-native speakers

Poonpon,

Manorom,

Chansanam

2023

CONT ED TECHNOLOGY

View full text Add to dashboard Cite

Automated essay scoring (AES) has become a valuable tool in educational settings, providing efficient and objective evaluations of student essays. However, the majority of AES systems have primarily focused on native English speakers, leaving a critical gap in the evaluation of non-native speakers’ writing skills. This research addresses this gap by exploring the effectiveness of automated essay-scoring methods specifically designed for non-native speakers. The study acknowledges the unique challenges posed by variations in language proficiency, cultural differences, and linguistic complexities when assessing non-native speakers’ writing abilities. This work focuses on the automated student assessment prize and Khon Kaen University academic English language test dataset and presents an approach that leverages variants of the long short-term memory network model to learn features and compare results with the Kappa coefficient. The findings demonstrate that the proposed framework and approach, which involve joint learning of different essay representations, yield significant benefits and achieve results comparable to state-of-the-art deep learning models. These results suggest that the novel text representation proposed in this paper holds promise as a new and effective choice for assessing the writing tasks of non-native speakers. The result of this study can apply to advance educational assessment practices and promote equitable opportunities for language learners worldwide by enhancing the evaluation process for non-native speakers

show abstract

Section: Discussionmentioning

confidence: 90%

Section: Discussionmentioning

confidence: 99%

Exploring effective methods for automated essay scoring of non-native speakers

Poonpon,

Manorom,

Chansanam

2023

CONT ED TECHNOLOGY

View full text Add to dashboard Cite

show abstract

“…Studies on the use of artificial intelligence in education focus on various areas such as intelligent tutoring system (ITS), (Chen, 2008;Rastegarmoghadam & Ziarati, 2017), personalized learning (Chen & Hsu, 2008;Narciss et al, 2014;Zhou et al, 2018), assessment-feedback (Cope et al, 2021;Muñoz-Merino et al, 2018;Ramnarain-Seetohul et al, 2022;Ramesh & Sanampudi, 2022;Samarakou et al, 2016;Wang et al, 2018), educational data mining (Chen & Chen, 2009;Munir et al, 2022) and adaptive learning (Arroyo et al, 2014;Wauters et al, 2010;Kardan et al, 2015). These studies aim to improve the quality of the learning-teaching process by providing individualized learning experiences and increasing the effectiveness of teaching methods.…”

Section: Related Workmentioning

confidence: 99%

Effectiveness of artificial intelligence integration in design-based learning on design thinking mindset, creative and reflective thinking skills: An experimental study

Saritepeci,

Yildiz Durak

2024

Educ Inf Technol

View full text Add to dashboard Cite

Integrating Artificial Intelligence (AI) into learning activities is an essential opportunity to develop students' varied thinking skills. On the other hand, design-based learning (DBL) can more effectively foster creative design processes with AI technologies to overcome real-world challenges. In this context, AI-supported DBL activities have a significant potential for teaching and developing thinking skills. However, there is a lack of experimental interventions in the literature examining the effects of integrating AI into learner-centered methods on active engagement and thinking skills. The current study aims to explore the effectiveness of AI integration as a guidance and collaboration tool in a DBL process. In this context, the effect of the experimental application on the participants’ design thinking mindset, creative self-efficacy (CSE), and reflective thinking (RT) self-efficacy levels and the relationship between them were examined. The participants used ChatGPT and Midjourney in the digital story development process as part of the experimental treatment. The only difference between the control and experimental groups in the digital storytelling process is the AI applications used in the experimental treatment (ChatGPT and Midjourney). In this quasi-experimental method study, participants were randomly assigned to treatment, an AI integration intervention, at the departmental level. 87 participants (undergraduate students) in the experimental group and 99 (undergraduate students) in the control group. The implementation process lasted five weeks. Partial Least Squares (PLS), Structural Equation Modeling (SEM), and Multi-Group Analysis (MGA) were made according to the measurements made at the T0 point before the experiment and at the T1 point after the experiment. According to the research result, the intervention in both groups contributed to the creative self-efficacy, critical reflection, and reflection development of the participants. On the other hand, the design thinking mindset levels of both groups did not show a significant difference in the comparison of the T0 point and the T1 point.

show abstract

“…As technology-powered advances are being incorporated into large-scale writing assessments, automated essay scoring (AES) has received increasing attention, offering a viable alternative to the traditionally time-intensive and laborious manual grading processes [1][2][3]. Due to remarkable advances in corpus linguistics [4,5], natural language processing (NLP) [6,7], and deep learning [3,8,9], AES has the benefits of improved consistency, reduced subjectivity, and constructive feedback by exploiting extensive linguistic features or incorporating cutting-edging algorithms [10][11][12][13][14]. Given the importance of AES, it is unsurprising that the investigation into the power of linguistic features characterizing writing quality has become a critical focus within the domains of writing assessment and instruction in the past five decades.…”

Section: Introductionmentioning

confidence: 99%

Incorporating Fine-Grained Linguistic Features and Explainable AI into Multi-Dimensional Automated Writing Assessment

Tang,

Chen,

Lin

et al. 2024

Applied Sciences

View full text Add to dashboard Cite

With the flourishing development of corpus linguistics and technological revolutions in the AI-powered age, automated essay scoring (AES) models have been intensively developed. However, the intricate relationship between linguistic features and different constructs of writing quality has yet to be thoroughly investigated. The present study harnessed computational analytic tools and Principal Component Analysis (PCA) to distill and refine linguistic indicators for model construction. Findings revealed that both micro-features and their combination with aggregated features robustly described writing quality over aggregated features alone. Linear and non-linear models were thus developed to explore the associations between linguistic features and different constructs of writing quality. The non-linear AES model with Random Forest Regression demonstrated superior performance over other benchmark models. Furthermore, SHapley Additive exPlanations (SHAP) was employed to pinpoint the most powerful linguistic features for each rating trait, enhancing the model’s transparency through explainable AI (XAI). These insights hold the potential to substantially facilitate the advancement of multi-dimensional approaches toward writing assessment and instruction.

show abstract

Similarity measures in automated essay scoring systems: A ten-year review

Cited by 8 publications

References 94 publications

Exploring effective methods for automated essay scoring of non-native speakers

Exploring effective methods for automated essay scoring of non-native speakers

Effectiveness of artificial intelligence integration in design-based learning on design thinking mindset, creative and reflective thinking skills: An experimental study

Incorporating Fine-Grained Linguistic Features and Explainable AI into Multi-Dimensional Automated Writing Assessment

Contact Info

Product

Resources

About