The Effects of Explanations in Automated Essay Scoring Systems on Student Trust and Motivation

Abstract

Ethical considerations, including transparency, play an important role when using artificial intelligence (AI) in education. Explainable AI has been coined as a solution to provide more insight into the inner workings of AI algorithms. However, carefully designed user studies on how to design explanations for AI in education are still limited. The current study aimed to identify the effect of explanations of an automated essay scoring system on students’ trust and motivation. The explanations were designed using a needs-elicitation study with students in combination with guidelines and frameworks of explainable AI. Two types of explanations were tested: full-text global explanations and an accuracy statement. The results showed that both explanations did not have an effect on student trust or motivation compared to no explanations. Interestingly, the grade provided by the system, and especially the difference between the student’s self-estimated grade and the system grade, showed a large influence. Hence, it is important to consider the effects of the outcome of the system (here: grade) when considering the effect of explanations of AI in education.

Assessing Perceived Trust and Satisfaction with Multiple Explanation Techniques in XAI-Enhanced Learning Analytics

“…The trust evaluation method utilized in our study allowed us to evaluate the initial user trust in a more detailed view. As no control group was included in our study, our findings are not directly comparable to [28], who reported no difference in trust between students who observed any of the two explanations and those in the control group. The results of this study have contradicted the existing literature on LA systems about the recognized effect that explanations have on user trust.…”

“…The need for the interpretability of the complex models used in education mining data techniques has been highlighted [26], and explanations of the model's predictions have been introduced slowly, by [27] offering verbal explanations (i.e., "Evaluation is Pass because the number of assessments is high"), and by [10] offering verbal and visual explanations to students. In a related study, Conijn et al [28] analyzed the effects of explanations of an automated essay scoring system on students' trust and motivation in the context of higher education. The results indicated there is no one-size-fits-all explanation for different stakeholders and in different contexts.…”

“…Some related work has been conducted. Conijn et al [28] analyzed the effects of explanations of an automated essay scoring system on students' trust and motivation in the context of higher education, observing two types of explanations: full-text global explanations and model accuracy statements. Ooge et al [10] focused on measuring multidimensional trust in XAI e-learning systems with adolescents.…”

The Impact of Artificial Intelligence Tools on Academic Writing Instruction in Higher Education: A Systematic Review

“…Overreliance on AI may reduce students' intrinsic motivation to write autonomously, limiting their creative and independent writing. Conijn et al, 2023;Gayed et al, 2022;Ouyang et al, 2022 Limited transferable skills AI-generated work may lack the critical thinking and analysis that companies and graduate programs value.…”

“…The significance of this study lies in the need to critically examine the role of AI in an essential component of university curricula -developing students' abilities in academic writing and composition. Strong writing skills are foundational for communicating ideas effectively, conducting research, critically analyzing texts, and succeeding in many educational and professional domains (Conijn et al, 2023;Zhai & Ma, 2022). However, the rise of AI writing assistants raises concerns about an overreliance on technologies that may impede the fostering of these crucial skills.…”