Computer-Automated Approach for Scoring Short Essays in an Introductory Statistics Course

Abstract

Over two semesters short essay prompts were developed for use with the Graphical Interface for Knowledge Structure (GIKS), an automated essay scoring system. Participants were students in an undergraduate-level online introductory statistics course. The GIKS compares students' writing samples with an expert's to produce keyword occurrence and links in common scores which can be used to construct a visual representation of an individual's knowledge structure. Each semester, students responded to the same two essay prompts during the first and last week of the course. All responses were scored by the GIKS and two instructors. Evidence for the validity of scores obtained using the GIKS was provided through the use of correlations with instructors' scores and final exam scores. Changes in scores from the beginning to end of the course were examined. Suggestions for writing open-ended prompts that work well with computer-automated scoring systems are given as well as suggestions for using the GIKS as a formative learning activity as opposed to summative assessment.

Contactless Academia – The Case for Automated Essay Scoring (AES) System in COVID 19 Pandemic

“…This review was conducted on published literatures searched between November 2019 and April 2020. Literatures were found by exploring Scopus, Google scholar, ERIC, EBSCO, PROQUEST and the Web of Science [2][3][4][5][6][7][8][9][10].…”

“…They further discovered that IEA and e-rater are superior choices for grading content, as PEG relies on writing quality to determine grades. Zimmerman examined Graphic Interface for Knowledge Structure (GIKS) and found that it was able to capture and visually represent the structure of semantic KS inherent in students' writings as a network graph that features key concepts and relations [9]. Giles investigated the impact of an automated grading system (SAGrader) on student's performance.…”

Improving summary writing through formative feedback in a technology‐enhanced learning environment

“…The SMART project was initiated to develop an advanced formative assessment and feedback technology based on the premise that a student's deep knowledge of a particular complex reading or a problem situation can be developed and measured through writing a high‐quality description of a text or a complex problem (Garnham, 1987, 2001; Johnson‐Laird, 2005; Kintsch, 1998; Westby, Culatta, Lawrence, & Hall‐Kenyon, 2010; Zimmerman et al, 2018). In short, students read a text and then write a summary of that text.…”

“…SMART elicits students' concept maps from their summaries as re‐represented their internal mental models. Thus, one can evaluate learner comprehension by examining multi‐dimensional aspects of concept maps (Clariana & Taricani, 2010; Gijbels, Dochy, den Bossche, & Segers, 2005; M. Kim et al, 2019; Zimmerman et al, 2018). A central assumption of the SMART design is that students' summary revisions facilitated by SMART feedback can indicate their evolving mental models towards the structure of the author's mind (Anzai & Yokoyama, 1984; Collins & Gentner, 1987; Johnson‐Laird, 2005; Pirnay‐Dummer & Ifenthaler, 2011; Seel, 2004; Smith et al, 1993).…”