Autoscoring Essays Based on Complex Networks

Abstract

This article presents a novel method, the Complex Dynamics Essay Scorer (CDES), for automated essay scoring using complex network features. Texts produced by college students in China were represented as scale-free networks (e.g., a word adjacency model) from which typical network features, such as the in-/out-degrees, clustering coefficient (CC), and dynamic networks, were obtained. The CDES integrates the classical concepts of network feature representation and essay score series variation. Several experiments indicated that the network measures different essay qualities and can be clearly demonstrated to develop complex networks for autoscoring tasks. The average agreement of the CDES and human rater scores was 86.5%, and the average Pearson correlation was .77. The results indicate that the CDES produced functional complex systems and autoscored Chinese essays in a method consistent with human raters. Our research suggests potential applications in other areas of educational assessment. 478

Predicting Evaluations of Essay by Computational Graph-Based Features

“…Table 7 shows that the number of concepts contributed 13% to the essay's predicted holistic score, 6% to the main idea, 14% to the content, and 10% to the coherence. In addition, from the perspective of the graph structure, the edges of the concept graph in this study have syntactic relationships between concepts, which reveal more information about the logical language relationship than the simple co-occurrence relationship (Ke et al, 2016;Somasundaran et al, 2016) used in previous research. Table 6 shows that the number of edges based on the grammatical relationship is also positively correlated with the essay score, and the predicted contribution range of the score is 5-9%.…”

“…In this study, the Big Cilin was used to merge the synonyms. Unlike previous studies (Ke et al, 2016;Somasundaran et al, 2016;Zupanc and Bosnic, 2017), a set of concepts in an essay were extracted instead of the original words due to the fact that there is no concept recognition and/or synonym merging, and the central node in the network is likely to be a preposition (Poiret and Liu, 2019) that has little contribution to the essay's ideas. Please note that a systematic evaluation of synonym-merging is not presented in this paper.…”

“…The research results show that the graph structure-based method can improve the scoring accuracy of the overall quality and the development of ideas. Ke et al (2016) used complex networks to score Chinese essays and adopted in-/out-degrees, clustering coefficient, and dynamic network features. However, these studies are all based on lexical networks, the relationship between the features based on the graph, and the traits of the essay evaluation criteria was not explored systematically.…”

“…The degree of a node refers to the number of edges associated with the node. The higher the degree, the more important the node is in the essay (Ke et al, 2016). In a graph constructed based on syntactic relationships, each node has an average syntactic relationship with three to five other nodes.…”

“…In this study, we propose a graph-based 1 text analysis technique to enhance existing state-of-the-art AES systems by incorporating novel features for measuring an essay's main idea, content, and development. Graph-based text analysis offers further possibilities for AES because of previously demonstrated close relation with text quality (Ke et al, 2016;Somasundaran et al, 2016;Zupanc and Bosnic, 2017). However, the features used in these studies are relatively simple and fragmented, and few studies, especially pertaining to essays written in Chinese, have explored the relationship between graph characteristics and essay quality systematically based on a scoring rubric.…”