Enhanced hybrid neural network for automated essay scoring

Abstract

In an online learning system, the automatic scoring of an essay is key to providing immediate feedback on essays submitted by students. To the best of our knowledge, existing approaches ignore the multidimensional and heterogeneous characteristics of essays or rely too heavily on the manual creation of features; therefore, a more comprehensive method of scoring essays is required. To address this issue, this paper proposes an enhanced hybrid neural network for automated essay scoring that extracts and fuses the linguistic, semantic, and structural attributes of an essay to achieve a comprehensive representation. Specifically, linguistic attributes include not only lexical features extracted from the words of an essay but also syntactic features obtained from sentences and syntax trees. Semantic attributes include the dynamic textual semantic representation and topic similarity obtained by the text encoder.We also considered the structural attributes. The text encoder provides the overall structural representation, while the sentence similarity matrix provides the two spatial features of connectivity and aggregation. Finally, we fused the three attributes and six features to achieve a more objective and comprehensive automatic scoring.We found that our model improves the Kappa index by an average of 1.4% over the current best model when tested against four state-of-the-art models using eight public data sets.

Integration of Prediction Scores From Various Automated Essay Scoring Models Using Item Response Theory

“…As explained in Section I, recent automatic feature extraction approach models generally use DNNs. Many DNN-AES models have recently been proposed (e.g., [27]- [35], [37]- [44]).…”

“…Coherence is an important factor for determining the quality of essays. Thus, some DNN-AES models with a function to capture textual coherence explicitly have been proposed [31]- [35], [44]. A representative model is the SkipFlow model [32], an extension of the CNN-RNN-based model that incorporates a neural tensor layer, which explicitly captures textual coherence.…”

“…See Appendix C for details on the BERT-based AES. The BERT-based AES also has various extensions, such as those incorporating architectures to capture the textual coherence [43], [44], those extended toward multi-task learning [39], [77], and those using the DistilBERT [78], a variant of BERT [79].…”

“…Furthermore, there are other DNN-AES models based on transformer networks [36] instead of CNNs and RNNs. Such models [37]- [44] generally use pre-trained transformer-based language models, including Bidirectional Encoder Representations from Transformers (BERT) [45].…”

“…These DNN-AES models predict a score from the sequence of words in the essay, meaning that no manually designed features are required. However, some recent studies have proposed hybrid models that incorporate manually designed features into DNN-AES models [10], [25], [42], [44], [46] and have reported that such a hybrid approach is effective for improving scoring accuracy.…”