Coherence-Based Automated Essay Scoring Using Self-attention

Li, Xia; Chen, Minping; Nie, Jian‐Yun; Liu, Zhenxing; Feng, Ziheng; Cai, Yingdan

doi:10.1007/978-3-030-01716-3_32

Cited by 14 publications

(7 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…See Appendix C for details on the BERT-based AES. The BERT-based AES also has various extensions, such as those incorporating architectures to capture the textual coherence [43], [44], those extended toward multi-task learning [39], [77], and those using the DistilBERT [78], a variant of BERT [79].…”

Section: B Automatic Feature Extraction Approachmentioning

confidence: 99%

Integration of Prediction Scores From Various Automated Essay Scoring Models Using Item Response Theory

Uto

Aomi

Tsutsumi

et al. 2023

IEEE Trans. Learning Technol.

View full text Add to dashboard Cite

In automated essay scoring (AES), essays are automatically graded without human raters. Many AES models based on various manually designed features or various architectures of deep neural networks have been proposed over the past few decades. Each AES model has unique advantages and characteristics. Therefore, rather than using a single AES model, appropriate integration of predictions from various AES models is expected to achieve higher scoring accuracy. In the present paper, we propose a method that uses item response theory to integrate prediction scores from various AES models while taking into account differences in the characteristics of scoring behavior among models. It is found that the proposed method achieves higher accuracy than that of individual AES models and conventional score-integration methods. Furthermore, the proposed method facilitates interpreting each AES model's scoring characteristics and score-integration mechanism.

show abstract

Section: B Automatic Feature Extraction Approachmentioning

confidence: 99%

Integration of Prediction Scores From Various Automated Essay Scoring Models Using Item Response Theory

Uto

Aomi

Tsutsumi

et al. 2023

IEEE Trans. Learning Technol.

View full text Add to dashboard Cite

show abstract

“…Simply put, the frequency of a term in a text determines its weight. Coherent essays are presumed to have a high degree of proximity between points in high-dimensional semantic space that are characterized by the TF − IDF vectors of essay parts [9,10,49,50].…”

Section: Semantic Spacementioning

confidence: 99%

“…Herein each essay's part will produce a vector that contains five elements. The final product of the coherence analysis will be a vector of five elements that represent the average of the corresponding elements of each part [49,50].…”

mentioning

confidence: 99%

An Automated English Essay Scoring Engine Based on Neutrosophic Ontology for Electronic Education Systems

2023

View full text Add to dashboard Cite

Most educators agree that essays are the best way to evaluate students’ understanding, guide their studies, and track their growth as learners. Manually grading student essays is a tedious but necessary part of the learning process. Automated Essay Scoring (AES) provides a feasible approach to completing this process. Interest in this area of study has exploded in recent years owing to the difficulty of simultaneously improving the syntactic and semantic scores of an article. Ontology enables us to consider the semantic constraints of the actual world. However, there are several uncertainties and ambiguities that cannot be accounted for by standard ontologies. Numerous AES strategies based on fuzzy ontologies have been proposed in recent years to reduce the possibility of imprecise knowledge presentation. However, no known efforts have been made to utilize ontologies with a higher level of fuzzification in order to enhance the effectiveness of identifying semantic mistakes. This paper presents the first attempt to address this problem by developing a model for efficient grading of English essays using latent semantic analysis (LSA) and neutrosophic ontology. In this regard, the presented work integrates commonly used syntactic and semantic features to score the essay. The integration methodology is implemented through feature-level fusion. This integrated vector is used to check the coherence and cohesion of the essay. Furthermore, the role of neutrosophic ontology is investigated by adding neutrosophic membership functions to the crisp ontology to detect semantic errors and give feedback. Neutrosophic logic allows the explicit inclusion of degrees of truthfulness, falsity, and indeterminacy. According to the comparison with state-of-the-art AES methods, the results show that the proposed model significantly improves the accuracy of scoring the essay semantically and syntactically and is able to provide feedback.

show abstract

“…Based on this point, Mesgar et al [7] introduced a local coherence model to obtain the flow of content that semantically connects adjacent sentences in the essay. Li et al [8] employed the self-attention mechanism to learn the relationship between long-distance words in the essay to estimate coherence scores.…”

Section: Introductionmentioning

confidence: 99%

Exploring Relevance and Coherence for Automated Text Scoring using Multi-task Learning

Yang¹,

Zhong²,

Wang³

et al. 2022

Proceedings of the 34th International Conference on Software Engineering and Knowledge Engineering

View full text Add to dashboard Cite

With the explosive growth of the information on the Internet, the evaluation of the quality and credibility of web content has become more important than ever before. In this work, we focus on the quality assessment of texts. Recently, various methods have been proposed for the automated text scoring task and obtained competitive results. However, few studies have focused on both relevance and coherence, which are two important factors in evaluating text quality. To improve the scoring task, we propose two auxiliary tasks using negative sampling and integrate them into a multi-task learning framework. The first auxiliary task is relevance modeling and the other one is coherence modeling. We evaluate our model on the Automated Student Assessment Prize (ASAP) dataset. Experimental results show that our model achieves higher Quadratic Weighted Kappa (QWK) scores with an improvement of 1.5% on average.

show abstract

Coherence-Based Automated Essay Scoring Using Self-attention

Cited by 14 publications

References 19 publications

Integration of Prediction Scores From Various Automated Essay Scoring Models Using Item Response Theory

Integration of Prediction Scores From Various Automated Essay Scoring Models Using Item Response Theory

An Automated English Essay Scoring Engine Based on Neutrosophic Ontology for Electronic Education Systems

Exploring Relevance and Coherence for Automated Text Scoring using Multi-task Learning

Contact Info

Product

Resources

About