Relevance-Based Automated Essay Scoring via Hierarchical Recurrent Model

Chen, Minping; Li, Xia

doi:10.1109/ialp.2018.8629256

Cited by 20 publications

(6 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We try to contact the authors to provide information about sub-features used in their research, but there was no response. The system Tpioc-BiLSTM-attention [86] which works via hierarchical recurrent model does not provide any details about the features used in their published research. We also want to emphasize that extraction of a massive number of features vs. 23 features adds to time complexity as well.…”

Section: G Results and Discussionmentioning

confidence: 99%

Syntactic, Semantic and Sentiment Analysis: The Joint Effect on Automated Essay Evaluation

Janda

Pawar

et al. 2019

IEEE Access

View full text Add to dashboard Cite

Manual grading of essays by humans is time-consuming and likely to be susceptible to inconsistencies and inaccuracies. In recent years, an abundance of research has been done to automate essay evaluation processes, yet little has been done to take into consideration the syntax, semantic coherence and sentiments of the essay's text together. Our proposed system incorporates not just the rule-based grammar and surface level coherence check but also includes the semantic similarity of the sentences. We propose to use Graph-based relationships within the essay's content and polarity of opinion expressions. Semantic similarity is determined between each statement of the essay to form these Graph-based spatial relationships and novel features are obtained from it. Our algorithm uses 23 salient features with high predictive power, which is less than the current systems while considering every aspect to cover the dimensions that a human grader focuses on. Fewer features help us get rid of the redundancies of the data so that the predictions are based on more representative features and are robust to noisy data. The prediction of the scores is done with neural networks using the data released by the ASAP competition held by Kaggle. The resulting agreement between human grader's score and the system's prediction is measured using Quadratic Weighted Kappa (QWK). Our system produces a QWK of 0.793. INDEX TERMS Natural language processing, semantic analysis, sentiment analysis and text mining, automated essay evaluation.

show abstract

Section: G Results and Discussionmentioning

confidence: 99%

Syntactic, Semantic and Sentiment Analysis: The Joint Effect on Automated Essay Evaluation

Janda

Pawar

et al. 2019

IEEE Access

View full text Add to dashboard Cite

show abstract

“…We compared our HNN-AES model with the following baselines: LSTM-CNN-ATT (Dong et al, 2017), Topic-BiLSTM-attention (Chen & Li, 2018), and SSS-AES (Janda et al, 2019). LSTM-CNN-ATT obtains essay representations via a hierarchical sentence-document model.…”

Section: Accuracymentioning

confidence: 99%

Enhanced hybrid neural network for automated essay scoring

Xia

Yang

et al. 2022

Expert Systems

Self Cite

View full text Add to dashboard Cite

In an online learning system, the automatic scoring of an essay is key to providing immediate feedback on essays submitted by students. To the best of our knowledge, existing approaches ignore the multidimensional and heterogeneous characteristics of essays or rely too heavily on the manual creation of features; therefore, a more comprehensive method of scoring essays is required. To address this issue, this paper proposes an enhanced hybrid neural network for automated essay scoring that extracts and fuses the linguistic, semantic, and structural attributes of an essay to achieve a comprehensive representation. Specifically, linguistic attributes include not only lexical features extracted from the words of an essay but also syntactic features obtained from sentences and syntax trees. Semantic attributes include the dynamic textual semantic representation and topic similarity obtained by the text encoder.We also considered the structural attributes. The text encoder provides the overall structural representation, while the sentence similarity matrix provides the two spatial features of connectivity and aggregation. Finally, we fused the three attributes and six features to achieve a more objective and comprehensive automatic scoring.We found that our model improves the Kappa index by an average of 1.4% over the current best model when tested against four state-of-the-art models using eight public data sets.

show abstract

“…The workflow of machine learning framework reported in Taghipour and Ng (2016) is illustrated in Figure 4. Looking at the recent trends in AES, the machine learning framework is gaining popularity in recent years due to the efficacy of SVM (Ratna et al, 2019b;Ratna, et al, 2019a;Xu et al, 2017;Awaida et al, 2019;Chen & Li, 2018) and the ability to represent text context with word embedding (Liang et al, 2018;Taghipour & Ng, 2016) propelled by Artificial Neural Network (ANN) (Loraksa & Peachavanish, 2007;Taghipour & Ng, 2016;Dong & Zhang, 2016;Liang et al, 2018).…”

Section: Machine Learning Frameworkmentioning

confidence: 99%

A Comprehensive Review of Automated Essay Scoring (AES) Research and Development

Lim¹,

Bong²,

Wong³

et al. 2021

JST

View full text Add to dashboard Cite

Automated Essay Scoring (AES) is a service or software that can predictively grade essay based on a pre-trained computational model. It has gained a lot of research interest in educational institutions as it expedites the process and reduces the effort of human raters in grading the essays as close to humans’ decisions. Despite the strong appeal, its implementation varies widely according to researchers’ preferences. This critical review examines various AES development milestones specifically on different methodologies and attributes used in deriving essay scores. To generalize existing AES systems according to their constructs, we attempted to fit all of them into three frameworks which are content similarity, machine learning and hybrid. In addition, we presented and compared various common evaluation metrics in measuring the efficiency of AES and proposed Quadratic Weighted Kappa (QWK) as standard evaluation metric since it corrects the agreement purely by chance when estimate the degree of agreement between two raters. In conclusion, the paper proposes hybrid framework standard as the potential upcoming AES framework as it capable to aggregate both style and content to predict essay grades Thus, the main objective of this study is to discuss various critical issues pertaining to the current development of AES which yielded our recommendations on the future AES development.

show abstract

Relevance-Based Automated Essay Scoring via Hierarchical Recurrent Model

Cited by 20 publications

References 12 publications

Syntactic, Semantic and Sentiment Analysis: The Joint Effect on Automated Essay Evaluation

Syntactic, Semantic and Sentiment Analysis: The Joint Effect on Automated Essay Evaluation

Enhanced hybrid neural network for automated essay scoring

A Comprehensive Review of Automated Essay Scoring (AES) Research and Development

Contact Info

Product

Resources

About