Earth Mover's Distance Pooling over Siamese LSTMs for Automatic Short Answer Grading

Kumar, Sachin; Chakrabarti, Soumen; Roy, Shourya

doi:10.24963/ijcai.2017/284

Cited by 48 publications

(31 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The task of automated content scoring has recently gained more attention (Kumar et al, 2017;Riordan et al, 2017;Burrows et al, 2015;Shermis, 2015). Our work is similar to Mizumoto et al (2019), who developed a multi-task neural model for assigning an overall holistic score as well as content-based analytic subscores.…”

Section: Related Workmentioning

confidence: 99%

An empirical investigation of neural methods for content scoring of science explanations

Riordan¹,

Bichler²,

Bradford³

et al. 2020

Proceedings of the Fifteenth Workshop on Innovative Use of NLP for Building Educational Applications

View full text Add to dashboard Cite

With the widespread adoption of the Next Generation Science Standards (NGSS), science teachers and online learning environments face the challenge of evaluating students' integration of different dimensions of science learning. Recent advances in representation learning in natural language processing have proven effective across many natural language processing tasks, but a rigorous evaluation of the relative merits of these methods for scoring complex constructed response formative assessments has not previously been carried out. We present a detailed empirical investigation of feature-based, recurrent neural network, and pre-trained transformer models on scoring content in real-world formative assessment data. We demonstrate that recent neural methods can rival or exceed the performance of feature-based methods. We also provide evidence that different classes of neural models take advantage of different learning cues, and pre-trained transformer models may be more robust to spurious, dataset-specific learning cues, better reflecting scoring rubrics.

show abstract

Section: Related Workmentioning

confidence: 99%

An empirical investigation of neural methods for content scoring of science explanations

Riordan¹,

Bichler²,

Bradford³

et al. 2020

Proceedings of the Fifteenth Workshop on Innovative Use of NLP for Building Educational Applications

View full text Add to dashboard Cite

show abstract

“…The task of automated content scoring has recently gained more attention (Kumar et al, 2017;Riordan et al, 2017;Shermis, 2015). Our work is similar to , who developed a multi-task neural model for assigning an overall holistic score as well as content-based analytic subscores.…”

Section: Related Workmentioning

confidence: 99%

Proceedings of the Fifteenth Workshop on Innovative Use of NLP for Building Educational Applications

2020

View full text Add to dashboard Cite

Readability assessment aims to automatically classify text by the level appropriate for learning readers. Traditional approaches to this task utilize a variety of linguistically motivated features paired with simple machine learning models. More recent methods have improved performance by discarding these features and utilizing deep learning models. However, it is unknown whether augmenting deep learning models with linguistically motivated features would improve performance further. This paper combines these two approaches with the goal of improving overall model performance and addressing this question. Evaluating on two large readability corpora, we find that, given sufficient training data, augmenting deep learning models with linguistically motivated features does not improve state-of-the-art performance. Our results provide preliminary evidence for the hypothesis that the state-of-theart deep learning models represent linguistic features of the text related to readability. Future research on the nature of representations formed in these models can shed light on the learned features and their relations to linguistically motivated ones hypothesized in traditional approaches.

show abstract

“…The problem of short answer grading has attracted significant attention of the researchers over the years. Various approaches, starting from traditional hand-crafted features (Mohler et al, 2011;Sultan et al, 2016) to more recent deep learning models (Riordan et al, 2017;Kumar et al, 2017) and their combination have been explored. However, similar to most downstream NLP tasks, ASAG also suffers from the overhead of task-specific architectures and thus scalability across different subjects has proven to be hard.…”

Section: Related Workmentioning

confidence: 99%

Pre-Training BERT on Domain Resources for Short Answer Grading

Sung

Dhamecha

Saha

et al. 2019

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conferen

View full text Add to dashboard Cite

Pre-trained BERT contextualized representations have achieved state-of-the-art results on multiple downstream NLP tasks by fine-tuning with task-specific data. While there has been a lot of focus on task-specific fine-tuning, there has been limited work on improving the pretrained representations. In this paper, we explore ways of improving the pre-trained contextual representations for the task of automatic short answer grading, a critical component of intelligent tutoring systems. We show that the pre-trained BERT model can be improved by augmenting data from the domainspecific resources like textbooks. We also present a new approach to use labeled short answering grading data for further enhancement of the language model. Empirical evaluation on multi-domain datasets shows that task-specific fine-tuning on the enhanced pretrained language model achieves superior performance for short answer grading.

show abstract

Earth Mover's Distance Pooling over Siamese LSTMs for Automatic Short Answer Grading

Cited by 48 publications

References 23 publications

An empirical investigation of neural methods for content scoring of science explanations

An empirical investigation of neural methods for content scoring of science explanations

Proceedings of the Fifteenth Workshop on Innovative Use of NLP for Building Educational Applications

Pre-Training BERT on Domain Resources for Short Answer Grading

Contact Info

Product

Resources

About