Neural Automated Essay Scoring and Coherence Modeling for
            Adversarially Crafted Input

Farag, Youmna; Yannakoudakis, Helen; Briscoe, Ted

doi:10.18653/v1/n18-1024

Cited by 64 publications

(52 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Extensive research has been done on text coherence, motivated by downstream utility of coherence models. In addition to the applications we demonstrate in Section 4, established applications include determining the readability of a text (coherent texts are easier to read) (Barzilay and Lapata, 2008), refinement of multi-document summaries (Barzilay and Elhadad, 2002), and essay scoring (Farag et al, 2018).…”

Section: Related Workmentioning

confidence: 99%

Learning Sentence Embeddings for Coherence Modelling and Beyond

Bohn¹,

Hu²,

Zhang³

et al. 2019

Proceedings - Natural Language Processing in a Deep Learning World

View full text Add to dashboard Cite

We present a novel and effective technique for performing text coherence tasks while facilitating deeper insights into the data. Despite obtaining ever-increasing task performance, modern deep-learning approaches to NLP tasks often only provide users with the final network decision and no additional understanding of the data. In this work, we show that a new type of sentence embedding learned through self-supervision can be applied effectively to text coherence tasks while serving as a window through which deeper understanding of the data can be obtained. To produce these sentence embeddings, we train a recurrent neural network to take individual sentences and predict their location in a document in the form of a distribution over locations. We demonstrate that these embeddings, combined with simple visual heuristics, can be used to achieve performance competitive with state-of-the-art on multiple text coherence tasks, outperforming more complex and specialized approaches. Additionally, we demonstrate that these embeddings can provide insights useful to writers for improving writing quality and informing document structuring, and assisting readers in summarizing and locating information.

show abstract

Section: Related Workmentioning

confidence: 99%

Learning Sentence Embeddings for Coherence Modelling and Beyond

Bohn¹,

Hu²,

Zhang³

et al. 2019

Proceedings - Natural Language Processing in a Deep Learning World

View full text Add to dashboard Cite

show abstract

“…Taghipour (2017) suggests using contextfree grammars, and language modeling to create spurious essays, before trying to detect whether an input essay is spurious or not. Farag et al (2018) construct adversarial essays by permuting the sentences of good scoring essays.…”

Section: Adversarial Essaysmentioning

confidence: 99%

Thank “Goodness”! A Way to Measure Style in Student Essays

Mathias

Bhattacharyya

2018

Proceedings of the 5th Workshop on Natural Language Processing Techniques for Educational Applications

View full text Add to dashboard Cite

Essays have two major components for scoring-content and style. In this paper, we describe a property of the essay, called goodness, and use it to predict the score given for the style of student essays. We compare our approach to solve this problem with baseline approaches, such as language modeling and also a state-ofthe-art deep learning system, proposed by Taghipour and Ng (2016). We show that, despite being quite intuitive, our approach is very powerful in predicting the style of the essays.

show abstract

“…These problems (and the success of deep learning in other areas of language processing) have led to the development of neural methods for automatic essay scoring, moving away from feature engineering. A variety of studies (mostly LSTM-based) have reported AES performance comparable to or better than feature-based models (Taghipour and Ng, 2016;Cummins and Rei, 2018;Wang et al, 2018;Jin et al, 2018;Farag et al, 2018;Zhang and Litman, 2018). However, the current state-of-the-art models still use a combination of neural models and hand-crafted features (Liu et al, 2019).…”

Section: Introductionmentioning

confidence: 99%

Automated Essay Scoring with Discourse-Aware Neural Models

Nadeem¹,

Nguyen²,

Liu³

et al. 2019

Proceedings of the Fourteenth Workshop on Innovative Use of NLP for Building Educational Applications

View full text Add to dashboard Cite

Automated essay scoring systems typically rely on hand-crafted features to predict essay quality, but such systems are limited by the cost of feature engineering. Neural networks offer an alternative to feature engineering, but they typically require more annotated data. This paper explores network structures, contextualized embeddings and pre-training strategies aimed at capturing discourse characteristics of essays. Experiments on three essay scoring tasks show benefits from all three strategies in different combinations, with simpler architectures being more effective when less training data is available.

show abstract

Neural Automated Essay Scoring and Coherence Modeling for Adversarially Crafted Input

Cited by 64 publications

References 15 publications

Learning Sentence Embeddings for Coherence Modelling and Beyond

Learning Sentence Embeddings for Coherence Modelling and Beyond

Thank “Goodness”! A Way to Measure Style in Student Essays

Automated Essay Scoring with Discourse-Aware Neural Models

Contact Info

Product

Resources

About