An Unsupervised Automated Essay Scoring System

Chen, Yen‐Yu; Liu, Chien‐Liang; Lee, Chia-Hoang; Chang, Tao-Hsing

doi:10.1109/mis.2010.3

Cited by 32 publications

(27 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These approaches are based on deep architectures, and include (Alikaniotis et al, 2016;Taghipour and Ng, 2016;Riordan et al, 2017;Dong et al, 2017). Finally, there also models based on domain adaptation (Phandi et al, 2015) and unsupervised learning (Chen et al, 2010).…”

Section: Related Workmentioning

confidence: 99%

Automated Essay Scoring in the Presence of Biased Ratings

Amorim¹,

Cançado²,

Veloso³

2018

Proceedings of the 2018 Conference of the North American Chapter Of the Association for Computational Linguistics: Hu

View full text Add to dashboard Cite

Studies in Social Sciences have revealed that when people evaluate someone else, their evaluations often reflect their biases. As a result, rater bias may introduce highly subjective factors that make their evaluations inaccurate. This may affect automated essay scoring models in many ways, as these models are typically designed to model (potentially biased) essay raters. While there is sizeable literature on rater effects in general settings, it remains unknown how rater bias affects automated essay scoring. To this end, we present a new annotated corpus containing essays and their respective scores. Different from existing corpora, our corpus also contains comments provided by the raters in order to ground their scores. We present features to quantify rater bias based on their comments, and we found that rater bias plays an important role in automated essay scoring. We investigated the extent to which rater bias affects models based on hand-crafted features. Finally, we propose to rectify the training set by removing essays associated with potentially biased scores while learning the scoring model.

show abstract

Section: Related Workmentioning

confidence: 99%

Automated Essay Scoring in the Presence of Biased Ratings

Amorim¹,

Cançado²,

Veloso³

2018

Proceedings of the 2018 Conference of the North American Chapter Of the Association for Computational Linguistics: Hu

View full text Add to dashboard Cite

show abstract

“…What's more, early study focused mainly on the introduction of major AES systems developed in the USA and other countries [5] [8] [10] [13]. Nevertheless, with the increasing development of Chinese science and technology, more and more researchers set about researching and developing their own AES systems which are more suitable to Chinese learners.…”

Section: Literaturementioning

confidence: 99%

Experimental Study on the Validity of AES Systems in the College EFL Classroom

Li¹

2014

Proceedings of the 2nd International Conference on Teaching and Computational Science

View full text Add to dashboard Cite

Abstract-Online Automated Essay Scoring (AES) is used to promote the teaching of EFL writing partly, but it can't work very effectively for some reasons, mainly including misuse and the shortcomings of AES itself. The study tries to investigate the validity of AES by analyzing a questionnaire survey by means of a statistical tool SPSS and discusses the problems existing in the use of AES and the shortcomings in AES itself. Then accordingly some suggestions are presented on how to use AES more effectively so as to enhance students' stimulus and motivation in writing and improve their writing ability and on how to improve AES.

show abstract

“…The field of computational linguistics often uses adjacent‐level accuracy, whereby predicting a text to be within one level of the level assigned to it by experts is still considered accurate. This method allows for a small finite prediction error and has been implemented in many studies (Chen et al., ; François & Fairon, ; Heilman, Collins–Thompson, & Eskenazi, ). Moreover, some CFL teaching materials have been designed and labeled with adjacent levels (e.g., Arslangul et al., ).…”

Section: Resultsmentioning

confidence: 99%

Leveling L2 Texts Through Readability: Combining Multilevel Linguistic Features with the CEFR

Sung

Lin

Dyson

et al. 2015

The Modern Language Journal

View full text Add to dashboard Cite

Selecting appropriate texts for L2 (second/foreign language) learners is an important approach to enhancing motivation and, by extension, learning. There is currently no tool for classifying foreign language texts according to a language proficiency framework, which makes it difficult for students and educators to determine the precise difficulty/complexity levels of an unclassified text. Taking the Chinese language as an example, this study aimed to create a readability assessment system, called the Chinese Readability Index Explorer for Chinese as a Foreign Language (CRIE-CFL), in order to level-that is, to sort by proficiency level-texts that will be used for instructional purposes. The framework of choice in this project is the Common European Framework of Reference (CEFR). A team of expert CFL teachers first classified 1,578 CFL texts into their appropriate CEFR levels. A set of 30 CFL readability features was then developed or drawn from previous research, and sorted according to importance using F-scores. In addition, a support vector machine model was trained by sequentially integrating the features into the model to optimize accuracy. The empirical evaluation of CRIE-CFL revealed average exact-and adjacent-level accuracies of 74.97% and 99.62%, respectively, for predicting the expert classification of a text. The functionalities of CRIE-CFL are introduced and discussed.

show abstract

An Unsupervised Automated Essay Scoring System

Cited by 32 publications

References 3 publications

Automated Essay Scoring in the Presence of Biased Ratings

Automated Essay Scoring in the Presence of Biased Ratings

Experimental Study on the Validity of AES Systems in the College EFL Classroom

Leveling L2 Texts Through Readability: Combining Multilevel Linguistic Features with the CEFR

Contact Info

Product

Resources

About