Assessing Students’ Use of Evidence and Organization in Response-to-Text Writing: Using Natural Language Processing for Rubric-Based Automated Scoring

Rahimi, Zahra; Litman, Diane; Correnti, Richard; Wang, Elaine; Matsumura, Lindsay Clare

doi:10.1007/s40593-017-0143-2

Cited by 41 publications

(35 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Text mining has been largely used to evaluate different aspects of essays automatically (Dikli, 2006;Rahimi, Litman, Correnti, Wang, & Matsumura, 2017). Some researchers analyzed shallow features and error detection (Burstein, 2003).…”

Section: Essaysmentioning

confidence: 99%

See 1 more Smart Citation

Text mining in education

Mello

Andre

Pinheiro

et al. 2019

WIREs Data Min & Knowl

143

View full text Add to dashboard Cite

The explosive growth of online education environments is generating a massive volume of data, specially in text format from forums, chats, social networks, assessments, essays, among others. It produces exciting challenges on how to mine text data in order to find useful knowledge for educational stakeholders. Despite the increasing number of educational applications of text mining published recently, we have not found any paper surveying them. In this line, this work presents a systematic overview of the current status of the Educational Text Mining field. Our final goal is to answer three main research questions: Which are the text mining techniques most used in educational environments? Which are the most used educational resources? And which are the main applications or educational goals? Finally, we outline the conclusions and the more interesting future trends. This article is categorized under: Application Areas > Education and Learning Ensemble Methods > Text Mining

show abstract

Section: Essaysmentioning

confidence: 99%

“…the analysis of different algorithms for identification of prejudiced phrases in essays (Rahimi et al, 2017). Essay NLP Evaluation This paper presents an investigation of score prediction based on natural language processing for two targeted constructs within analytic text-based writing (Balyan et al, 2017).…”

mentioning

confidence: 99%

Text mining in education

Mello

Andre

Pinheiro

et al. 2019

WIREs Data Min & Knowl

143

View full text Add to dashboard Cite

show abstract

“…We have developed several AES systems for RTA assessment (Rahimi et al 2017;Zhang and Litman 2017;). Our first model (denoted by Rubric) (Rahimi et al 2017) used NLP to represent an essay in terms of features that largely correspond to cells in the RTA Evidence rubric. This rubric, as well as the correspondence between the rubric and features that serve as input to the scoring model, are shown in Table 2.…”

Section: Aes Feature Extractionmentioning

confidence: 99%

“…While we have developed pilot data-driven methods that can extract such topical components automatically (Rahimi and Litman 2016), our methods need to be improved so that they do not degrade SG model performance. eRevise will also be enhanced to provide feedback for Organization, a second substantive RTA writing dimension for which we already have a pilot AES (Rahimi et al 2017). We also plan to move from feedback selection to more personalized feedback generation, and to create a teacher dashboard which can automatically generate summaries such as Figure 3a.…”

Section: Current and Future Directionsmentioning

confidence: 99%

eRevise: Using Natural Language Processing to Provide Formative Feedback on Text Evidence Usage in Student Writing

Zhang

Magooda

Litman

et al. 2019

AAAI

View full text Add to dashboard Cite

Writing a good essay typically involves students revising an initial paper draft after receiving feedback. We present eRevise, a web-based writing and revising environment that uses natural language processing features generated for rubricbased essay scoring to trigger formative feedback messages regarding students' use of evidence in response-to-text writing. By helping students understand the criteria for using text evidence during writing, eRevise empowers students to better revise their paper drafts. In a pilot deployment of eRevise in 7 classrooms spanning grades 5 and 6, the quality of text evidence usage in writing improved after students received formative feedback then engaged in paper revision.

show abstract

“…The RTA was designed to evaluate writing skills in Analysis, Evidence, Organization, Style, and MUGS (Mechanics, Usage, Grammar, and Spelling) dimensions. To both score the RTA and provide formative feedback to students and teachers at scale, an automated RTA scoring tool is now being developed (Rahimi et al, 2017). This paper focuses on the Evidence dimension of the RTA, which evaluates students' ability to find and use evidence from an article to support their position.…”

Section: Introductionmentioning

confidence: 99%

Word Embedding for Response-To-Text Assessment of Evidence

Zhang

Litman

2017

Proceedings of ACL 2017, Student Research Workshop

Self Cite

View full text Add to dashboard Cite

Manually grading the Response to Text Assessment (RTA) is labor intensive. Therefore, an automatic method is being developed for scoring analytical writing when the RTA is administered in large numbers of classrooms. Our long-term goal is to also use this scoring method to provide formative feedback to students and teachers about students' writing quality. As a first step towards this goal, interpretable features for automatically scoring the evidence rubric of the RTA have been developed. In this paper, we present a simple but promising method for improving evidence scoring by employing the word embedding model. We evaluate our method on corpora of responses written by upper elementary students.

show abstract

Assessing Students’ Use of Evidence and Organization in Response-to-Text Writing: Using Natural Language Processing for Rubric-Based Automated Scoring

Cited by 41 publications

References 25 publications

Text mining in education

Text mining in education

eRevise: Using Natural Language Processing to Provide Formative Feedback on Text Evidence Usage in Student Writing

Word Embedding for Response-To-Text Assessment of Evidence

Contact Info

Product

Resources

About