Predicting Comprehension from Students’ Summaries

Dascălu, Mihai; Stavarache, Larise Lucia; Dessus, Philippe; Trăușan-Matu, Ștefan; McNamara, Danielle S.; Bianco, Maryse

doi:10.1007/978-3-319-19773-9_10

Cited by 5 publications

(3 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…From a different perspective, the ReaderBench framework has also been used to assess the textual complexity of texts by providing a wide range of complexity indices covering surface, lexical, syntactic and semantic levels of discourse Dascalu, Stavarache, et al, 2015). In future research, we will examine the assessment of learning and comprehension in the context of collaborative discourse using analogous indices adapted for chat conversation (characterized by short contributions).…”

Section: Discussionmentioning

confidence: 99%

ReaderBench: Automated evaluation of collaboration based on cohesion and dialogism

Dascălu

Trăușan-Matu

McNamara

et al. 2015

Intern. J. Comput.-Support. Collab. Learn

Self Cite

View full text Add to dashboard Cite

Dascalu, M., Trausan-Matu, S., McNamara, D.S., & Dessus, P. (2015). ReaderBench – Automated Evaluation of Collaboration based on Cohesion and Dialogism. International Journal of Computer-Supported Collaborative Learning, 10(4), 395–423. doi: 10.1007/s11412-015-9226-yAs Computer-Supported Collaborative Learning (CSCL) gains a broader usage, the need for automated tools capable of supporting tutors in the time-consuming process of analyzing conversations becomes more pressing. Moreover, collaboration, which presumes the intertwining of ideas or points of view among participants, is a central element of dialogue performed in CSCL environments. Therefore, starting from dialogism and a cohesion-based model of discourse, we propose and validate two computational models for assessing collaboration. The first model is based on a cohesion graph and can be perceived as a longitudinal analysis of the ongoing conversation, thus accounting for collaboration from a social knowledge-building perspective. In the second approach, collaboration is regarded from a dialogical perspective as the intertwining or synergy of voices pertaining to different speakers, therefore enabling a transversal analysis of subsequent discussion slices.This study is part of the RAGE project. The RAGE project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 644187. This publication reflects only the author's view. The European Commission is not responsible for any use that may be made of the information it contains

show abstract

Section: Discussionmentioning

confidence: 99%

ReaderBench: Automated evaluation of collaboration based on cohesion and dialogism

Dascălu

Trăușan-Matu

McNamara

et al. 2015

Intern. J. Comput.-Support. Collab. Learn

Self Cite

View full text Add to dashboard Cite

show abstract

“…When researchers compared three methods for identifying students' mental models of a topic from the paragraphs: content-based measures derived from LSA, cohesionbased measures from Coh-Metrix, and word-weighting features specially derived from their corpus of paragraphs; they found that the word-weighting features outperformed the other approaches. To provide another example, researchers have found that neither general indicators of reading strategies nor indicators of textual complexity were effective at predicting 3-5 th graders comprehension of stories, but a machine learning approach using a combination of some of these features was effective (Dascalu et al, 2015).…”

Section: Automated Scoring Using Machine Learningmentioning

confidence: 99%

Different Approaches to Assessing the Quality of Explanations Following a Multiple-Document Inquiry Activity in Science

Wiley

Hastings

Blaum

et al. 2017

Int J Artif Intell Educ

View full text Add to dashboard Cite

This article describes several approaches to assessing student understanding using written explanations that students generate as part of a multiple-document inquiry activity on a scientific topic (global warming). The current work attempts to capture the causal structure of student explanations as a way to detect the quality of the students' mental models and understanding of the topic by combining approaches from Cognitive Science and Artificial Intelligence, and applying them to Education. First, several attributes of the explanations are explored by hand-coding and leveraging existing technologies (LSA and Coh-Metrix). Then, we describe an approach for inferring the quality of the explanations using a novel, two-phase machine learning approach for detecting causal relations and the causal chains that are present within student essays. The results demonstrate the benefits of using a machine learning approach for detecting content, but also highlight the promise of hybrid methods that combine ML, LSA and Coh-Metrix approaches for detecting student understanding. Opportunities to use automated approaches as part of Intelligent Tutoring Systems that provide feedback toward improving student explanations and understanding are discussed.

show abstract

“…Magliano and Millis (2003) found that LSA variables used to measure semantic similarity between verbal text comprehension prompts and undergraduate students' responses to those prompts predicted scores on a comprehension test. Other tasks for which automated scoring has been attempted include open-ended short-answer questions (Brew & Leacock, 2013), computer-generated cloze tasks whereby a student must identify the missing word from a sentence in a passage (Stenner, Fisher, Stone, & Burdick, 2013), and students' summaries of texts (Dascalu et al, 2015).…”

Section: Previous Research On Automated Scoring In Reading and Writingmentioning

confidence: 99%

Automated Scoring of Students’ Small‐Group Discussions to Assess Reading Ability

Kosh

Greene

Murphy

et al. 2017

Educational Measurement

View full text Add to dashboard Cite

We explored the feasibility of using automated scoring to assess upper‐elementary students’ reading ability through analysis of transcripts of students’ small‐group discussions about texts. Participants included 35 fourth‐grade students across two classrooms that engaged in a literacy intervention called Quality Talk. During the course of one school year, data were collected at 10 time points for a total of 327 student‐text encounters, with a different text discussed at each time point. To explore the possibility of automated scoring, we considered which quantitative discourse variables (e.g., variables to measure language sophistication and latent semantic analysis variables) were the strongest predictors of scores on a multiple‐choice and constructed‐response reading comprehension test. Convergent validity evidence was collected by comparing automatically calculated quantitative discourse features to scores on a reading fluency test. After examining a variety of discourse features using multilevel modeling, results showed that measures of word rareness and word diversity were the most promising variables to use in automated scoring of students’ discussions.

show abstract

Predicting Comprehension from Students’ Summaries

Cited by 5 publications

References 18 publications

ReaderBench: Automated evaluation of collaboration based on cohesion and dialogism

ReaderBench: Automated evaluation of collaboration based on cohesion and dialogism

Different Approaches to Assessing the Quality of Explanations Following a Multiple-Document Inquiry Activity in Science

Automated Scoring of Students’ Small‐Group Discussions to Assess Reading Ability

Contact Info

Product

Resources

About