ReaderBench: Automated evaluation of collaboration based on cohesion and dialogism

Abstract

Dascalu, M., Trausan-Matu, S., McNamara, D.S., & Dessus, P. (2015). ReaderBench – Automated Evaluation of Collaboration based on Cohesion and Dialogism. International Journal of Computer-Supported Collaborative Learning, 10(4), 395–423. doi: 10.1007/s11412-015-9226-yAs Computer-Supported Collaborative Learning (CSCL) gains a broader usage, the need for automated tools capable of supporting tutors in the time-consuming process of analyzing conversations becomes more pressing. Moreover, collaboration, which presumes the intertwining of ideas or points of view among participants, is a central element of dialogue performed in CSCL environments. Therefore, starting from dialogism and a cohesion-based model of discourse, we propose and validate two computational models for assessing collaboration. The first model is based on a cohesion graph and can be perceived as a longitudinal analysis of the ongoing conversation, thus accounting for collaboration from a social knowledge-building perspective. In the second approach, collaboration is regarded from a dialogical perspective as the intertwining or synergy of voices pertaining to different speakers, therefore enabling a transversal analysis of subsequent discussion slices.This study is part of the RAGE project. The RAGE project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 644187. This publication reflects only the author's view. The European Commission is not responsible for any use that may be made of the information it contains

Collaboration scripts and scaffolding

“…Both studies presented so far build on recent work in the area of automated collaborative process analysis (Rosé et al 2008;Mu et al 2012;Gweon et al 2013) and dynamic support for collaborative learning (Wang et al 2011;Adamson et al 2014), which have been of interest in this journal for some time (e.g., Berland et al 2015;Dascalu et al 2015;Erkens and Janssen 2008). This line for research gives news perspectives on what computational tools can do to support learning.…”

ReaderBench Learns Dutch: Building a Comprehensive Automated Essay Scoring System for Dutch Language

“…The framework provides a more in-depth perspective of discourse structure based on Cohesion Network Analysis [8,9], a multi-layered cohesion graph [31] that considers semantic links between different text constituents. We further describe the indices integrated in our framework and used for this study, categorized by their textual analysis scope.…”

“…In the research presented in this paper, we used the ReaderBench NLP framework [6,7], which integrates a wide range of metrics and techniques, covering both the cognitive and socio-cultural paradigms. ReaderBench makes extensive usage of Cohesion Network Analysis (CNA) [8,9] in order to represent discourse in terms of semantic links; this enables the computation of various local and global cohesion measures described later on. In addition, ReaderBench is grounded in Bakhtin's dialogism [10], which provides a unified framing for both individual and collaborative learning [9,11].…”

“…ReaderBench makes extensive usage of Cohesion Network Analysis (CNA) [8,9] in order to represent discourse in terms of semantic links; this enables the computation of various local and global cohesion measures described later on. In addition, ReaderBench is grounded in Bakhtin's dialogism [10], which provides a unified framing for both individual and collaborative learning [9,11].…”

What Makes Your Writing Style Unique? Significant Differences Between Two Famous Romanian Orators

“…Thus, the lack of cohesion flow can increase the difficulty of a text [30] as readers can easily loose interest by finding text segments too unrelated one to another. In order to evaluate local and global cohesion, our model uses Cohesion Network Analysis (CNA) [31] to compute cohesion as the average semantic similarity [32,33] at the following levels: intra-paragraph (between sentences of each paragraph), inter-paragraph (between any pair of paragraphs), or adjacency/transition from one paragraph or sentence to the next one. Cohesion between any two text segments is estimated as the average value of the cosine similarity in Latent Semantic Analysis (LSA) vector spaces [34,35] and the inverse of the Jensen Shannon dissimilarity (JSD) [36] between Latent Dirichlet Allocation (LDA) topic distributions [37,38].…”