Analytic Frameworks for Assessing Dialogic Argumentation in Online Learning Environments

Abstract

Over the last decade, researchers have developed sophisticated online learning environments to support students engaging in argumentation. This review first considers the range of functionalities incorporated within these online environments. The review then presents five categories of analytic frameworks focusing on (1) formal argumentation structure, (2) normative quality, (3) nature and function of contributions within the dialog, (4) epistemic nature of reasoning, and (5) patterns and trajectories of participant interaction. Example analytic frameworks from each category are presented in detail rich enough to illustrate their nature and structure. This rich detail is intended to facilitate researchers' identification of possible frameworks to draw upon in developing or adopting analytic methods for their own work. Each framework is applied to a shared segment of student dialog to facilitate this illustration and comparison process. Synthetic discussions of each category consider the frameworks in light of the underlying theoretical perspectives on argumentation, pedagogical goals, and online environmental structures. Ultimately the review underscores the diversity of perspectives represented in this research, the importance of clearly specifying theoretical and environmental commitments throughout the process of developing or adopting an analytic framework, and the role of analytic frameworks in the future development of online learning environments for argumentation.

Learning Analytics for Online Discussions: Embedded and Extracted Approaches

“…In terms of speaking quality, posts that are clear, critical, and connected to the existing conversation support the generation of insight and understanding of the topic (Rovai, 2007). Posts whose arguments are based on evidence and/or theory can also trigger others to build on or contest the points productively (Clark, Sampson, Weinberger, & Erkens, 2007), and responses that clarify points, elaborate or question existing ideas, or synthesize different ideas together help deepen the exploration of ideas and move the discussion forward (Pena-Shaff & Nicholls, 2004). In terms of listening activity, there are also multiple dimensions requiring attention.…”

Effects of representational guidance during computer-supported collaborative learning

“…The distinction between grounds quality (how well students used evidence or examples to support their claims) and conceptual quality (how conceptually correct the arguments were) is made by several authors (e.g., Clark et al 2007;Munneke et al 2007). These authors argue that examining grounds quality gives important information about how well students develop and support their arguments with evidence and explanations, but provides no details about the validity and conceptual quality of these arguments.…”

“…The evidence provided by students to back up the claims and opinions in their texts formed the starting point for the analyses of grounds quality. Each text segment was judged in terms of how well and how elaborately it was supported by evidence or explanations (Clark et al 2007). Grounds quality of each segment was rated on a 4-point scale, ranging from 0 to 3.…”

“…The conceptual adequacy of the arguments given by the students, constituted the basis for the analyses of conceptual quality (Clark et al 2007). Each segment was judged in terms of its conceptual correctness; thus segments that contained, for example, flawed conclusions, misinterpretations, or incorrect statements, received lower scores for conceptual quality than did segments that contained no errors.…”

Discourse Centric Learning Analytics: Mapping the Terrain

“…That review, and an earlier review (Clark, Sampson, Weinberger, & Erkens, 2007) of argumentation in Computer-Supported Collaborative Learning (CSCL) environments, largely focus on existing argumentation frameworks -that is, they take the "topdown" theory-driven approach. However, it is important to note the potential of DCLA approaches for contributing back to theory, in -well conceptualized -bottom-up, data-driven approaches, as we discuss in the section "Top-down and Bottom-up Approaches" below.…”