Re-evaluating LARGO in the Classroom: Are Diagrams Better Than Text for Teaching Argumentation Skills?

Abstract

Abstract. Diagrams appear to be a convenient vehicle for teaching argumentation skills in ill-defined domains, but can an ITS provide useful feedback on students' argument diagrams without assuming a well-defined procedure for objectively evaluating argument? LARGO is an ITS for legal argumentation that supports students as they diagram transcripts of US Supreme Court oral argument. It provides on-demand advice by identifying small, interesting or incomplete patterns within students' graphs. We conducted a study in which LARGO was used as mandatory part of a first-year law school class. In contrast to prior findings in lab studies with voluntary participants, the use of LARGO did not lead to superior learning as compared to a text-based note-taking tool. These results can be partially attributed to low use of the graphical tools and advice by the students as well as (and possibly due to) a different motivational focus. Some evidence was found that higher engagement with the system led to better learning, leaving open the tantalizing possibility of helping especially lower-aptitude students through use of LARGO.

Computer-supported argumentation: A review of the state of the art

“…Fourth, on-demand feedback allows the tutoring component to appear less authoritative, possibly leading to less student discouragement ). On the downside, some students take minimal or even no advantage of on-demand feedback, even when they are stuck, as observed by Suthers et al (2001) and Pinkwart et al (2008b).…”

“…Hence, it is not clear whether the observed effects stem from the feedback, from the system's user interface, or from some other tool-specific factor. A first LARGO study (Pinkwart et al 2007) shows encouraging results in that low-aptitude students benefit from using LARGO; however, their second study (Pinkwart et al 2008b) did not confirm these results. This result illustrates the importance of contextual factors.…”

“…However, a post hoc analysis suggested some positive effects: Students requested feedback quite often and, perhaps more important, the use of feedback increased over time indicating that students found it helpful. Pinkwart et al (2008b) performed a second study in which they tested whether the results from the first experiment could be replicated in a more realistic setting. In this study, LARGO was used as a mandatory part of a first-year law school course, not by volunteers in a lab as in the first study.…”

“…The effect of adaptive support Pinkwart et al (2007Pinkwart et al ( , 2008b Using LARGO (and its feedback) to teach hypothetical reasoning Schank (1995) Using Convince Me (and its feedback) to improve students' reasoning skills that diagrams are more readable and comprehensible than prose. In particular, they usually require less interpretation; they can be understood more easily via colors, shapes, position in space, and other visual clues; and they are well suited to the non-sequential structure of most arguments.…”

Supporting Students in the Analysis of Case Studies for Professional Ethics Education

“…The analysis of graphs is more amenable to machine processing than the analysis of arguments represented in textual form, which allows more reliable diagnosis of students' knowledge. For example, Pinkwart et al [75] demonstrated that diagrams' features made by students in their Largo system are correlated with students' number of years in law school and students' standardized test scores that assess their ability to evaluate argumentation reasoning.…”