Correlations between dialogue acts and learning in spoken tutoring dialogues

Abstract

We examine correlations between dialogue behaviors and learning in tutoring, using two corpora of spoken tutoring dialogues: a human-human corpus and a human-computer corpus. To formalize the notion of dialogue behavior, we manually annotate our data using a tagset of student and tutor dialogue acts relative to the tutoring domain. A unigram analysis of our annotated data shows that student learning correlates both with the tutor's dialogue acts and with the student's dialogue acts. A bigram analysis shows that student learning also correlates with joint patterns of tutor and student dialogue acts. In particular, our human-computer results show that the presence of student utterances that display reasoning (whether correct or incorrect), as well as the presence of reasoning questions asked by the computer tutor, both positively correlate with learning. Our human-human results show that student introductions of a new concept into the dialogue positively correlates with learning, but student attempts at deeper reasoning (particularly when incorrect), and the human tutor's attempts to direct the dialogue, both negatively correlate with learning. These results suggest that while the use of dialogue act n-grams is a promising method for examining correlations between dialogue behavior and learning, specific findings can differ in human versus computer tutoring, with the latter better motivating adaptive strategies for implementation.

Designing and evaluating a wizarded uncertainty-adaptive spoken dialogue tutoring system

“…In fact, it may be that different behaviors are actually optimally effective in computer and human tutors. This hypothesis is supported by our prior research, which has shown that although our students learn significantly from both our human tutor and ITSPOKE, their behaviors are very different (Forbes-Riley and Litman, 2008;Litman and Forbes-Riley, 2006b). However, we do not want to conclude that human tutor-based affect adaptations are less effective in general, because although our Complex adaptation was derived from statistical generalizations about human tutor responses to uncertainty, the effectiveness of these responses was not empirically tested before implementation.…”

Exploring the Effects of Redundancy within a Tutorial Dialogue System: Restating Students' Responses

“…Several recent studies of human tutorial dialogue have looked at particular aspects of restatements, for example, (Chi and Roy, 2010;Becker et al, 2011;Dzikovska et al, 2008;Litman and Forbes-Riley, 2006). One study examines face-toface naturalistic tutorial dialogue in which a tutor helps a student work through a physics problem (Chi and Roy, 2010).…”

Investigating Human Tutor Responses to Student Uncertainty for Adaptive System Development

“…Each dialogue contains 47 student turns and 43 tutor turns on average. This corpus was collected in tandem with a computer tutoring corpus using our ITSPOKE spoken dialogue tutoring system; the human tutor and ITSPOKE performed the same task [11]. Each dialogue consists of a question-answer discussion between tutor and student about one qualitative physics problem.…”

“…Here we distinguish two labels 3 : the uncertain label is used for answers expressing uncertainty or confusion about the material being learned, and the non-uncertain label is used for all other answers. The same annotator also manually labeled each student answer for correctness, based on the human tutor's response to the answer [11]. Here we distinguish two labels 4 : the correct label is used for answers the tutor considered to be wholly or partly correct, and the incorrect label is used for answers the tutor considered to be wholly incorrect.…”