Investigating Human Tutor Responses to Student Uncertainty for Adaptive System Development

Abstract

Abstract. We use a χ 2 analysis on our spoken dialogue tutoring corpus to investigate dependencies between uncertain student answers and 9 dialogue acts the human tutor uses in his response to these answers. Our results show significant dependencies between the tutor's use of some dialogue acts and the uncertainty expressed in the prior student answer, even after factoring out the answer's (in)correctness. Identification and analysis of these dependencies is part of our empirical approach to developing an adaptive version of our spoken dialogue tutoring system that responds to student affective states as well as to student correctness.

Benefits and challenges of real-time uncertainty detection and adaptation in a spoken dialogue computer tutor

“…In particular, we believe automatic correctness features will be useful since correctness and uncertainty are related [23,24,25]. We plan to include the correctness value of the current turn and a running total and average for correctness to represent dialogue performance so far.…”

“…Although uncertainty is not one of the "big 6" emotions such as anger and happiness [21], tutoring research suggests uncertainty and closely related states such as confusion and self-efficacy are among the most frequently occurring affective states in tutoring dialogues (e.g., [22,23,4]), and they play an important role in the learning process [23,24,25]. Due to the complexity of the information exchange in a tutoring dialogue, most dynamic affect-adaptive tutoring systems -that is, systems that recognize and respond to student affect on a turn by turn basis -have been evaluated within a "Wizard of Oz" scenario, where an unseen human "wizard" performs tasks such as speech recognition, natural language understanding, and affect detection or adaptation.…”

Individual Differences

“…This includes work on variation using parameters based on pragmatic effects (Hovy 1988;Fleischman and Hovy 2002), stylistic factors such as formality, sentence length, and syntactic structure (Power et al 2003;Bouayad-Agha et al 2000;DiMarco and Hirst 1993;Green and DiMarco 1996;Paiva and Evans 2005;Belz 2005;Walker et al 2002;Paris and Scott 1994), emotion (Cahn 1990), lexical choice (Inkpen and Hirst 2004), user expertise or confidence (Porayska-Pomsta and Mellish 2004;Wang et al 2005;DiMarco and Hirst 1993;Forbes-Riley and Litman 2007;Forbes-Riley et al 2008), a theory of linguistic politeness (Brown and Levinson 1987;Walker et al 1997;Gupta et al 2007Gupta et al , 2008Wilkie et al 2005;Porayska-Pomsta and Mellish 2004;Wang et al 2005), theories of personality (Ball and Breese 1998;André et al 2000;Loyall and Bates 1995;Isard et al 2006), and individual differences and preferences for both style and content Walker et al 2007;Reiter and Sripada 2002;Lin 2006;Belz 2008). While there are strong relations between these different notions of style, and the types of linguistic variation associated with personality factors, here we limit our detailed discussion of prior work to personality generation.…”

“…In natural conversation, humans adapt to one another across many levels of utterance production via processes variously described as linguistic style matching, entrainment, alignment, audience design, and accommodation (Niederhoffer and Pennebaker 2002;Brennan 1996;Levelt and Kelter 1982;Pickering and Garrod 2004;Brennan and Clark 1996;Nenkova et al 2008;Giles et al 1991). A number of recent studies strongly suggest that dialogue systems that adapted to the user in a similar way would be more effective (Murray 1997;Hayes-Roth and Brownston 1994;André et al 2000;Cassell and Bickmore 2003;Tapus and Mataric 2008;Mott and Lester 2006;Hirschberg 2008;Reeves and Nass 1996;Brennan 1991;Forbes-Riley and Litman 2007;Forbes-Riley et al 2008;Stenchikova and Stent 2007;Reitter et al 2006). Several of these studies provide empirical evidence that adaptation to the conversational partner is also beneficial at the personality level through experiments using hand-crafted utterances designed intuitively to express a particular personality (Reeves and Nass 1996;Tapus and Mataric 2008).…”

Designing and evaluating a wizarded uncertainty-adaptive spoken dialogue tutoring system

“…Our dialogue acts are based on similar schemes from other tutorial dialogue projects (e.g., Graesser et al, 1995). This analysis is summarized below, and discussed in greater detail in Forbes-Riley and Litman (2007).…”