“How do you know that I don’t understand?” A look at the future of intelligent tutoring systems

Abstract

Many software systems would significantly improve performance if they could adapt to the emotional state of the user, for example if Intelligent Tutoring Systems (ITSs), ATM's, ticketing machines could recognise when users were confused, frustrated or angry they could guide the user back to remedial help systems so improving the service. Many researchers now feel strongly that ITSs would be significantly enhanced if computers could adapt to the emotions of students. This idea has spawned the developing field of Affective Tutoring Systems (ATSs): ATSs are ITSs that are able to adapt to the affective state of students. The term "Affective Tutoring System" can be traced back as far as Rosalind Picard's book Affective Computing in 1997.This paper presents research leading to the development of Easy with Eve, an ATS for primary school mathematics. The system utilises a network of computer systems, mainly embedded devices to detect student emotion and other significant bio-signals. It will then adapt to students and displays emotion via a lifelike agent called Eve. Eve's tutoring adaptations are guided by a case-based method for adapting to student states; this method uses data that was generated by an observational study of human tutors. This paper presents the observational study, the case-based method, the ATS itself and its implementation on a distributed computer systems for real-time performance, and finally the implications of the findings for Human Computer Interaction in general and e-learning in particular. Web-based applications of the technology developed in this research are discussed throughout the paper.

Communication skills training exploiting multimodal emotion recognition

“…Contemporary, instructional approaches increasingly address emotional dimensions by accommodating challenges, excitement, ownership, and responsibility among other things in the learning environment. Software systems for e-learning (e.g., ITS, serious games, personal learning environments) could better foster learning if they also adapt the instruction and feedback to the emotional state of the learner (Sarrafzadeh, et al, 2008). Within the scope of ITS, Feidakis and his colleagues (Feidakis, Daradoumis, & Caballe, 2011) categorized emotion measurement into three types of tools, which have been described in several previous studies: 1) psychological (Wallbott, 1998), 2) physiological (Kramer, 1991), and 3) motorbehavioural (Leventhal, 1984).…”

“…The influence of emotions on learning is traditionally well recognized in classroom teaching practice (Bower, 1981). More recently, emotions have also received attention in the domain of intelligent tutoring systems (ITS) (Sarrafzadeh, Alexander, Dadgostar, Fan, & Bigdeli, 2008). An ITS is a computer-based system that is capable of providing immediate and personalized instruction and feedback to learners (Psotka & Mutter, 1988).…”

Multimodal Intelligent Tutoring Systems

“…Sarrafzadeh videoed several tutors as they tutored students individually and a coding scheme was developed to extract data from each tutoring video to describe the behaviors, facial expressions and expression intensities of students and tutors. Tutoring actions are guided by a case-based method for adapting to student states that recommends a weighted set of tutor actions and expressions (Sarrafzadeh Alexander, 2008). However, this approach has two main shortcomings: firstly, the video only recorded three human tutors' behaviors and the students' reactions to these behaviors are not considered.…”

“…Many researchers have integrated the facial expressions into ITS (Reategui & Boff, 2008;Roger, 2006;Sarrafzadeh & Alexander, 2008). However, the performance of facial expression recognition could be influenced by occlusion on the face caused by pose variation, glass wearing, and hair or hand covering etc.…”

The Application of Affective Computing Technology to E-Learning

“…Easy with Eve is an affect sensitive mathematics tutor developed by the Next Generation Tutoring Systems project (Alexander, Sarrafzadeh, & Hill, 2006;Sarrafzadeh, Alexander, Dadgostar, Fan, & Bigdeli, 2008) at Massey University in New Zealand. Affect recognition is performed by video analysis to capture facial expression and gesture information from the user.…”