Automatic analysis of affective postures and body motion to detect engagement with a game companion

Abstract

The design of an affect recognition system for socially perceptive robots relies on representative data: human-robot interaction in naturalistic settings requires an affect recognition system to be trained and validated with contextualised affective expressions, that is, expressions that emerge in the same interaction scenario of the target application. In this paper we propose an initial computational model to automatically analyse human postures and body motion to detect engagement of children playing chess with an iCat robot that acts as a game companion. Our approach is based on vision-based automatic extraction of expressive postural features from videos capturing the behaviour of the children from a lateral view. An initial evaluation, conducted by training several recognition models with contextualised affective postural expressions, suggests that patterns of postural behaviour can be used to accurately predict the engagement of the children with the robot, thus making our approach suitable for integration into an affect recognition system for a game companion in a real world scenario.

Detecting and Addressing Frustration in a Serious Game for Military Training

“…If correct, this explanation underscores the challenges inherent in scaling and generalizing sensor-based affect detectors. One direction for addressing these sources of variation in the future is to investigate alternate positioning for the Kinect sensors, including use of side angles (Sanghvi et al 2011). Additional directions include engineering predictor features using standard units rather than Cartesian space, as well features that account for individual differences in body shape and size (Worsley et al 2015).…”

The NoXi database: multimodal recordings of mediated novice-expert interactions

“…For example, Poggi formulates engagement as: " e value that a participant in an interaction attributes to the goal of being together with the other participant(s) and of continuing the interaction" [37]. is de nition has been used in several works [6,13,20,35,39]. Based on this de nition we added continuous annotations of engagement and then provided discrete instances when a change occurs ranging from strongly disengaged to strongly engaged.…”

Improving classification of posture based attributed attention assessed by ranked crowd-raters

“…a pressure sensing chairs [6], [7], or generic inputs e.g. video [8] or Kinect [9], [10]. The number of cognitive and affective states being inferred from observable posture continues to increase [9], [10]: frustration, involvement, endurability, engagement or attention.…”