Motion capturing empowered interaction with a virtual agent in an Augmented Reality environment

Abstract

We present an Augmented Reality (AR) system where we immerse the user's whole body in the virtual scene using a motion capturing (MoCap) suit. The goal is to allow for seamless interaction with the virtual content within the AR environment. We describe an evaluation study of a prototype application featuring an interactive scenario with a virtual agent. The scenario contains two conditions: in one, the agent has access to the full tracking data of the MoCap suit and therefore is aware of the exact actions of the user, while in the second condition, the agent does not get this information. We then report and discuss the differences we were able to detect regarding the users' perception of the interaction with the agent and give future research directions.

Ability of Wearable Accelerometers-Based Measures to Assess the Stability of Working Postures

“…However, researchers have to rely on force plates (e.g., AMTI Force Plate, AMTI, Watertown, MA, USA) to obtain the COP data, which is quite expensive, difficult to set up and cumbersome to transport [21]. Recently, inertial sensor technology has been widely used in the balance-related research in the fields of biomechanics [22][23][24][25], ergonomics and human factors [26,27], sports science [28][29][30][31][32][33][34] and virtual or augmented reality [35][36][37], because the miniature inertial sensors are cost effective, wearable, compact and lightweight. With the rapid development and widespread application of wearable inertial sensors in the above fields, it is increasingly being used in the assessment of human body balance or postural stability [38][39][40][41][42][43].…”

Accuracy of Base of Support Using an Inertial Sensor Based Motion Capture System

“…The Xsens MVN BIOMECH (Xsens Technologies B.V., Enschede, The Netherlands) is a commercially available inertial sensor-based motion capture system composed of 17 miniature inertial sensors placed over the full body [ 4 , 5 ]. It has been widely used in the research fields of biomechanics [ 6 , 7 , 8 , 9 ], ergonomics and human factors [ 10 , 11 ], sports science [ 12 , 13 , 14 , 15 , 16 , 17 , 18 ] and virtual or augmented reality [ 19 , 20 , 21 ]. Xsens MVN has a good accuracy in human kinematics estimation, such as joint angle and segment orientation [ 22 , 23 , 24 , 25 , 26 ], it has been validated against optical motion capture system [ 7 , 26 , 27 ] and is currently considered as the ‘gold reference’ for kinematics measurements [ 28 , 29 ].…”

“…Michnik et al [ 30 ] used Xsens MVN to measure COM position and its speed to study the similarities and differences of body control during professional, externally forced fall to the side performed by men aged 24 and 65 years. Damian et al [ 21 ] used the position and orientation data provided by MVN Studio to synchronize the real and virtual environments and to create intuitive interaction modalities with virtual entities in the augmented reality environment. Carson et al [ 13 , 15 , 53 ] used the Xsens MVN position data as the input of Visual3D software (C-Motion Inc., Germantown, MD, USA) to analyze movement variability and identify swing events of male golfers.…”

User Perceptions of Communicative and Task-competent Agents in a Virtual Basketball Game

“…On the other hand, we focus on full body movement as a driver for interaction. This has been seen as an important function for communication in agents and robots (de Gelder, 2009;Sanghvi et al, 2011;Damian et al, 2013;Kistler et al, 2013). Extensive research on body movement and engagement has also been undertaken, particularly with regards to body movement as an engaging and affective mechanism (Bianchi-Berthouze, 2013; Kleinsmith and Bianchi-Berthouze, 2013).…”