Unintentional synchronization with self-avatar for upper- and lower-body movements

Abstract

The subjective experience of embodying an avatar when immersed in virtual reality (VR) is known to support the sense of presence and to help with the interaction in a virtual environment. Virtual embodiment is often thought of as the consequence of replacement of the physical body by a virtual one, with a sense of agency for the avatar obtained by making the avatar’s body follow the user’s movements. This unidirectional motor link was, however, challenged by studies observing the opposite effect under different circumstances, for example, in a slow-motion context or when an arm movement was snapped on a predefined axis. These reports are, however, still rare or anecdotal. With the idea of a generalized bidirectional relationship between the user and the avatar in mind, we established a methodology to systematically provoke and study the circumstances under which participants follow the movements of their avatar during long repetitive movements without having been instructed to do so. A preliminary study confirmed that our virtual experimental setup, using full-body motion capture, avatar animation, and virtual mirrors, supports a strong sense of agency and body ownership for the avatar while enabling the experimental manipulation of the avatar’s movement. In the main experimental study, where participants performed repetitive upper- and lower-body movements while their avatar animations were either congruent or out-of-phase, we observed that almost all participants synchronized with their avatar at least once, for ∼47% of trials for lower limb movements and ∼38% for upper limb movements. Participants still reported low agency and ownership for the avatar under the incongruent condition, but, most interestingly, some of them also reported that their movements were not influenced by the avatar despite the behavioral effect. Our methodological approach and results contribute to the characterization of the conditions of occurrence of the self-avatar follower effect and, thereby, to identifying an enriched interaction design for VR, involving complex avatar–user mutual interdependencies.

In Case of Doubt, One Follows One's Self: The Implicit Guidance of the Embodied Self-Avatar

“…Indeed, no-one is expected to follow their avatar if the seen gesture patently prevents the completion of an ongoing task. For instance, if the instructions explicitly require participants to lift an arm in front of them, seeing the avatar acting by itself doing a very different movement, such as lifting a knee, would break the sense of embodiment and, likely, no follower-effect would be observed [9]. Therefore, to investigate whether the self-avatar follower effect could occur for substantial movement distortions, we experimentally induced uncertainty about which movement should be performed at given times, to put the participant in front of several plausible options.…”

“…This behavior, called the self-avatar follower effect and described as "how when a spatial offset is introduced between the real and the virtual body, and the system allows for compensations, participants automatically act to reduce the spatial offset" [25], was later extended by considering a vertical distortion equivalent, further showing that participants tended to both horizontally and vertically move towards their avatar without having been instructed to do so [14]. Such motor behavior, that have been associated to motor contagion and mimicry [9,25], echoes previous observations made while participants were asked to draw straight lines while their virtual hand drew ellipses [12]. In such a context, participants were unaware that their actual motor performance was "attracted" toward the seen movement, suggesting that a conscious detection of the distortion can be associated with an unintentional motor response.…”

“…Further extending these results to full-body movements, it was shown that when alternating between a real-time animation of the avatar and slow-motion, participants tended to quickly adjust their movement speed to align with the avatar's one, thereby maintaining a consistent visuo-motor experience [43]. This latter phenomenon was further investigated for asynchronous cyclic and repetitive movements by asking participants to perform repetitive upper-and lower-body movements while their avatar animation was either congruent or out-of-phase [9]. In this case, participants tended to synchronize with their avatar, while some reported that their avatar did not influence their movements, despite the behavioral effect.…”

“…The bidirectionality of the participant-avatar relationship is perhaps best illustrated by the participants' spontaneous reaction to the "loss" of their avatar when a movement distortion is artificially introduced (voluntarily or due to technical limitations). In such cases participants would, when possible, compensate for the observed discrepancy between their avatar and their physical body posture [4,11] while otherwise, under certain conditions, would follow the avatar's movement [9,25,43]. When occurring, this phenomenon, called the Self-Avatar Follower Effect by Gonzalez-Franco et al [25], results in a reduction of the multi-sensory discrepancies between one's physical body and the visual feedback, which in turn contributes to the preservation of the sense of embodiment.…”

“…It is thus within a subtle range of postures and movement distortions that the spontaneous corrections characterizing the self-avatar follower effect seem to be triggered, and the challenge is to identify the requirements for this motor attraction to be elicited, regarding the nature of the movement distortion, the magnitude of the manipulation, and favorable context (e.g., type of task, previous movements). The self-avatar follower effect was previously investigated for movement distortions preserving a strong correlation between the user's initial movement and the avatar's one, such as when subtly distorting a reaching-forward movement [14,25], or when manipulating only the speed of participant's movements [43] or the synchronicity of the avatar animation [9]. Concerning more drastic movement distortions, interesting behaviors were anecdotally observed in our lab in situations when participants would see a pre-animated full-body avatar viewed from a first-person perspective; without specific instructions about how to react while observing the virtual body co-localized with themselves but moving on it's own, some participants would explicitly mirror the seen actions.…”