Visual perspective taking for avatars in a Simon task

Abstract

In modern digital applications, users often interact with virtual representations of themselves or others, called avatars. We examined how these avatars and their perspectives influence stimulus-response compatibility in a Simon task. Participants responded to light/dark blue stimuli with left/right key presses in the presence of a task-irrelevant avatar. Changes in stimulus-response compatibility were used to quantify changes in the mental representation of the task and perspective taking toward this avatar. Experiments 1 and 2 showed that perspective taking for an avatar occurred in orthogonal stimulus-response mappings, causing a compatibility effect from the avatar's point of view. In the following two experiments we introduced a larger variety of angular disparities between the participant and avatar. In Experiment 3, the Simon effect with lateralized stimulus positions remained largely unaffected by the avatar, pointing toward an absence of perspective taking. In Experiment 4, after avatar hand movements were added in order to strengthen the participants' sense of agency over the avatar, a spatial compatibility effect from the avatar's perspective was observed again, and hints of the selective use of perspective taking on a trial-by-trial basis were found. Overall, the results indicate that users can incorporate the perspective of an avatar into their mental representation of a situation, even when this perspective is unnecessary to complete a task, but that certain contextual requirements have to be met.

Spatial–numerical associations in the presence of an avatar

“…Instead, the stimulus feature responsible for the observed compatibility effect-the magnitude of the number-was not influenced by the perspective manipulation and it can therefore be viewed as constant between conditions. In contrast to earlier studies that investigated the impact of avatars on spatial compatibility (e.g., Böffel & Müsseler, 2019b), the results cannot be explained by differences in stimulus coding. Conversely, the avatar offered a reference frame to code the responses and that reference frame was different between both avatar positions.…”

“…In this experiment, we used compatibility effects based on numerical-spatial associations to examine whether the presence of an avatar leads to perspective-based changes in response coding. We asked participants to perform a parity judgment of a centrally presented number in the presence of an avatar (Böffel & Müsseler, 2019b). The magnitude of the number in such a task generally causes a compatibility effect resulting in faster responses when large numbers are mapped to right or upper responses and smaller numbers mapped to left or lower responses (Ito & Hatta, 2004;Wood et al, 2008).…”

“…The magnitude of the number in such a task generally causes a compatibility effect resulting in faster responses when large numbers are mapped to right or upper responses and smaller numbers mapped to left or lower responses (Ito & Hatta, 2004;Wood et al, 2008). In contrast to a Simon task with visual stimuli, where the stimulus position can be coded from a different perspective (Böffel & Müsseler, 2019b), the association between numbers and space is based on the magnitude of the number and therefore constant, no matter what perspective is used to view the number. The results are therefore in line with the assumption that the avatar caused changes in the response codes that lead to the responses being perceived as right or left, instead of below or above.…”

“…Changes in stimulus-response compatibility can therefore be used to quantify changes in the mental representation of certain aspects of an event, for example the location of a stimulus. Böffel and Müsseler (2019b) give an example for this, as stimuli that were presented at the top or bottom of the screen produced a compatibility effect as if they were regarded as left or right, depending of the perspective of an avatar. The interpretation was that the vertically presented stimuli gained horizontal features, for example changing the mental representation of a bottom stimulus to a left one, if presented on the left of an avatar.…”

No evidence for automatic response activation with target onset in the avatar-compatibility task

“…Even though compatibility effects are overall reliable, they can be influenced by additional manipulations, for example, regarding intention (Hommel, 1993a ), reference frames (Böffel & Müsseler, 2019b , 2019a , 2020 ; Hommel & Lippa, 1995 ; Müsseler et al 2019 ), the frequency of compatible and incompatible trials (Hommel, 1994 ), or instruction (Böffel & Müsseler, 2018 ; Heister & Schroeder-Heister, 1994 ). In the present study, we examine whether one of the central claims of the dimensional overlap model, the automatic activation of the spatially corresponding response, holds true in the context of the avatar-compatibility task in which compatibility is manipulated by the presentation of an avatar.…”