The Impacts of Visual Effects on User Perception With a Virtual Human in Augmented Reality Conflict Situations

Abstract

Virtual humans (VHs) in augmented reality (AR) can provide users an illusory sense of being together in the real space. However, such an illusion can easily break when the augmented VH conflicts (or is overlaid) with the real objects. Recent spatial understanding technology is starting to make physically plausible VHs in response to collisions, but there are still limitations (e.g., resolution, accuracy) and inevitable conflict situations (e.g., unexpected passer-by), especially in daily life. Moreover, depending on the situation, VH's plausible behavior to avoid collision may rather interfere with the original interaction with the users. In this paper, we investigate three such situations: (1) when VH appears in a room through a closed door, (2) when the VH's body overlaps with static real objects, and (3) when a real moving object passes through the VH. While we considered (2) as an avoidable situation where physically plausible behaviors of VHs might be required, (1) and (3) were considered as inevitable situations (e.g., VH appearing out of nowhere, or passer-by cannot be aware of a virtual being), and we may not present VH's plausible behaviors, so alternatives might be required. Thus, for each of these notable situations in AR, we tested different visual effects as presentation methods for physical conflicts between a VH and real objects. Our findings indicate that visual effects improve VH's social/co-presence and physicality depending on the situations and effect types as well as influence users' attention/social behaviors. We discuss the implications of our findings and future research directions.INDEX TERMS Augmented reality, pervasive AR, virtual human, visual effects, perceptual issue, physicality conflict, social presence, co-presence, inevitable collision, human perception.

Silhouettes from Real Objects Enable Realistic Interactions with a Virtual Human in Mobile Augmented Reality

“…After deployment, these proxies are rendered transparently in AR views to prevent users from being aware of proxies. Thus, virtual humans interact with these invisible virtual proxies, but users perceive that the virtual human is interacting with real objects [6].…”

“…Such an illusion by physical-virtual interactions can affect users positively [6,7], but modeling real objects and importing them on the system are not trivial tasks [16,18]. This is because commercial systems often add new objects or new interactions to keep users interested.…”

“…Since users can see the real environment (see Figure 1a), AR systems often require an accurate registration of virtual objects to provide seamless interactions in various situations [2,3]. Incorrect registration of a virtual object in the real space can cause unrealistic occlusions [4,5] or physically implausible situations [6,7], leading to perceptual quality degradation and breaks in presence [8].…”

CIRO: The Effects of Visually Diminished Real Objects on Human Perception in Handheld Augmented Reality

“…A related problem has to do with when the rendered AR scene looks physically implausible [6,19], e.g., objects floating in the air, or overlapping with the real object (i.e., physicality conflict), because of the lack of or erroneous 3D understanding of the environment. An additional provision, such as enabling the virtual objects to react to objects in the real environment in a physically plausible way, can help improve the sense of co-presence with the virtual object [4,6,26].…”

“…Such augmentations are often targeted for predesignated objects that are recognized and tracked by the AR system [2]. However, real environments are dynamic such that new objects may appear and inadvertently interfere with the main content [3][4][5][6]. For instance, the Pokémon Go application overlays the Pokémon character onto the video captured environment, making it appear as if it is standing on the ground, but a passerby can break such an illusion (see Figure 1b).…”