Towards an articulated avatar in VR: Improving body and hand tracking using only depth cameras

25th ACM Symposium on Virtual Reality Software and Technology

Jung

et al. 2019

Self Cite

Figure 1: The virtual interview experiment in different avatar control conditions with first-person view (FPV), third-person view (TPV), and real-world view (RWV): (a) Controller-based interview session, task 1: Answer the questions, (b) Controllerbased interview session, task 2: Route planning task, (c) Depth-sensor-based interview session, task 1: Answer the questions, (d) Depth-sensor-based interview session, task 2: Route planning task

Section: Methodsmentioning

confidence: 99%

Exploring the Use of a Robust Depth-sensor-based Avatar Control System and its Effects on Communication Behaviors

25th ACM Symposium on Virtual Reality Software and Technology

Jung

et al. 2019

Self Cite

“…The full-body movement data was collected by four Kinect v2 devices placed in the corners of the tracking area. This system was based on the work of Wu et al (Wu et al, 2019a;Wu et al, 2019b) with both body (21 joints including the torso, arms, and legs) and hand-gesture tracking (19 joints with pointing, grasp, and pinch). However, we improved the avatar control algorithm compared to that of Wu et al (2019a) in the following aspects: 1) we recalculated joint rotations based on joint positions and only used the joint rotations information provided by the Kinect cameras as a reference.…”

Section: Body Trackingmentioning

confidence: 99%

“…The solution used in the work (Wu et al, 2019b) has a limited tracking range and area (single LMC), which requires the user to keep their hands in front of the eyes to avoid tracking loss. If the user moves the hands out of the tracking area, the data switches to the Kinect system, and the hands would only be tracked without finger movement.…”

Section: Hand Trackingmentioning

confidence: 99%

Using a Fully Expressive Avatar to Collaborate in Virtual Reality: Evaluation of Task Performance, Presence, and Attraction

Jung

et al. 2021

Front. Virtual Real.

Self Cite

Avatar-mediated collaboration in virtual environments is becoming more and more prevalent. However, current consumer systems are not suited to fully replicate real-world nonverbal communication. We present a novel avatar system for collaboration in virtual reality, which supports high levels of nonverbal expression by tracking behavior such as body movement, hand gesture, and facial expression. The system was built using camera tracking technology only. Therefore, in contrast to many other high-level tracking systems, it does not require users to wear additional trackers on their bodies. We compared our highly expressive system with a consumer setup extended with two body-worn trackers in a dyadic study. We investigated users’ performance, such as completion time and accuracy, as well as the presence and interpersonal attraction in a virtual charades game using an asymmetric control scheme. The results show that participants interacting with highly expressive avatars felt more social presence and attraction and exhibited better task performance than those interacting with partners represented using low-expressive avatars. Hence, we conclude that virtual reality avatar systems benefit from a higher level of nonverbal expressiveness, which can be achieved without additional body-worn trackers.

“…Visual-based approaches use visual cues [41] or improved display interface [23] to enhance users' spatial awareness in virtual environments. This approach only relies on basic VR devices and can work with mark-less body tracking system [46,48].…”

Section: Assembly Simulation In Virtual Environmentmentioning

confidence: 99%

Multiple perspectives integration for virtual reality-aided assemblability assessment in complex virtual assembly environment

et al. 2021

Preprint

Self Cite

Virtual Reality (VR) has been proved as a promising tool for industrial design, but the traditional VR interface of first-person perspective (1PP) is not useful enough to support assemblability assessment in complex virtual assembly environments. In this paper, we proposed the multi-perspectives interface (MPI) which integrates the 1PP and the third-person perspective (3PP) using handheld World-in-Miniature (WIM). The MPI allows users to simulate the assembly operations in a natural manner similar to 1PP, while providing users with an overview of the assembly status through the WIM to assess the assemblability with superior spatial awareness. Two studies were included in this paper. The first study tested MPI in a general interaction task, which reveals stronger spatial awareness in MPI than in 1PP without the cost of losing natural interaction. The second study tested the performance, usability, and workload of MPI in an assemblability assessment task. The results show the advantages of MPI in the reachability evaluation. The contribution of this paper is exploring a novel interface for VR-aided assembly design and evaluation system.