Exploring the Use of a Robust Depth-sensor-based Avatar Control System and its Effects on Communication Behaviors

Wu, Yuanjie; Wang, Yu; Jung, Sungchul; Hoermann, Simon; Lindeman, Robert W.

doi:10.1145/3359996.3364267

Cited by 17 publications

(16 citation statements)

References 35 publications

(30 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The ndings in our rst experiment indicate the impact of the control method for full upper-limbs on the interaction between the user and the virtual environment. Using the depth sensor based avatar control system has been proved to result in a higher senses of body ownership and agency compared to controller based avatar control [20]. Here, we show that our control method for full upper-limbs can elicit higher feelings of body ownership, agency, location of the body as well as system usability which supports the above hypothesis.…”

Section: Discussionsupporting

confidence: 81%

See 1 more Smart Citation

The Effect of a Full Upper-Limb Control System and Its Usage in Stroke Rehabilitation

Cha¹,

Wang

Yan³

et al. 2021

Preprint

View full text Add to dashboard Cite

Backgrounds: Transferring behaviors of a human’s upper-limbs to an avatar is widely used in the field of virtual reality rehabilitation. To realize the transfer, movement tracking technology is required. Traditionally, wearable tracking devices are used to do the tracking, however the devices are expensive and cumbersome. Recently, non-wearable upper-limb tracking solutions are proposed, which are cheaper and more comfortable to interact. But most of the existing products cannot track full upper-limbs including both the arms and all the fingers, which limits the motion paradigm and further may lead to limited rehabilitation effect. Methods: In this paper, a novel method was first proposed for full avatar’s upper-limb control which integrates the fine finger motion and the arm wide range motion. Then, based on the method, a Virtual Reality Rehabilitation System (VRRS) was developed for upper-limb rehabilitation. To test the performance of VRRS, two experiments were designed. First, in order to investigate the effect of VRRS on virtual body ownership, agency, location of the body and usability, we compared it with the partial upper-limb tracking method based on Leap Motion controller (LP) in same virtual environments. Then, to study the feasibility of VRRS in rehabilitation, we recruited 16 stroke patients and split them into two groups: the experimental group and the control group. Each group consisted 8 patients, with and without employing VRRS respectively.Results: The control of full avatar’s upper-limbs improved the users’ senses on body ownership, agency and location of the body. The users preferred to use VRRS. In addition, although the upper-limb motor function of patients from both groups were improved, the difference between the FM scores tested on the first day and the last day of the experimental group was more significant than that of the control group. Conclusions: VRRS based on the proposed method for full avatar’s upper-limbs control was developed, which improved the user experience on embodiment and effectively improved the rehabilitation effect for upper-limbs of stroke patients.Trial registration:The study was registered at the First Affiliated Hospital of Jinan University Identifier: KY-2020-036; Date of registration - June 01, 2020.

show abstract

Section: Discussionsupporting

confidence: 81%

“…In the experiment, the system usabilities were measured with the System Usability Scale [20] and the game performance was de ned as the sum of scores of normal motion mode and mirror motion mode.…”

Section: System Usability and Game Performancementioning

confidence: 99%

The Effect of a Full Upper-Limb Control System and Its Usage in Stroke Rehabilitation

Cha¹,

Wang

Yan³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…To compensate for these constraints, camera-based tracking devices, such as the Leap Motion controller (LMC), can capture natural hand gestures without using any controller. For example, Wu et al (2019a) developed a multisensor system that integrates multiple Kinects and an LMC to control an avatar. Other nonverbal cues for avatar control are eye gaze and facial expression.…”

Section: Avatar Control Systems and Representationmentioning

confidence: 99%

Using a Fully Expressive Avatar to Collaborate in Virtual Reality: Evaluation of Task Performance, Presence, and Attraction

Wang

Jung

et al. 2021

Front. Virtual Real.

Self Cite

View full text Add to dashboard Cite

Avatar-mediated collaboration in virtual environments is becoming more and more prevalent. However, current consumer systems are not suited to fully replicate real-world nonverbal communication. We present a novel avatar system for collaboration in virtual reality, which supports high levels of nonverbal expression by tracking behavior such as body movement, hand gesture, and facial expression. The system was built using camera tracking technology only. Therefore, in contrast to many other high-level tracking systems, it does not require users to wear additional trackers on their bodies. We compared our highly expressive system with a consumer setup extended with two body-worn trackers in a dyadic study. We investigated users’ performance, such as completion time and accuracy, as well as the presence and interpersonal attraction in a virtual charades game using an asymmetric control scheme. The results show that participants interacting with highly expressive avatars felt more social presence and attraction and exhibited better task performance than those interacting with partners represented using low-expressive avatars. Hence, we conclude that virtual reality avatar systems benefit from a higher level of nonverbal expressiveness, which can be achieved without additional body-worn trackers.

show abstract

“…In this case, the right elbow, right wrist, right hand, right thumb and right hand tip key points were almost aligned on the same vector (directed toward the sensor) causing a possible occlusion issue. This may have altered the tracking of the hand/arm position during the pointing gesture and resulted in imprecise movements as observed in previous studies using the Kinect [32]. This in turn, may have made the interpretation of the pointing direction difficult for these participants.…”

Section: Interpreting Pointing Gestures Of the Partnermentioning

confidence: 88%

Point-cloud avatars to improve spatial communication in immersive collaborative virtual environments

Gamelin

Chellali

Cheikh

et al. 2020

Pers Ubiquit Comput

View full text Add to dashboard Cite

Collaborative virtual environments allow remote users to work together in a shared 3D space. To take advantage of the possibilities offered by such systems, their design must allow the users to interact and communicate efficiently. One open question in this field concerns the avatar fidelity of remote partners. This can impact communication between the remote users, more particularly when performing collaborative spatial tasks. In this paper, we present an experimental study comparing the effects of two partner's avatars on collaboration during spatial tasks. The first avatar was based on a 2.5D streamed point-cloud and the second avatar was based on a 3D preconstructed avatar replicating the remote user movements. These avatars differ in their fidelity levels described through two components: visual and kinematic fidelity. The collaborative performance was evaluated through the efficacy of completing two spatial communication tasks, a pointing task and spatial guidance task. The results indicate that the streamed point-cloud avatar permitted a significant improvement of the collaborative performance for both tasks. The subjective evaluation suggests that these differences in performance can mainly be attributed to the higher kinematic fidelity of this representation as compared to the 3D preconstructed avatar representation. We conclude that, when designing spatial collaborative virtual environments, it is important to reach a high kinematic fidelity of the partner's representation while a moderate visual fidelity of this representation can suffice.

show abstract

Exploring the Use of a Robust Depth-sensor-based Avatar Control System and its Effects on Communication Behaviors

Cited by 17 publications

References 35 publications

The Effect of a Full Upper-Limb Control System and Its Usage in Stroke Rehabilitation

The Effect of a Full Upper-Limb Control System and Its Usage in Stroke Rehabilitation

Using a Fully Expressive Avatar to Collaborate in Virtual Reality: Evaluation of Task Performance, Presence, and Attraction

Point-cloud avatars to improve spatial communication in immersive collaborative virtual environments

Contact Info

Product

Resources

About