A novel AR remote collaborative platform for sharing 2.5D gestures and gaze

Wang, Yue; Wang, Peng; Luo, Zhiyong; Yan, Yuxiang

doi:10.1007/s00170-022-08747-7

Cited by 17 publications

(3 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In previous studies, remote presentation was performed by projection [12,14] and 3D displays [15] to enhance the observation experience of remote users. Additionally, pro-jectors [16,17] can deliver visual communication cues from remote users to local users. Amores et al [18] used head-mounted displays and depth cameras to create a system that can immerse the remote user in the first-person view of another user and provide a way for remote experts to provide guidance through 3D gestures and sound.…”

Section: Related Work 21 Mr Remote Collaboration Systemmentioning

confidence: 99%

Viewpoint-Controllable Telepresence: A Robotic-Arm-Based Mixed-Reality Telecollaboration System

Luo

Weng

Hao

et al. 2023

Sensors

View full text Add to dashboard Cite

In mixed-reality (MR) telecollaboration, the local environment is remotely presented to a remote user wearing a virtual reality (VR) head-mounted display (HMD) via a video capture device. However, remote users frequently face challenges in naturally and actively manipulating their viewpoints. In this paper, we propose a telepresence system with viewpoint control, which involves a robotic arm equipped with a stereo camera in the local environment. This system enables remote users to actively and flexibly observe the local environment by moving their heads to manipulate the robotic arm. Additionally, to solve the problem of the limited field of view of the stereo camera and limited movement range of the robotic arm, we propose a 3D reconstruction method combined with a stereo video field-of-view enhancement technique to guide remote users to move within the movement range of the robotic arm and provide them with a larger range of local environment perception. Finally, a mixed-reality telecollaboration prototype was built, and two user studies were conducted to evaluate the overall system. User study A evaluated the interaction efficiency, system usability, workload, copresence, and user satisfaction of our system from the remote user’s perspective, and the results showed that our system can effectively improve the interaction efficiency while achieving a better user experience than two traditional view-sharing techniques based on 360 video and based on the local user’s first-person view. User study B evaluated our MR telecollaboration system prototype from both the remote-user side and the local-user side as a whole, providing directions and suggestions for the subsequent design and improvement of our mixed-reality telecollaboration system.

show abstract

Section: Related Work 21 Mr Remote Collaboration Systemmentioning

confidence: 99%

Viewpoint-Controllable Telepresence: A Robotic-Arm-Based Mixed-Reality Telecollaboration System

Luo

Weng

Hao

et al. 2023

Sensors

View full text Add to dashboard Cite

show abstract

“…Table 1 shows the previous works. [16] has a novel remote collaborative platform based on gestures and gaze to support collaboration works during assembly or training tasks. They have motivated by the remote collaboration enables local workers and remote experts to collaborate in real-time without geographic restrictions.…”

Section: Background Studymentioning

confidence: 99%

Multi-Scale Interactions Between Augmented Reality and Virtual Reality Users in Mixed Reality Environment

Ajune Wanis Ismail,

Norhaida Mohd Suaib,

Noranita Fairos

et al. 2023

ARASET

View full text Add to dashboard Cite

Multi-scale interaction in Mixed Reality (MR) allows the avatars to shrink down themselves into a small size or enlarge into giant size. Multi-scale in MR is important in some situations that required precise avatar’s measurements to interact and manipulate with the content. However, the correct or appropriate scaling for multi-scale still remain unsolved in MR, very less people work in Augmented Reality (AR). It is critical to align precisely the way humans perceive the virtual world with the ways perceiving the real world. Therefore, this paper aims to produce a correct multi-scale technique for VR and AR users in MR environment. This paper presents a preliminary study on the previous works multi-scale techniques for MR remote collaboration. Then it explains the development of a multi-scale interaction between VR and AR users for the MR environment. Finally, this paper shows the results and implement the technique into an application. The results used to measure how multi-scale interaction can be leveraged with MR technology. The remote MR application involves AR and VR users with different interaction techniques.

show abstract

“…In C2TAM(Riazuelo et al, 2014) system, each agent separately sends keyframe images instead of all video data to the server for fusion and 3D reconstruction. Although sharing gestures and gazes can improve AR remote collaboration, most current systems only allow collaborators to share image data Wang et al (2022). describe a novel remote collaborative platform based on 2.5D gestures and gaze to address this issue (2.5DGG).…”

mentioning

confidence: 99%

Advanced Visual SLAM and Image Segmentation Techniques for Augmented Reality

Jiang

Tran

Williams

2022

International Journal of Virtual and Augmented Reality

View full text Add to dashboard Cite

Augmented reality can enhance human perception to experience a virtual-reality intertwined world by computer vision techniques. However, the basic techniques cannot handle complex large-scale scenes, tackle real-time occlusion, and render virtual objects in augmented reality. Therefore, this paper studies potential solutions, such as visual SLAM and image segmentation, that can address these challenges in the augmented reality visualizations. This paper provides a review of advanced visual SLAM and image segmentation techniques for augmented reality. In addition, applications of machine learning techniques for improving augmented reality are presented.

show abstract

A novel AR remote collaborative platform for sharing 2.5D gestures and gaze

Cited by 17 publications

References 31 publications

Viewpoint-Controllable Telepresence: A Robotic-Arm-Based Mixed-Reality Telecollaboration System

Viewpoint-Controllable Telepresence: A Robotic-Arm-Based Mixed-Reality Telecollaboration System

Multi-Scale Interactions Between Augmented Reality and Virtual Reality Users in Mixed Reality Environment

Advanced Visual SLAM and Image Segmentation Techniques for Augmented Reality

Contact Info

Product

Resources

About