Mixed Reality Light Fields for Interactive Remote Assistance

Mohr, Peter; Mori, Shohei; Langlotz, Tobias; Thomas, Bruce H.; Schmalstieg, Dieter; Kalkofen, Denis

doi:10.1145/3313831.3376289

Cited by 29 publications

(18 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The on-site environment was captured using technologies such as 360-degree cameras providing pictures [22] or videos [17,21,22], light fields constructed out of pictures taken with a smartphone [29], multiple RGB-D [30] cameras, as well as built-in spatial mapping [22,31] to generate a mesh of the environment. While some captured the on-site environment prior to the actual collaboration [29,31], others provided online reconstruction of the on-site environment [17,21,30] allowing the virtual scene to be updated in line with changes on-site. Reconstructed scenes could be accessed via VR-HMDs [17,19,21,22,30,31] or HHDs [29].…”

Section: Consistent and Accessible Visualizationsmentioning

confidence: 99%

“…While some captured the on-site environment prior to the actual collaboration [29,31], others provided online reconstruction of the on-site environment [17,21,30] allowing the virtual scene to be updated in line with changes on-site. Reconstructed scenes could be accessed via VR-HMDs [17,19,21,22,30,31] or HHDs [29].…”

Section: Consistent and Accessible Visualizationsmentioning

confidence: 99%

“…Real-time manipulation was enabled by performing interactions on a voxel grid-a volumetric representation of the reconstructed space where each voxel held information on possible manipulations-which was computationally less expensive than performing interactions on the underlying mesh. Mohr et al [29], however, note that the visual quality of such approaches may be impaired by shiny and transparent surfaces which are common in industrial environments-a major application field for XR technologies. Addressing this issue, they proposed to provide off-site collaborators with a light field of the relevant parts of the workspace.…”

Section: Consistent and Accessible Visualizationsmentioning

confidence: 99%

“…However, simple virtual overlays such as annotations that are not semantically anchored in space will become useless whenever the on-site collaborator's viewpoint differs from the one from which the annotation was added. This issue was addressed in several research papers [29,[39][40][41][42][43][44]. Nuernberger et al [43] proposed an advanced way to anchor 2D sketches of circles and arrows in a world-stabilized way.…”

Section: Consistent and Accessible Visualizationsmentioning

confidence: 99%

See 3 more Smart Citations

Scalable Extended Reality: A Future Research Agenda

Memmesheimer

Ebert

2022

BDCC

View full text Add to dashboard Cite

Extensive research has outlined the potential of augmented, mixed, and virtual reality applications. However, little attention has been paid to scalability enhancements fostering practical adoption. In this paper, we introduce the concept of scalable extended reality (XRS), i.e., spaces scaling between different displays and degrees of virtuality that can be entered by multiple, possibly distributed users. The development of such XRS spaces concerns several research fields. To provide bidirectional interaction and maintain consistency with the real environment, virtual reconstructions of physical scenes need to be segmented semantically and adapted dynamically. Moreover, scalable interaction techniques for selection, manipulation, and navigation as well as a world-stabilized rendering of 2D annotations in 3D space are needed to let users intuitively switch between handheld and head-mounted displays. Collaborative settings should further integrate access control and awareness cues indicating the collaborators’ locations and actions. While many of these topics were investigated by previous research, very few have considered their integration to enhance scalability. Addressing this gap, we review related previous research, list current barriers to the development of XRS spaces, and highlight dependencies between them.

show abstract

Section: Consistent and Accessible Visualizationsmentioning

confidence: 99%

Section: Consistent and Accessible Visualizationsmentioning

confidence: 99%

Section: Consistent and Accessible Visualizationsmentioning

confidence: 99%

Section: Consistent and Accessible Visualizationsmentioning

confidence: 99%

See 2 more Smart Citations

Scalable Extended Reality: A Future Research Agenda

Memmesheimer

Ebert

2022

BDCC

View full text Add to dashboard Cite

show abstract

“…Mohr et al [60] proposed an MR collaboration system using mobile devices that uses light fields for orientation. The system records the local space and sends images to the remote user, who annotates them and sends them back to be visualized in AR by the local user.…”

Section: Internetmentioning

confidence: 99%

A Review on Methods and Systems for Remote Collaboration

et al. 2021

View full text Add to dashboard Cite

Due to the appearance of COVID-19 in 2019, person-to-person interactions were drastically reduced. The impact of these restrictions on the economic environment was significant. For example, technical assistance for commissioning or adjusting the parameters of some complex machines/installations had to be postponed. Economic operators became interested in the possibility of remote collaboration, depending on the manufactured products and the performance of the production lines that they owned. This bibliographic research was undertaken to address these needs. The purpose of this review was to analyze the current solutions, approaches, and technologies that workers and specialists can implement to obtain a reliable remote collaboration system. This survey focuses on techniques, devices, and tools that are being used in different contexts to provide remote guidance. We present communication cues and methods being employed, the implemented technological support, and the areas that benefit from remote collaboration. We hope that our effort will be useful to those who develop such systems and people who want to learn about the existence of collaborative solutions, and that it will increase awareness about the applications and the importance of the domain. We are convinced that, with the development of communication systems, the advancement of remote support systems will be a goal for many economic operators.

show abstract

Fusion information refresh rate improvement based on adaptive visual tracking in vehicle augmented reality sightseeing interactive system

Tsai,

Dai,

Yan

et al. 2024

J Soc Info Display

View full text Add to dashboard Cite

This paper proposes a dynamic information fusion interactive system with a transparent display applied in the vehicle. The dynamic information fusion interactive system integrates three leading technologies: relative position acquisition, fusion information mapping, and visual field tracking adaptive information generation. For visual field tracking adaptive information generation, the position of fusion information changes as the observer's field of view varies to help observers use this system more comfortably and intuitively. The frame rate of the dynamic virtual information display can be increased from 10 to 20 Hz without increasing the AI computing power and providing observers with a better observer experience. Integrating those technologies could achieve an AR‐based interactive system on a direct‐view transparent display. Such a system can provide passengers with location‐based interactive virtual and real integration information in the mobile field.

show abstract

Mixed Reality Light Fields for Interactive Remote Assistance

Cited by 29 publications

References 33 publications

Scalable Extended Reality: A Future Research Agenda

Scalable Extended Reality: A Future Research Agenda

A Review on Methods and Systems for Remote Collaboration

Fusion information refresh rate improvement based on adaptive visual tracking in vehicle augmented reality sightseeing interactive system

Contact Info

Product

Resources

About