Virtual, mixed, and augmented reality: a systematic review for immersive systems research

Liberatore, Matthew J.; Wagner, William P.

doi:10.1007/s10055-020-00492-0

Cited by 79 publications

(50 citation statements)

References 110 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…AR technology has already found its way into a large field of applications, where holographic elements enrich the perception of the real environment, e.g., by providing cognitive support during different tasks (65). In the wide field of rehabilitation, AR will introduce new pathways for therapeutic or assistive approaches with the potential of providing an engaging and motivating training environment (31), improving physical outcomes when applied as an adjunct therapy (29), supporting mental rehabilitation (44) or cognitive rehabilitation (57,66). Based on our findings, we envision HMD-based AR systems to assist patients in their ADLs in the future, thus maintaining autonomy.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Improvement of Apraxia With Augmented Reality: Influencing Pantomime of Tool Use via Holographic Cues

et al. 2021

View full text Add to dashboard Cite

Background: Defective pantomime of tool use is a hall mark of limb apraxia. Contextual information has been demonstrated to improve tool use performance. Further, knowledge about the potential impact of technological aids such as augmented reality for patients with limb apraxia is still scarce.Objective: Since augmented reality offers a new way to provide contextual information, we applied it to pantomime of tool use. We hypothesize that the disturbed movement execution can be mitigated by holographic stimulation. If visual stimuli facilitate the access to the appropriate motor program in patients with apraxia, their performance should improve with increased saliency, i.e., should be better when supported by dynamic and holographic cues vs. static and screen-based cues.Methods: Twenty one stroke patients and 23 healthy control subjects were randomized to mime the use of five objects, presented in two Environments (Screen vs. Head Mounted Display, HMD) and two Modes (Static vs. Dynamic) resulting in four conditions (ScreenStat, ScreenDyn, HMDStat, HMDDyn), followed by a real tool demonstration. Pantomiming was analyzed by a scoring system using video recordings. Additionally, the sense of presence was assessed using a questionnaire.Results: Healthy control participants performed close to ceiling and significantly better than patients. Patients achieved significantly higher scores with holographic or dynamic cues. Remarkably, when their performance was supported by animated holographic cues (e.g., striking hammer), it did not differ significantly from real tool demonstration. As the sense of presence increases with animated holograms, so does the pantomiming.Conclusion: Patients' performance improved with visual stimuli of increasing saliency. Future assistive technology could be implemented upon this knowledge and thus, positively impact the rehabilitation process and a patient's autonomy.

show abstract

Section: Discussionmentioning

confidence: 99%

“…Note that our experimental task and digital content do not require any interaction. Thus, the term "interaction" within this study does not reflect the overall accepted definition in the AR domain [for a recent review on immersive systems (44)].…”

Section: Data Analysis Scoring Systemmentioning

confidence: 99%

Improvement of Apraxia With Augmented Reality: Influencing Pantomime of Tool Use via Holographic Cues

et al. 2021

View full text Add to dashboard Cite

show abstract

“…This is particularly useful for remote field work, which often has high logistical costs (e.g., travel, food, lodging) that limit the number of individuals that can be involved. By navigating a DHPC in VR or AR, users can study the field conditions of remote sites in a cost-effective [6] and repeatable manner [7 , 8] . For fragile ecosystems, VR/AR visualization of the DHPC also allows multiple users to analyze the same field site without disturbing the natural dynamics of the system.…”

Section: Methods Detailsmentioning

confidence: 99%

“…VR/AR visualization of the DHPC makes the analyzed field site more accessible for individuals that might not have the funding, time or permission to study the site firsthand. Without the structural information provided by the elevation data, the same level of immersion cannot be obtained with conventional rasters, which are viewed in two-dimensions [6 , 8] .…”

Section: Methods Detailsmentioning

confidence: 99%

Implementation of the directly-georeferenced hyperspectral point cloud

et al. 2021

View full text Add to dashboard Cite

Before pushbroom hyperspectral imaging (HSI) data can be applied in remote sensing applications, it must typically be preprocessed through radiometric correction, atmospheric compensation, geometric correction and spatial resampling procedures. After these preprocessing procedures, HSI data are conventionally given as georeferenced raster images. The raster data model compromises the spatial-spectral integrity of HSI data, leading to suboptimal results in various applications. Inamdar et al. (2021) developed a point cloud data format, the Directly-Georeferenced Hyperspectral Point Cloud (DHPC), that preserves the spatial-spectral integrity of HSI data more effectively than rasters. The DHPC is generated through a data fusion workflow that uses conventional preprocessing protocols with a modification to the digital surface model used in the geometric correction. Even with the additional elevation information, the DHPC is still stored with file sizes up to 13 times smaller than conventional rasters, making it ideal for data distribution. Our article aims to describe the DHPC data fusion workflow from Inamdar et al. (2021), providing all the required tools for its integration in pre-existing processing workflows. This includes a MATLAB script that can be readily applied to carry out the modification that must be made to the digital surface model used in the geometric correction. The MATLAB script first derives the point spread function of the HSI data and then convolves it with the digital surface model input in the geometric correction. By breaking down the MATLAB script and describing its functions, data providers can readily develop their own implementation if necessary. The derived point spread function is also useful for characterizing HSI data, quantifying the contribution of materials to the spectrum from any given pixel as a function of distance from the pixel center. Overall, our work makes the implementation of the DHPC data fusion workflow transparent and approachable for end users and data providers. Our article describes the Directly-Georeferenced Hyperspectral Point Cloud (DHPC) data fusion workflow, which can be readily implemented with existing processing protocols by modifying the input digital surface model used in the geometric correction. We provide a MATLAB function that performs the modification to the digital surface model required for the DHPC workflow. This MATLAB script derives the point spread function of the hyperspectral imager and convolves it with the digital surface model so that the elevation data are more spatially consistent with the hyperspectral imaging data as collected. We highlight the increased effectiveness of the DHPC over conventional raster end products in terms of spatial-spectral data integrity, data storage requirements, hyperspectral imaging application results and site exploration via virtual and augmented reality.

show abstract

“…2021, 11, 4269 2 of 16 (MR) reality devices with the digital twin for full digitalization of the used technology. A nice review of virtual, mixed, and augmented reality for immersive systems research is presented in [4]. Some other research results of the mixed assembly process between human and collaborative robots are described in [5][6][7].…”

Section: Introduction and Related Workmentioning

confidence: 99%

CNN Training Using 3D Virtual Models for Assisted Assembly with Mixed Reality and Collaborative Robots

et al. 2021

View full text Add to dashboard Cite

The assisted assembly of customized products supported by collaborative robots combined with mixed reality devices is the current trend in the Industry 4.0 concept. This article introduces an experimental work cell with the implementation of the assisted assembly process for customized cam switches as a case study. The research is aimed to design a methodology for this complex task with full digitalization and transformation data to digital twin models from all vision systems. Recognition of position and orientation of assembled parts during manual assembly are marked and checked by convolutional neural network (CNN) model. Training of CNN was based on a new approach using virtual training samples with single shot detection and instance segmentation. The trained CNN model was transferred to an embedded artificial processing unit with a high-resolution camera sensor. The embedded device redistributes data with parts detected position and orientation into mixed reality devices and collaborative robot. This approach to assisted assembly using mixed reality, collaborative robot, vision systems, and CNN models can significantly decrease assembly and training time in real production.

show abstract

Virtual, mixed, and augmented reality: a systematic review for immersive systems research

Cited by 79 publications

References 110 publications

Improvement of Apraxia With Augmented Reality: Influencing Pantomime of Tool Use via Holographic Cues

Improvement of Apraxia With Augmented Reality: Influencing Pantomime of Tool Use via Holographic Cues

Implementation of the directly-georeferenced hyperspectral point cloud

CNN Training Using 3D Virtual Models for Assisted Assembly with Mixed Reality and Collaborative Robots

Contact Info

Product

Resources

About