This manuscript identifies and documents unsolved problems and research challenges in the extended reality (XR) domain (i.e., virtual (VR), augmented (AR), and mixed reality (MR)). The manuscript is structured to include technology, design, and human factor perspectives. The text is visualization/display-focused, that is, other modalities such as audio, haptic, smell, and touch, while important for XR, are beyond the scope of this paper. We further narrow our focus to mainly geospatial research, with necessary deviations to other domains where these technologies are widely researched. The main objective of the study is to provide an overview of broader research challenges and directions in XR, especially in spatial sciences. Aside from the research challenges identified based on a comprehensive literature review, we provide case studies with original results from our own studies in each section as examples to demonstrate the relevance of the challenges in the current research. We believe that this paper will be of relevance to anyone who has scientific interest in extended reality, and/or uses these systems in their research.

No abstract

Structure from motion (SfM) has emerged as a popular method for characterising marine benthos (seabed organisms), particularly in clear, tropical waters. However, there are many environmentally sensitive benthic organisms inhabiting temperate waters, including the reef‐forming glass sponges of the north‐east Pacific Ocean. Broader questions are raised, not just about whether SfM is a capable spatial data acquisition and ecological characterisation method in temperate waters; but whether a systematic assessment of capture methods in dry and wet laboratory conditions reveals critical relationships between SfM parameters, data products and their implications for underwater surveys. This paper, the first of two empirical assessments, reports on a series of dry‐lab tests quantifying the impact that lighting, camera type, camera settings and capture strategy have on data accuracy. These tests provide a crucial accuracy baseline for subsequent wet‐lab and field‐based surveys, revealing that photographs captured from a controlled and stable platform produce superior data products. While the measurable differences were small, they may be critical for accurate change detection in temperate environments.

Abstract. Structure-from-motion (SfM) has emerged as a popular method of characterizing marine benthos in tropical marine environments and could be of tremendous value to glass sponge monitoring and management efforts in the Northeast Pacific Ocean. However, temperate marine environments present a unique set of challenges to SfM workflows, and the combined impact that cold, dark, and turbid waters have on the veracity of SfM derived data must be critically evaluated in order for SfM to become a meaningful tool for ongoing glass sponge research. This paper discusses the design, development, testing, and deployment of an innovative underwater SfM workflow for generating high-resolution 3D models in temperate marine environments. This multi-phase research project (dry-lab, wet-lab, and field), while possibly seen as unconventional, was designed to innovate in two ways. First to build an operational data capture platform to support low-cost SfM-based seafloor surveys. And second, to enable systematic isolation and evaluation of SfM data capture parameters and their implications for representational veracity and data quality. This paper reports the challenges and outcomes from a series of field surveys conducted in Howe Sound, BC, one of which serves as the first of two data sets in a temporal analysis of 3D morphometric change. This research demonstrates that accurate, high-resolution morphometric characterization, of all benthic species and habitats, is dependent on a range of equipment, procedural, and environmental variables. It is also intended to share our applied problem-solving path to successful 3D capture, backed up by robust data science.

This article presents exploratory research to develop new workflows that address the challenges of adequately capturing the geometry and topology of complex institutional spaces, the analysis of prescriptive evacuation plans, and the simulation of human movement and behavior in emergency scenarios. We present a collection of geovisual analytical environments that were developed to permit new ways to view and assess risk, evacuation, and human movement. Part of this research considers how different approaches to the representation of complex institutional space, using three-dimensional capture technologies at multiple resolutions (or derived from conventional formats, such as building plans), have implicit advantages or liabilities in the analysis of risk and human evacuation. We combine three-dimensional data capture methods with geographical information science theory, three-dimensional game engines, three-dimensional evacuation simulations and spatial analyses that address the variability of campus populations, and draw upon three-dimensional modeling and photogrammetry for the assessment of real-world features in digital space. The outcome of this research demonstrates agile workflows that address emergency planning requirements, but could also enable enhanced visual analysis and interactive learning by all campus citizens. Furthermore, this work reveals key considerations and limitations associated with the dynamic nature of evacuation events and the static environments in which they have been simulated.

Advancements in extended reality (XR) have inspired new uses and users of advanced visualization interfaces, transforming geospatial data visualization and consumption by enabling interactive 3D geospatial data experiences in 3D. Conventional metrics (e.g., mental rotations test (MRT)) are often used to assess and predict the appropriateness of these visualizations without accounting for the effect the interface has on those metrics. We developed the Immersive MRT (IMRT) to evaluate the impact that virtual reality (VR) based visualizations and 3D virtual environments have on mental rotation performance. Consistent with previous work, the results of our pilot study suggest that mental rotation tasks are performed more accurately and rapidly with stereo 3D stimuli than with 2D images of those stimuli.

<p><strong>Abstract.</strong> Emergency preparedness is a fundamental component of a successful emergency management strategy. This includes a proactive communication strategy that informs all stakeholders of the emergency plan and helps translate that knowledge to real spaces. Communicating multilevel built environments can be difficult, as the architectural complexity creates problems for both visual and mental representations of networks in 3D space. Modern mobile technology offers emerging opportunities for emergency managers to develop and deploy 3D visualizations of multilevel spaces that preserve the topology of those spaces while adding the spatial context that allows the individual to better understand their position within it. In this paper, we present a collection of mixed reality (specifically augmented reality) geovisualizations that overcome the visual limitations associated with the traditional static 2D methods of communicating the evacuation plans of multilevel structures. We demonstrate how this technology can provide spatially contextualized 3D geovisualizations that promote spatial knowledge acquisition and support cognitive mapping. These geovisualizations are designed as a proactive emergency management tool to educate and prepare at risk populations prior to the occurrence of a hazardous event.</p>

Structure from motion (SfM) is an accessible and non‐intrusive method of three‐dimensional (3D) data capture popular for tropical coral reef surveying. In the north‐east Pacific Ocean, where there are many environmentally sensitive benthic organisms whose morphology and function are equally important, SfM surveys are less commonly studied. Temperate waters pose unique challenges to SfM workflows, which must be systematically unpacked to understand their impact on data quality and veracity. This uncertainty raises broader questions concerning SfM as a spatial data‐acquisition and ecological characterisation method in temperate waters, and whether a systematic workflow assessment reveals vital relationships between SfM implementation parameters, 3D data products and their implications for underwater SfM surveys. This paper, the second of two empirical assessments, reports on a series of wet‐lab and dryland tests quantifying the impact that temperate waters, underwater cameras, and photograph quantity and configuration have on SfM accuracy. These tests provided crucial accuracy benchmarks informing subsequent field‐based surveys and revealed that underwater SfM workflows can generate highly accurate 3D models in temperate waters.