A Mixed Reality and 2D Display Hybrid Approach for Visual Analysis of 3D Tissue Maps

Abstract

We present the design and a qualitative evaluation of a novel hybrid system that integrates a Mixed Reality (MR) stereoscopic view of 3D spatial data with linked views on a traditional 2D display. Our system tackles visualization challenges in the rapidly growing field of spatial biology that measures molecular components of tissues in their native spatial context and produces 3D tissue maps with unprecedented complexity and resolution. This field is particularly challenging due to the diversity in techniques in active development that generate 3D maps with vastly different properties. Throughout our process, we collaborated closely with experts in this biological domain to iteratively design, prototype, and ultimately test a hybrid system we used in three in-depth case studies. We extended the web-based Vitessce framework for single-cell analysis, which domain experts already use. We integrated a WebXR spatial view to enable direct access through a web browser. We used the first prototype in a formative user study (n=20 users) at a scientific meeting and obtained informal feedback used to improve the prototype. A qualitative evaluation (n=15 users) of a second prototype revealed that all participants see value in a hybrid system, even among those who self-reported as being skeptical of MR. We share insights from user feedback sessions that strongly support combining direct hand interaction in MR with traditional mouse input on a 2D display.