Enhancing User Experiences of Mobile-Based Augmented Reality via Spatial Augmented Reality: Designs and Architectures of Projector-Camera Devices

Abstract

As smartphones, tablet computers, and other mobile devices have continued to dominate our digital world ecosystem, there are many industries using mobile or wearable devices to perform Augmented Reality (AR) functions in their workplaces in order to increase productivity and decrease unnecessary workloads. Mobile-based AR can basically be divided into three main types: phonebased AR, wearable AR, and projector-based AR. Among these, projector-based AR or Spatial Augmented Reality (SAR) is the most immature and least recognized type of AR for end users. This is because there are a small number of commercial products providing projector-based AR functionalities in a mobile manner. Also, prices of mobile projectors are still relatively high. Moreover, there are still many technical problems regarding projector-based AR that have been left unsolved. Nevertheless, it is projector-based AR that has potential to solve a fundamental problem shared by most mobile-based AR systems. Also the always-visible nature of projectorbased AR is one good answer for solving current user experience issues of phone-based AR and wearable AR systems. Hence, in this paper, we analyze what are the user experience issues and technical issues regarding common mobile-based AR systems, recently widespread phone-based AR systems, and rising wearable AR systems. Then for each issue, we propose and explain a new solution of how using projector-based AR can solve the problems and/or help enhance its user experiences. Our proposed framework includes hardware designs and architectures as well as a software computing paradigm towards mobile projector-based AR systems. The proposed design is evaluated by three experts using qualitative and semiquantitative research approaches.

Switchable Glass Enabled Contextualization for a Cyber-Physical Safe and Interactive Spatial Augmented Reality PCBA Manufacturing Inspection System

“…This advantage is extended to our system by integrating Vuforia-based PCBA tracking to facilitate the real-time alignment of projected virtual components upon transforming the coordinate frames. Additionally, to ensure the projection appearance is invariant to its intended state, we perform geometric and radiometric (photometric) compensation [ 21 ] prior to rendering the components upon the scene.…”

“…A key consideration to any SAR system is existence of crosstalk between projected imagery and camera feedback in the visible light spectrum in vision-based pro-cam devices [ 21 ]. Additionally, ambient light interference (brightness) [ 9 , 11 , 19 , 21 , 39 ], occlusion [ 7 , 19 , 39 ], deformation of projected information [ 7 , 11 ], low color fidelity [ 11 ], higher latency [ 11 ], aliasing artifacts, limited projector’s FOV [ 11 ], registration inaccuracy [ 11 , 13 ], projector mounting decision [ 13 ], and texture surface complexity (contrast) [ 11 , 19 , 21 , 39 ] (rectified by real-time radiometric compensation) causing transparent or confusing visual appearances [ 11 , 13 , 21 ] are important factors that are not ignorable when designing such a system. The authors in [ 40 ] propose a number of algorithm and hardware solutions to these challenges including, but not limited to, dually modulating or reallocating a light source in high dynamic range (HDR) projection, applying a surface modulation technique to project a static HDR image in ambient light, providing natural augmentation by employing high speed and low latency projection systems, subpixel shifting of overlapped multiple image planes for high resolution projection, applying more than three color primaries for multispectral projection, projecting view-dependent images from various directions to optically varying screens for light field projection, etc.…”

“…Furthermore, SAR allows lowering of skill level requirement during inspection as it projects digital information onto surfaces [ 15 ]. Despite these advantages, SAR is less mature and not well known yet [ 21 ].…”

Augmented reality as news

“…Overall, AR user experience has been studied in many contexts, however (see e.g. Dirin & Laine 2018;Ko, Chang, & Ji 2013;Siriborvornratanakul 2018;Zhou 2018).…”

Projection-Mapping-Based Object Pointing Using a High-Frame-Rate Camera-Projector System

“…The projection patterns and their complementarity at 120 Hz are sufficient to generate undetected projections to the HVS [35,36,37]. Color-wheel filter-based 3D projectors with the DLP principle can emit 120-Hz color-plane patterns [38,39].…”