A comparison of natural user interface and graphical user interface for narrative in HMD-based augmented reality

Abstract

Over the years, the various mediums available for storytelling have progressively expanded, from spoken to written word, then to film, and now to Virtual Reality (VR) and Augmented Reality (AR). In 2016, the cutting-edge Head-Mounted Display (HMD) AR Microsoft HoloLens was released. However, though it has been several years, the quality of the user experience with narration using HMD-based AR technology has been rarely discussed. The present study explored interactive narrative in HMD-based AR regarding different user interfaces and their influence on users’ presence, narrative engagement and reflection. Inspired by an existing exhibition at the National Holocaust Centre and Museum in the U.K., a HoloLens narrative application, entitled The AR Journey, was developed by the authors using two different interaction methods, Natural User Interface (NUI) and Graphical User Interface (GUI), which were used to perform an empirical study. As revealed from the results of the between-subject design experiment, NUI exhibited statistically significant advantages in creating presence for users without 3D Role Playing Game (RPG) experience, and GUI was superior in creating presence and increasing narrative engagement for users with 3D RPG experience. As indicated by the results of the interviews, the overall narrative experience in HMD-based AR was acceptable, and the branching narrative design was engaging. However, HoloLens hardware issues, as well as virtuality and reality mismatch, adversely affected user experience. Design guidelines were proposed according to the qualitative results. Supplementary Information The online version contains supplementary material available at 10.1007/s11042-021-11723-0.

Conceptual Model of Augmented Reality Mobile Application Design (ARMAD) to Enhance user Experience: An Expert Review

“…As reported by [51], the AR UI that is supplemented with unnecessary information has confused the users and has caused them to have problems to understand the features. This is also supported by [52] who conducted a study on graphical user interface, which found that some of the users have reported funny or negative feelings when using the AR mobile applications, and the design may give users confusion with the unsuitable visual cues displayed on the mobile screen. The limited screen size of a mobile devices also plays a role in triggering user negative emotions, if the screen is cluttered or improperly designed [31], [47].…”

“…The limited screen size of a mobile devices also plays a role in triggering user negative emotions, if the screen is cluttered or improperly designed [31], [47]. Hence, [51], [52] has suggested to use proper and only necessary visual design elements in AR design. Therefore, there is a need to explore the possible design components for the AR-based mobile application design that can foster positive emotions and user experience.…”

Contextualising the Contested: Xr as Experimental Museology

“…Stuart Jeffrey (2015) describes the latter as a field which engages with modes of co-production, physical replication and aesthetic quality, authenticity (Jeffrey et al, 2017), and the complex entanglements of its own production (Huvila, 2012). A majority of practice-oriented XR scholarship focuses on user experience (Wither et al, 2010;Herbst et al, 2008;Hussein & Ali, 2022), technological experimentation (Cavallo et al, 2016;Jin et al, 2021;Shin et al, 2021;Hartmann & Vogel, 2021), pedagogical efficacy (Smørdal et al, 2016;Georgiou & Kyza, 2021;Remolar et al, 2021;Ibharim et al, 2021), or the "placeproducing dimensions of an experience-driven application" (Engberg, 2017, p.3). As an emerging field, a majority of studies involving XR are experimental but also, as Marcos Llobera (2012) notes, limited by the pixels and soundbites which constitute their virtual presence.…”

Leveraging Computer-Aided Education for Enhanced Learning: Innovations, Benefits, and Challenges in the Medical Sector

“…Argument realism is the term for a 2D or 3D virtual interface that improves our view of the outside world by superimposing digital content (more information) on top of it. Because we can always see the real world around us, true immersion in the virtual world is not possible [19].…”