Evaluating Usability and User Experience of AR Applications in VR Simulation

Abstract

Validating an augmented reality application in a virtual reality simulation can offer many advantages compared to testing in real conditions and can speed up development processes. With such a simulation, developers and designers do not need to have constant physical access to the real place. They can save physical navigation, experiment with different kinds of devices and isolate testing parameters. While the validity of functional testing in virtual reality simulations is not particularly challenged, the validity of such simulations to evaluate user experience and usability, similarly as in real conditions, still needs to be assessed. We then conducted a user study to explore the validity of evaluating these criteria with a virtual reality simulation tool and the importance of simulation fidelity for that purpose. Indeed, we also seek to determine whether it is necessary to simulate the limited field of view of augmented reality glasses and if the simulation can take place in a virtual world that is not a replica of the real targeted environment. To do so, we have developed an augmented reality application for smart-homes where a user can interact with different connected objects. One group of users performed the experiment in the real place with augmented reality glasses and three other groups performed the same experiment in virtual reality with various simulation conditions (field of view and environment). Users’ subjective feedback and quantitative results only highlight very few differences between real-world conditions and simulation in virtual reality, whatever the simulation parameters used. These results suggest the interest in using virtual reality simulation to evaluate an augmented reality application but should be confirmed on other use cases and interaction tasks.

VR Simulation and Implementation of Robotics: A Tool for Streamlining and Optimization

“…Both subjective feedback and quantitative user results revealed minimal differences between real conditions and VR simulations, regardless of the simulation parameters. These findings suggest the potential of using VR simulations to evaluate AR applications, although further validation across different scenarios and interactions is necessary [28].…”

Design a Virtual Reality Interface to Enhance the User Experience of Intangible Cultural Heritage Protection

“…If users are confused, inconvenient, or dissatisfied during use, they may give up or switch to other better options. On the contrary, if users have a good experience, they will be more willing to use the VR interactive interface for a long time and improve their participation in the protection of intangible cultural heritage (Lacoche et al, 2022).…”

The UUXR-Framework: A Draft Classification for Using Extended Reality in Usability and User Experience Research

“…The current technological barriers to immersive technologies are shrinking and the differences to the real world for the user are becoming smaller and smaller. This has an effect on the use of XR technologies in user research [17].…”