Evaluating the Effectiveness of Mixed Reality Simulations for Developing UAV Systems

Abstract

Abstract. The development cycle of an Unmanned Aerial Vehicle (UAV) system can be long and challenging. Mixed Reality (MR) simulations can reduce cost, duration and risk of the development process by enabling the replacement of expensive, dangerous, or not yet fully developed components with virtual counterparts. However, there has been little validation of such hybrid simulation methods in practical robot applications. This paper evaluates the use of MR simulations for prototyping a UAV system to be deployed for a dairy farming monitoring task. We show that by augmenting the robot's sensing with a virtual moving cow using an extensible Augmented Reality (AR) tracking technique, MR simulations could help to provide efficient testing and identify improvements to the UAV controller. User study findings reveal the importance of both virtual and MR simulations to robot development, with MR simulations helping developers transition to development in a more physical environment.

Communication Architecture in Mixed-Reality Simulations of Unmanned Systems

“…They present a world where both virtual (SW) and physical (HW) objects and entities (further referred as the SWEs and HWEs for the virtual entities and entities embodied in the physical world, respectively) can co-exist and interact in real time. A combination of entities allows the system developers to obtain more insight into the behavior of the entities (e.g., by visualization of their inner states) and to perform much cheaper and safer experiments, with a part of the system being real and another part virtualized [ 12 , 13 ]. MR simulations also relieve simulators from recreating entire worlds since simulation occurs in a partially real world where certain phenomena, such as noise and complex physics, do not have to be modeled [ 14 , 15 ].…”