Augmented Reality and Internet of Things – Implementation in Projects by Using Simplified Robotic Models

Abstract

Augmented reality (AR) and internet of things (IoT), technologies that have recently expanded in various fields of theory, scientific researches, and business practice, provide innovative approach for development of projects and their implementation. In this paper, one approach of the implementation is undertaken as an experimental use case, applied on a simplified robotic model of centrifuge for pilot training and flight simulation. An AR/IoT mobile app is designed to be developed in two phases. The first phase is finished, and the second one is under development. The app contains four main parts with the following functions: presentation, monitor/control, pilot testing, and analytics. The system and possibilities of the implementation of the app in the belonging project are explained in the paper. This use case indicates that AR/IoT apps can be developed by using simplified robotic models and later transferred to the real objects that are subject of research projects, even before their actual construction.

Multimodal Interaction Systems Based on Internet of Things and Augmented Reality: A Systematic Literature Review

“…AR is used to visualize graphical objects on a view of the real world, where a real-time camera feed on a device or head-mounted display (HMD) typically represents the view. AR objects can not only be used as simple data representations [9,10], but can also be combined with an input modality, such as touch on a mobile device [11], as part of a user interface (UI) to allow an agent to interact with an MI system [12,13]. This property distinguishes between applications that only utilize AR as a data visualization tool and those that use AR as part of their UI to interact with the underlying MI system.…”

“…The combination of AR and IoT technologies makes it possible to bridge the real world, virtual world (i.e., digital content), and ambient environment (i.e., information from IoT sensors), thereby increasing the richness of information and opening possibilities for context-aware application development. For example, visualized data of an ambient environment can serve as digital content that is superimposed on the real world [10,12,13]. However, the use and development of multimodal interaction in applications that combine AR and IoT has remained a little researched area.…”

“…IoT and AR are used differently in MI systems, depending on the intended purpose. For example, Dodevska et al [10] utilized AR as a data visualization tool by displaying data from IoT sensors that were attached to a centrifuge, whereas Sun et al [12] created an AR interface that could be manipulated by an agent's hand gestures to control IoT devices. The agent was able to send commands to control an IoT device in both studies.…”

“…The MI system that was developed by Dodevska et al [10] is a case that employs the architecture displayed in Figure 4c. The AR mobile application connected to their MI system can recognize an AR marker placed on a miniature centrifuge.…”

The concept and application of simplified robotic models

“…A primitive model of the centrifuge which did not have all functionalities (there were only two rotational axes) was built from LEGO elements at the end of 2018, as well as an AR/IoT mobile app under development [9]. In this paper, a new advanced version of this SRM is presented.…”