Augmented reality for robots: Virtual sensing technology applied to a swarm of e-pucks

Abstract

We present a novel technology that allows real robots to perceive an augmented reality environment through virtual sensors. Virtual sensors are a useful and desirable technology for research activities because they allow researchers to quickly and efficiently perform experiments that would otherwise be more expensive, or even impossible. In particular, augmented reality is useful (i) for prototyping and assessing the impact of new sensors before they are physically produced; and (ii) for developing and studying the behaviour of robots that should deal with phenomena that cannot be easily reproduced in a laboratory environment because, for example, they are dangerous (e.g., fire, radiations). We realised an augmented reality system for robots in which a simulator retrieves real-time data on the real environment through a multi-camera tracking system and delivers post-processed information to the robot swarm according to each robot's sensing range. We illustrate the proposed virtual sensing technology through an experiment involving 15 e-pucks.

Sophisticated collective foraging with minimalist agents: a swarm robotics test

“…Smart environments may be difficult to install and use for real applications; rather, such setups are often employed for targeted research experiments. This category can be further divided into three classes: the usage of (i) radio-frequency identification (RFID) tags Zambonelli 2005, 2007;Herianto et al 2007;Herianto and Kurabayashi 2009;Bosien et al 2012;Khaliq et al 2014); (ii) simulated pheromone environments, using projected light or other custom hardware for virtual pheromone implementations (Sugawara et al 2004;Garnier et al 2007Garnier et al , 2013Arvin et al 2015;Valentini et al 2018) , and (iii) augmented reality tools in which a virtual environment is sensed and acted on by robots using virtual sensors and actuators (Reina et al 2015b(Reina et al , 2017.…”

ARK: Augmented Reality for Kilobots

“…Other works have proposed systems for augmented reality for robots that present a structure very similar to ARK [14], [15], [16], [17], [18], [19], [20]. In all these works, the robots' locations are tracked through overhead cameras connected to a base control station which then delivers virtual environment information to the robots.…”

“…In most works, the system is designed for a specific use case (rather than as a general purpose tool); for example, to generate virtual pheromone trails that are either projected on the ground through an overhead projector [14], [16] or displayed on an LCD screen which acts as the robots' ground [21]. Alternatively in [17], [19], robots can sense any type of virtual environment. However, the robots used in these studies are equipped with unique markers for recognition and wireless modules for direct individual communication.…”

The Pi-puck extension board: A raspberry Pi interface for the e-puck robot platform

“…Like the BRL Linux extension board, the Gumstix Overo COM turret has enhanced the e-puck's usefulness as a research platform, and has been used for the automatic design of robot controllers [17], implementing virtual sensors [18], and running ARGoS controllers on real robots [19]. However, it suffers from similar software maintenance issues -the turret connects to the e-puck's camera via the OMAP Camera Interface Subsystem, necessitating customisations to the Linux kernel to provide drivers for the camera sensor.…”