Mixed reality for robotics

Hönig, Wolfgang; Milanes, Christina; Scaria, Lisa; Phan, Thai; Bolas, Mark; Ayanian, Nora

doi:10.1109/iros.2015.7354138

Cited by 132 publications

(74 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…To get the high-quality tracking of the quadrotor during the experiments, we used Vicon motion capture system with 12 cameras (Vantage V5) covering a 5m × 5m × 5m space. We used the Robot Operating System (ROS) Kinetic framework to run the developed software and ROS stack [Hönig et al 2015;Preiss et al 2017] for Crazyflie 2.0. The position and attitude update rate was 100 Hz for all drones.…”

Section: Methodsmentioning

confidence: 99%

SlingDrone: Mixed Reality System for Pointing and Interaction Using a Single Drone

Tsykunov

Ibrahimov

Vasquez

et al. 2019

25th ACM Symposium on Virtual Reality Software and Technology

View full text Add to dashboard Cite

Figure 1: a) The drone points in the slingshot mode: the user pulls the drone using a leash and a hand grip causing a displacement. b) While the drone is in slingshot mode, the trajectory and the object of interest is visualized in virtual scene by simplifying pointing. c) The drone is in projectile mode: it approaches and grabs the object of interest. ABSTRACTWe propose SlingDrone, a novel Mixed Reality interaction paradigm that utilizes a micro-quadrotor as both pointing controller and interactive robot with a slingshot motion type. The drone attempts to hover at a given position while the human pulls it in desired direction using a hand grip and a leash. Based on the displacement, a virtual trajectory is defined. To allow for intuitive and simple control, we use virtual reality (VR) technology to trace the path of the drone based on the displacement input. The user receives force feedback propagated through the leash. Force feedback from SlingDrone coupled with visualized trajectory in VR creates an intuitive and user friendly pointing device. When the drone is released, it follows the trajectory that was shown in VR. Onboard payload (e.g. magnetic gripper) can perform various scenarios for real interaction with the surroundings, e.g. manipulation or sensing. Unlike HTC Vive controller, SlingDrone does not require handheld devices, thus it can be used as a standalone pointing technology in VR.

show abstract

Section: Methodsmentioning

confidence: 99%

SlingDrone: Mixed Reality System for Pointing and Interaction Using a Single Drone

Tsykunov

Ibrahimov

Vasquez

et al. 2019

25th ACM Symposium on Virtual Reality Software and Technology

View full text Add to dashboard Cite

show abstract

“…To get the high-quality tracking of the quadrotors and human glove during the experiments, we used Vicon motion capture system with 12 cameras (Vantage V5) covering a 5 m 5 m 5 m space. We used the Robot Operating System (ROS) Kinetic framework to run the development software and ROS stack [17] for Crazyflie 2.0. The position and attitude update rate was 60 Hz for all drones.…”

Section: Verificationmentioning

confidence: 99%

“…Before conducting any type of experiment, we ensured that we were able to perform a stable and smooth flight, following the desired trajectory. In order to do so, all PID coefficients for position controller were set to default values for Crazyflie 2.0, according to [17] (for x,y-axis…”

Section: Verificationmentioning

confidence: 99%

SwarmTouch: Guiding a Swarm of Micro-Quadrotors With Impedance Control Using a Wearable Tactile Interface

Tsykunov

Agishev

Ibrahimov

et al. 2019

IEEE Trans. Haptics

View full text Add to dashboard Cite

To achieve a smooth and safe guiding of a drone formation by a human operator, we propose a novel interaction strategy for a human-swarm communication which combines impedance control and vibrotactile feedback. The presented approach takes into account the human hand velocity and changes the formation shape and dynamics accordingly using impedance interlinks simulated between quadrotors, which helps to achieve a natural swarm behavior. Several tactile patterns representing static and dynamic parameters of the swarm are proposed. The user feels the state of the swarm at the fingertips and receives valuable information to improve the controllability of the complex formation. A user study revealed the patterns with high recognition rates. A flight experiment demonstrated the possibility to accurately navigate the formation in a cluttered environment using only tactile feedback. Subjects stated that tactile sensation allows guiding the drone formation through obstacles and makes the human-swarm communication more interactive. The proposed technology can potentially have a strong impact on the human-swarm interaction, providing a higher level of awareness during the swarm navigation.Index Terms-Human-robot interaction, tactile display, wearable computers ! • All authors are with the

show abstract

“…Other works have proposed systems for augmented reality for robots that present a structure very similar to ARK [14], [15], [16], [17], [18], [19], [20]. In all these works, the robots' locations are tracked through overhead cameras connected to a base control station which then delivers virtual environment information to the robots.…”

Section: Related Workmentioning

confidence: 99%

“…In most works, the system is designed for a specific use case (rather than as a general purpose tool); for example, to generate virtual pheromone trails that are either projected on the ground through an overhead projector [14], [16] or displayed on an LCD screen which acts as the robots' ground [21]. Alternatively in [17], [19], robots can sense any type of virtual environment. However, the robots used in these studies are equipped with unique markers for recognition and wireless modules for direct individual communication.…”

Section: Related Workmentioning

confidence: 99%

ARK: Augmented Reality for Kilobots

Reina

Cope

Nikolaidis³

et al. 2017

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

Abstract-Working with large swarms of robots has challenges in calibration, sensing, tracking, and control due to the associated scalability and time requirements. Kilobots solve this through their ease of maintenance and programming, and are widely used in several research laboratories worldwide where their low cost enables large-scale swarms studies. However, the small, inexpensive nature of the Kilobots limits their range of capabilities as they are only equipped with a single sensor. In some studies, this limitation can be a source of motivation and inspiration, while in others it is an impediment. As such, we designed, implemented, and tested a novel system to communicate personalised location-and-state-based information to each robot, and receive information on each robots' state. In this way, the Kilobots can sense additional information from a virtual environment in real-time; for example, a value on a gradient, a direction towards a reference point or a pheromone trail. The Augmented Reality for Kilobots (ARK) system implements this in flexible base control software which allows users to define varying virtual environments within a single experiment using integrated overhead tracking and control. We showcase the different functionalities of the system through three demos involving hundreds of Kilobots. The ARK provides Kilobots with additional and unique capabilities through an open-source tool which can be implemented with inexpensive, off-the-shelf hardware.

show abstract

Mixed reality for robotics

Cited by 132 publications

References 14 publications

SlingDrone: Mixed Reality System for Pointing and Interaction Using a Single Drone

SlingDrone: Mixed Reality System for Pointing and Interaction Using a Single Drone

SwarmTouch: Guiding a Swarm of Micro-Quadrotors With Impedance Control Using a Wearable Tactile Interface

ARK: Augmented Reality for Kilobots

Contact Info

Product

Resources

About