Remote microscale teleoperation through virtual reality and haptic feedback

Abstract

This paper reports the remote handling of microscale objects, between two sites approximately 630 km distant. To manipulate objects less than 10 µm, specific equipments such as AFM (Atomic Force Microscope) cantilevers integrated into a SEM (Scanning Electron Microscope) are generally required. Enabling remote access to such a system would benefit any micro/nanoresearcher. However, vision feedback and sensor data of a micromanipulation system are generally limited, hence the implementation of a teleoperation scenario is not straightforward. Specific tools are proposed here for an intuitive manipulation in a wide range of applications. To ensure ease of manipulation, both a 3D virtual representation of the scene and haptic feedback are provided. Force sensor feedback is limited since only two measures are available. In order to extend this information, vision algorithms are developed to estimate the respective positions of the tool and objects, which are then used to calculate the haptic feedback. The stability of the overall scheme is very sensitive to time delays. This requirement is taken into account in vision algorithms and the communication module which transfers the data between the two remote sites. In addition, the proposed robotic control architecture is modular so that the platform can be used for a wide range of applications. First results are obtained on a teleoperation between Paris, France, and Oldenburg, Germany.

Intention prediction approach to interact naturally with the microworld

“…Some recent works have focused on providing both haptic and 3D virtual reality feedbacks to enable the operator to manipulate micro objects intuitively in a wide range of applications [10] [11]. Virtual reality simulators are interesting solutions to provide users with a reconstructed 3D view of a manipulation scene, which can be displayed at the human scale and from different vantage points.…”

Experiments on Intercontinental Haptic Control of Multiple UAVs

“…This problem has been investigated in [11] by simulating a visual feedback in the operator site through an intercountry experiment. To the best of our knowledge, however, there was no result in the bilateral teleoperation of multi-robot UAV systems which were truly communicating over an intercontinental Internet connection.…”

Stable haptic feedback based on a dynamic vision sensor for microrobotics

“…The first pick-and-place manipulation with 3D haptic feedback using a microgripper is successfully achieved. This work is based on several previous studies on the stability of haptic coupling schemes for applications at microscales [26], on the definition of virtual guides for pick-and-place operation of microspheres based on two atomic force microscopy cantilevers [27], on the use of visual feedback coming from a scanning electron microscope for teleoperation [28] and on the use of dynamic vision sensors [29]. However, it is the first time, to our knowledge, that dynamic vision sensors are used to provide stable haptic feedback to assist users while performing a pick-and-place operation.…”