Stable haptic feedback based on a dynamic vision sensor for microrobotics

Abstract

Abstract-This work presents a stable vision based haptic feedback for micromanipulation using both an asynchronous Address Event Representation (AER) silicon retina and a conventional frame-based camera. At this scale, most of the grippers used to manipulate objects lack of force sensing. High frequency vision detection thus provides a sound solution to get information about the position of the object and the tool to provide virtual haptic guides. Artificial retinas present high update rates, which enables to address one of the major challenge of haptic feedback teleoperation systems, namely stability. However static objects are not detected. The haptic feedback is thus based on an asynchronous silicon retina to provide a high update rate of moving objects and a framebased camera to retrieve the position of the target object. This approach is validated by pick-and-place of microspheres (diameter: around 50 micrometers) using a piezoelectric microgripper. The displacement of the tool, as well as the opening and closing of the gripper are controlled by the haptic device. Haptic virtual guides are transmitted to users to assist them in the different steps of the pick-and-place task: a virtual stiffness ensures the correct alignment of the tool with respect to the object, a repulsive haptic force enables users to monitor the gripping step, and operators are assisted while picking and placing the object.

Inspired by nature: timescale-free and grid-free event-based computing with spiking neural networks

“…This is important for decision making and other cognitive functions. The many studies conducted with the DVS and similar event-based visual devices have covered a variety of tasks ranging from classification and recognition [6-8, 11, 14, 21, 24, 29, 31-33, 39, 41-46], tracking [10, 16-18, 22, 23, 26, 28, 30, 37, 38, 40, 47-50], stereo matching and depth reconstruction [19,[51][52][53][54], robotic applications [9,12,13,16,17,20,25,27,30,37,38,47,48,[55][56][57], and others [15,36,[58][59][60][61][62][63][64]. The prediction task, however, remains infrequently researched [63,64].…”

“…As the DVS is built for mobile robotic use, there are many studies that utilise the DVS with some robotic platform [12,13,16,17,25,30,37,38,47,48,55,56]. Robotic studies usually first recognise or track certain objects, and then control the robot motor action accordingly to accomplish their goals.…”

“…• Tracking the position of a probe which then guides an optic tweezer for micro-operation [16,30,47,48] • Tracking a moving robot [10] • Tracking the position of a pencil and balancing it on its tip [17] • Multiple microsphere tracking [18] • Ball movement tracking which drives a robotic goalie arm [37,38] • Part-based shape tracking [49] • Fingertip tracking for a touchless user interface [50] • Tracking particles [26] • Tracking lane changing cars [22] • Tracking shapes [23] • Tracking movement of multiple people [40] • Tracking features and edges [28] Stereo matching and depth reconstruction…”

“…• Probe position tracking and guiding a pair of robotic optic tweezer for micro-operation [16,30,47,48] • Tracking the position of a pencil and balancing it on its tips [17] • Tracking a ball's movement with a robotic goalie arm and blocking it accordingly [37,38] • Open loop control of a robot based on DVS inputs [25,55] • Visual attention [12,13] • Generate visual odometry [56] Continued on next page • Using particle filtering to track a pre-marked line of a ceiling for self-localisation [20] and an extended SLAM version [27] • Self-localisation of a flying robot with active LED marking tracking [57] • Localisation of micro aerial vehicles [9] Others…”

“…Other studies processed the events as they were available while accumulating the processed events for further computation [29,41,42,46,62]. Others incorporated the temporal information by making old events [30,48] Each approach to handle timing information has its pros and cons. A frame-based approach can be used directly with traditional visual processing paradigms but the precise timing information is lost.…”