Vision-assisted micromanipulation using closed-loop actuation of multiple microrobots

Abstract

Accurate control and precise positioning of opto-thermocapillary flow-addressed bubble microrobots are necessary for micromanipulation. In addition, micromanipulation using the simultaneous actuation of multiple microrobots requires a robust control system to enable independent motion. This paper demonstrates a hybrid closed-loop vision-assisted control system capable of actuating multiple microrobots simultaneously and positioning them at precise locations relative to micro-objects under manipulation. A vision-assisted grasp-planning application was developed and used to calculate the necessary trajectories of the microrobots to form cages around micro-objects. The location of the microrobots and the micro-objects was detected at the caging locations using a particle-tracking application that used image feedback for precise positioning. The closed-loop image feedback information enabled the position update of the microrobots, allowing them to precisely follow the trajectory and caging locations calculated by the grasp-planning application. Four microrobots were assigned to cage a star-shaped micro-object using the closed-loop control system. Once caged, the micro-object was transported to a location within the workspace and uncaged, demonstrating the micromanipulation task. This microrobotic system is well suited for the micromanipulation of single cells.Electronic supplementary materialThe online version of this article (doi:10.1186/s40638-017-0064-4) contains supplementary material, which is available to authorized users.

Autonomous Microrobotic Manipulation Using Visual Servo Control

“…The experiments demonstrated that our multiphase controller can be used to direct a microrobot to position a manipuland. Average positional accuracies of 8.33 pixels (66 µm) were demonstrated over numerous trials, which is comparable to the positional accuracies described in prior work [20,22,25,26]. The performance of this controller could be improved by basing the controller mode not simply on microrobot-manipuland separation but on the measured observation of the degree to which manipuland position is affected by microrobot motion.…”

“…Banerjee and Gupta provide a review of research in automated planning and control for micromanipulation in [18]. A variety of groups [19][20][21][22][23][24][25][26] have demonstrated micromanipulation of one or more manipulands in fluidic environments. Table 1 compares a number of these works, looking at the types and amount of actuation, path planning, feedback control, release strategy, and positioning accuracy.…”

“…By varying the commanded robot position for each phase, we are able to autonomously direct the microrobot to push the manipuland into position. Many groups have demonstrated this approach [20,22,[24][25][26]. The separation of the robot from the manipuland once the manipuland has been brought into position (termed exit stage) can be problematic in micromanipulation systems in which van der Waals forces and local flow fields can cause the manipuland to retract with the microrobot [20,26].…”

Micromanipulation With Microrobots

“…2). OFB microrobots have demonstrated micromanipulation with increased resolution with caging and grasping [64]. As an example, a spherical OFB microrobot has difficulty pushing or pulling a spherical micro-object on a linear trajectory [62].…”

Thermocapillary micromanipulation: force characterization and Cheerios interactions

“…Many applications require to manipulate several particles simultaneously and independently, for example to perform assembly of components, or to manipulate objects indirectly by using several microrobots that make a cage around them [20]. However, multiple particles handling remains challenging.…”