Addressing perception uncertainty induced failure modes in robotic bin-picking

Abstract

We present a comprehensive approach to handle perception uncertainty to reduce failure rates in robotic bin-picking. Our focus is on mixed-bins. We identify the main failure modes at various stages of the bin-picking task and present methods to recover from them. If uncertainty in part detection leads to perception failure, then human intervention is invoked. Our approach estimates the confidence in the part match provided by an automated perception system, which is used to detect perception failures. Human intervention is also invoked if uncertainty in estimated part location and orientation leads to a singulation planning failure. We have developed a user interface that enables remote human interventions when necessary. Finally, if uncertainty in part posture in the gripper leads to failure in placing the part with the desired accuracy, sensor-less fine-positioning moves are used to correct the final placement errors. We have developed a fine-positioning planner with a suite of fine-motion strategies that offer different tradeoffs between completion time and postural accuracy at the destination. We report our observations from system characterization experiments with a dual-armed Baxter robot, equipped with a Ensenso three-dimensional camera, to perform bin-picking on mixed-bins.

Handling Perception Uncertainty in Simulation-Based Singulation Planning for Robotic Bin Picking

“…The part was placed in the bin and the boundary of the part was marked such that the part could be placed in the same position and orientation for later parts of the experiment. The point cloud of the scene was captured and a user interface designed in MAT-LAB was used to estimate the pose of the part by manually docking the CAD model of the part in the point cloud [23]. The points belonging to the part were deleted from the point cloud and the point cloud of the CAD model was rendered at the pose of the part.…”

“…They used a perception module to evaluate the outcome of pushing actions. In our previous work [23, 24], any failure in the automatic pose estimation was handled with the help of human assistance. The human operator used an interface to estimate the pose of the part and sent it back to the robot.…”

Design of Hybrid Cells to Facilitate Safe and Efficient Human–Robot Collaboration During Assembly Operations

“…We will compliment this capability in the future by exploiting the KUKA robot's inbuilt force sensing and impedance control features to implement compliant control for handling postcollision scenarios. In our previous work, we have developed other modules including ontology for task partitioning in human-robot collaboration for kitting operations [55] and resolving perception uncertainties [56] and occlusions in robotic bin-picking in hybrid cells [57]. Future work consists of investigating how to integrate them into the development of hybrid work cells for assembly applications.…”

Improving 3D vision-robot registration for assembly tasks

“…For example, a larger uncertainty of the starting point for the search of the hole center using F/T sensors led to increased search time [27]. Kaipa et al generated singulation strategies for binned parts and showed that the probability of a successful singulation diminished with increasing error of the part position [28]. Mahler et al reported that increased pose error of a part degraded the quality of grasp planning even after the application of sophisticated algorithms to reduce these errors [29].…”