A Bin-Picking Benchmark for Systematic Evaluation of Robotic Pick-and-Place Systems

Abstract

Pick-and-place operations constitute the majority of today's industrial robotic applications. However, comparability and reproducibility of results has remained an issue that delays further advances in this field. Evaluation of manipulation systems can be carried out at different levels, but for the final application the performance of the overall system is the critical one. This paper proposes a benchmarking framework for pick-and-place tasks, inspired by a typical task in the logistic domain: picking up fruits and vegetables from a container and placing them in an order bin. The framework uses an easy-to-reproduce environment, a publicly available object set, and guidelines for creating scenarios of different complexity. The proposed benchmark is applied to evaluate the performance of four variants of a robotic system with different end-effectors.

Experimental Evaluation of Tactile Sensors for Compliant Robotic Hands

“…As a test case to verify the benefits of the tactile sensors, we used the established benchmark in Mnyusiwalla et al (2020), where multiple grippers failed in the most challenging scenarios. Thinking on this use case, we decided to create a multi-modal fingertip that integrates IMU, a time of flight sensor VL6180X, and two 3-axis magnetic sensors MLX90395.…”

“…To verify the interface and its behavior in practice, we used a simple test scenario. The user has to program how to grasp a punnet in the benchmark scenario P2 in Mnyusiwalla et al (2020). A direct grasp with CLASH 3F is not possible, as a two-finger diagonal grasp is not reachable with the kinematics of the hand.…”

“…To test the applicability of the selected sensors, we use a published benchmarking framework for pick-and-place tasks, designed originally for the logistic domain (Mnyusiwalla et al, 2020). It entails picking up fruits and vegetables from a container and placing them in an order bin.…”

“…Besides the special problems we solved for the two particular scenarios presented here, for example a special vision pipeline for the cucumbers, the selected tactile sensors will improve the results for all other objects in the benchmark in Mnyusiwalla et al (2020), as they will allow for instance precise pre-position of the fingers in the cluttered crate if necessary. Slip detection will help to find the lowest possible grasp force and reduce damages or lost objects during transportation.…”

“…To verify the applicability of the sensors to a real use case, we used a benchmark proposed in Mnyusiwalla et al (2020) for logistic use cases. We used two benchmark scenarios involving cucumbers and punnets, where all grippers shown in Mnyusiwalla et al (2020) failed, and we tested here if additional sensor information helps to solve the intended tasks. After trying the two selected scenarios using the CLASH 2F, we verified that now our system performs much better.…”