Abstract. This paper describes Team Delft's robot, which won the Amazon Picking Challenge 2016, including both the Picking and the Stowing competitions. The goal of the challenge is to automate pick and place operations in unstructured environments, specifically the shelves in an Amazon warehouse. Team Delft's robot is based on an industrial robot arm, 3D cameras and a customized gripper. The robot's software uses ROS to integrate off-the-shelf components and modules developed specifically for the competition, implementing Deep Learning and other AI techniques for object recognition and pose estimation, grasp planning and motion planning. This paper describes the main components in the system, and discusses its performance and results at the Amazon Picking Challenge 2016 finals.

No abstract

No abstract

This paper describes Team Delft's robot winning the Amazon Robotics Challenge 2016. The competition involves automating pick and place operations in semistructured environments, specifically the shelves in an Amazon warehouse. Team Delft's entry demonstrated that the current robot technology can already address most of the challenges in product handling: object recognition, grasping, motion, or task planning; under broad yet bounded conditions. The system combines an industrial robot arm, 3-D cameras and a custom gripper. The robot's software is based on the robot operating system to implement solutions based on deep learning and other state-of-the-art artificial intelligence techniques, and to integrate them with off-the-shelf components. From the experience developing the robotic system, it was concluded that: 1) the specific task conditions should guide the selection of the solution for each capability required; 2) understanding the characteristics of the individual solutions and the assumptions they embed is critical to integrate a performing system from them; and 3) this characterization can be based on "levels of robot automation." This paper proposes automation levels based on the usage of information at design or runtime to drive the robot's behavior, and uses them to discuss Team Delft's design solution and the lessons learned from this robot development experience.

Sampling-based kinodynamic planners, such as Rapidly-exploring Random Trees (RRTs), pose two fundamental challenges: computing a reliable (pseudo-)metric for the distance between two randomly sampled nodes, and computing a steering input to connect the nodes. The core of these challenges is a Two Point Boundary Value Problem, which is known to be NP-hard. Recently, the distance metric has been approximated using supervised learning, reducing computation time drastically. The previous work on such learning RRTs use direct optimal control to generate the data for supervised learning. This paper proposes to use indirect optimal control instead, because it provides two benefits: it reduces the computational effort to generate the data, and it provides a low dimensional parametrization of the action space. The latter allows us to learn both the distance metric and the steering input to connect two nodes. This eliminates the need for a local planner in learning RRTs. Experimental results on a pendulum swing up show 10-fold speed-up in both the offline data generation and the online planning time, leading to at least a 10-fold speed-up in the overall planning time. IntroductionFor motion planning of robotic manipulators, kinodynamic planning and sampling-based planning are getting increasingly popular. Kinodynamic planning, i.e., planning in state-space rather than configuration space, improves robustness, speed and energy efficiency of robots [3,28,19]. Sampling based planning has been shown to be the most viable way to handle high dimensional spaces and obstacles [9,11]. In this paper, we will consider how to apply Rapidly-exploring Random Trees [11], the most popular sampling-based planning algorithm, to kinodynamic planning.RRT builds a tree graph structure with the states of the system as tree nodes and the trajectories of the system be-