Cartman: The Low-Cost Cartesian Manipulator that Won the Amazon Robotics Challenge

Morrison, D.; Tow, Adam W.; McTaggart, M.; Smith, Robert; Kelly-Boxall, N.; Wade-McCue, S.; Erskine, J.; Grinover, R.; Gurman, Alan S.; Hunn, T.; Lee, D.; Milan, Anton; Pham, Trung; Rallos, G.; Razjigaev, Andrew; Röggla, Thomas; Kotra, Vijay; Zhuang, Zheyu; Lehnert, Christopher; Reid, Ian; Corke, Peter; Leitner, Jürgen

doi:10.1109/icra.2018.8463191

Cited by 125 publications

(95 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In 2016, many teams used deep learning to segment objects for the alignment phase, training semantic segmentation networks with separate classes for each object instance on hand-labeled [46] or self-supervised datasets [9]. Team ACRV, the winners of the 2017 ARC, fine-tuned RefineNet to segment and classify 40 unique known objects in a bin, with a system to quickly learn new items with a semi-automated procedure [10,47]. In contrast, our method uses deep learning for category-agnostic segmentation, which can can be used to segment a wide variety of objects not seen in training.…”

Section: Related Workmentioning

confidence: 99%

“…An attempt was considered successful if the robot lifted the target object out of the bin and successfully transported the object to a receptacle. One approach to this problem is to collect real images of the items piled in the bin, labeling object masks in each image, and using that data to train or fine-tune a deep neural network for object classification and segmentation [10,11]. However, that data collection process is time consuming and must be re-performed for new object sets, and training and fine-tuning a Mask R-CNN can take some time.…”

Section: B Precision-recall Evaluationmentioning

confidence: 99%

“…Adding new object classes or generating data for new types of environments requires additional manual labeling. Thus, in robotics, pixelwise object segmentation is often avoided [8,9] or used for a small number of object classes, where semantic segmentation networks [10,11] or predefined features [12] can be used.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Segmenting Unknown 3D Objects from Real Depth Images using Mask R-CNN Trained on Synthetic Data

Danielczuk

Matl

Gupta

et al. 2019

2019 International Conference on Robotics and Automation (ICRA)

176

108

View full text Add to dashboard Cite

The ability to segment unknown objects in depth images has potential to enhance robot skills in grasping and object tracking. Recent computer vision research has demonstrated that Mask R-CNN can be trained to segment specific categories of objects in RGB images when massive handlabeled datasets are available. As generating these datasets is time-consuming, we instead train with synthetic depth images. Many robots now use depth sensors, and recent results suggest training on synthetic depth data can transfer successfully to the real world. We present a method for automated dataset generation and rapidly generate a synthetic training dataset of 50,000 depth images and 320,000 object masks using simulated heaps of 3D CAD models. We train a variant of Mask R-CNN with domain randomization on the generated dataset to perform category-agnostic instance segmentation without any handlabeled data and we evaluate the trained network, which we refer to as Synthetic Depth (SD) Mask R-CNN, on a set of real, high-resolution depth images of challenging, densely-cluttered bins containing objects with highly-varied geometry. SD Mask R-CNN outperforms point cloud clustering baselines by an absolute 15% in Average Precision and 20% in Average Recall on COCO benchmarks, and achieves performance levels similar to a Mask R-CNN trained on a massive, hand-labeled RGB dataset and fine-tuned on real images from the experimental setup. We deploy the model in an instance-specific grasping pipeline to demonstrate its usefulness in a robotics application. Code, the synthetic training dataset, and supplementary material are available at https://bit.ly/2letCuE. arXiv:1809.05825v2 [cs.CV]

show abstract

Section: Related Workmentioning

confidence: 99%

Section: B Precision-recall Evaluationmentioning

confidence: 99%

See 1 more Smart Citation

Segmenting Unknown 3D Objects from Real Depth Images using Mask R-CNN Trained on Synthetic Data

Danielczuk

Matl

Gupta

et al. 2019

2019 International Conference on Robotics and Automation (ICRA)

176

108

View full text Add to dashboard Cite

show abstract

“…Lastly, many researchers have presented complete robotic pick-and-place systems [28], [29], offering a broader view of different modules such as motion planning, perception, control and grasping. Regarding motion planning, these reported systems commonly employ reactive strategies that are based on online visual feedback.…”

Section: Related Workmentioning

confidence: 99%

Critically fast pick-and-place with suction cups

Phạm

Pham

2019

2019 International Conference on Robotics and Automation (ICRA)

View full text Add to dashboard Cite

Fast robotics pick-and-place with suction cups is a crucial component in the current development of automation in logistics (factory lines, e-commerce, etc.). By "critically fast" we mean the fastest possible movement for transporting an object such that it does not slip or fall from the suction cup. The main difficulties are: (i) handling the contact between the suction cup and the object, which fundamentally involves kinodynamic constraints; and (ii) doing so at a low computational cost, typically a few hundreds of milliseconds. To address these difficulties, we propose (a) a model for suction cup contacts, (b) a procedure to identify the contact stability constraint based on that model, and (c) a pipeline to parameterize, in a time-optimal manner, arbitrary geometric paths under the identified contact stability constraint. We experimentally validate the proposed pipeline on a physical robot system: the cycle time for a typical pick-and-place task was less than 5 seconds, planning and execution times included. The full pipeline is released as opensource for the robotics community.

show abstract

“…While bin packing is studied extensively, to the best of the authors' knowledge there are few attempts to deploy bin packing solutions on real robots, where inaccuracies in vision and control are taken into account. Such inaccuracies have been considered in the context of efforts relating to the Amazon Robotics Challenge [12] [13] [14] [15] [16] [17] but most of these systems do not deal with bin packing. Most deployments of automatic packing use mechanical components, such as conveyor trays, that are specifically designed for certain products [18], rendering them difficult to customize and deploy.…”

Section: Related Workmentioning

confidence: 99%

Towards Robust Product Packing with a Minimalistic End-Effector

Shome

Tang

Song

et al. 2019

2019 International Conference on Robotics and Automation (ICRA)

View full text Add to dashboard Cite

Advances in sensor technologies, object detection algorithms, planning frameworks and hardware designs have motivated the deployment of robots in warehouse automation. A variety of such applications, like order fulfillment or packing tasks, require picking objects from unstructured piles and carefully arranging them in bins or containers. Desirable solutions need to be low-cost, easily deployable and controllable, making minimalistic hardware choices desirable. The challenge in designing an effective solution to this problem relates to appropriately integrating multiple components, so as to achieve a robust pipeline that minimizes failure conditions. The current work proposes a complete pipeline for solving such packing tasks, given access only to RGB-D data and a single robot arm with a vacuum-based end-effector, which is also used as a pushing finger. To achieve the desired level of robustness, three key manipulation primitives are identified, which take advantage of the environment and simple operations to successfully pack multiple cubic objects. The overall approach is demonstrated to be robust to execution and perception errors. The impact of each manipulation primitive is evaluated by considering different versions of the proposed pipeline, which incrementally introduce reasoning about object poses and corrective manipulation actions.

show abstract

Cartman: The Low-Cost Cartesian Manipulator that Won the Amazon Robotics Challenge

Cited by 125 publications

References 12 publications

Segmenting Unknown 3D Objects from Real Depth Images using Mask R-CNN Trained on Synthetic Data

Segmenting Unknown 3D Objects from Real Depth Images using Mask R-CNN Trained on Synthetic Data

Critically fast pick-and-place with suction cups

Towards Robust Product Packing with a Minimalistic End-Effector

Contact Info

Product

Resources

About