Semantic Segmentation from Limited Training Data

Milan, Anton; Pham, Trung; Kotra, Vijay; Morrison, Douglas; Tow, Adam W.; Liu, L.; Erskine, J.; Grinover, R.; Gurman, Alan S.; Hunn, T.; Kelly-Boxall, N.; Lee, D.; McTaggart, M.; Rallos, G.; Razjigaev, Andrew; Röggla, Thomas; Shen, Tong; Smith, Robert; Wade-McCue, S.; Zhuang, Zheyu; Lehnert, Christopher; Lin, Guosheng; Reid, Ian; Corke, Peter; Leitner, Jürgen

doi:10.1109/icra.2018.8461082

Cited by 45 publications

(28 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In 2016, many teams used deep learning to segment objects for the alignment phase, training semantic segmentation networks with separate classes for each object instance on hand-labeled [46] or self-supervised datasets [9]. Team ACRV, the winners of the 2017 ARC, fine-tuned RefineNet to segment and classify 40 unique known objects in a bin, with a system to quickly learn new items with a semi-automated procedure [10,47]. In contrast, our method uses deep learning for category-agnostic segmentation, which can can be used to segment a wide variety of objects not seen in training.…”

Section: Related Workmentioning

confidence: 99%

Segmenting Unknown 3D Objects from Real Depth Images using Mask R-CNN Trained on Synthetic Data

Danielczuk

Matl

Gupta

et al. 2019

2019 International Conference on Robotics and Automation (ICRA)

176

108

View full text Add to dashboard Cite

The ability to segment unknown objects in depth images has potential to enhance robot skills in grasping and object tracking. Recent computer vision research has demonstrated that Mask R-CNN can be trained to segment specific categories of objects in RGB images when massive handlabeled datasets are available. As generating these datasets is time-consuming, we instead train with synthetic depth images. Many robots now use depth sensors, and recent results suggest training on synthetic depth data can transfer successfully to the real world. We present a method for automated dataset generation and rapidly generate a synthetic training dataset of 50,000 depth images and 320,000 object masks using simulated heaps of 3D CAD models. We train a variant of Mask R-CNN with domain randomization on the generated dataset to perform category-agnostic instance segmentation without any handlabeled data and we evaluate the trained network, which we refer to as Synthetic Depth (SD) Mask R-CNN, on a set of real, high-resolution depth images of challenging, densely-cluttered bins containing objects with highly-varied geometry. SD Mask R-CNN outperforms point cloud clustering baselines by an absolute 15% in Average Precision and 20% in Average Recall on COCO benchmarks, and achieves performance levels similar to a Mask R-CNN trained on a massive, hand-labeled RGB dataset and fine-tuned on real images from the experimental setup. We deploy the model in an instance-specific grasping pipeline to demonstrate its usefulness in a robotics application. Code, the synthetic training dataset, and supplementary material are available at https://bit.ly/2letCuE. arXiv:1809.05825v2 [cs.CV]

show abstract

Section: Related Workmentioning

confidence: 99%

Segmenting Unknown 3D Objects from Real Depth Images using Mask R-CNN Trained on Synthetic Data

Danielczuk

Matl

Gupta

et al. 2019

2019 International Conference on Robotics and Automation (ICRA)

176

108

View full text Add to dashboard Cite

show abstract

“…After investigating a number of possible approaches, including recent advancements in Deep Metric Learning [12], we found that the best results were obtained by fine-tuning our base RefineNet network on a minimal dataset of the unseen items [13]. Our base RefineNet model was initially trained for 200 epochs on a labelled dataset of approximately 200 images of cluttered scenes containing the 40 known items in the Amazon tote or our storage system.…”

Section: Key System Features a Quick Item Learningmentioning

confidence: 99%

“…An in depth analysis of our RefineNet quick-training approach and a comparison with an alternative Deep Metric Learning approach are provided in [13].…”

Section: Key System Features a Quick Item Learningmentioning

confidence: 99%

Cartman: The Low-Cost Cartesian Manipulator that Won the Amazon Robotics Challenge

Morrison

Tow

McTaggart

et al. 2018

2018 IEEE International Conference on Robotics and Automation (ICRA)

Self Cite

125

View full text Add to dashboard Cite

The Amazon Robotics Challenge enlisted sixteen teams to each design a pick-and-place robot for autonomous warehousing, addressing development in robotic vision and manipulation. This paper presents the design of our custombuilt, cost-effective, Cartesian robot system Cartman, which won first place in the competition finals by stowing 14 (out of 16) and picking all 9 items in 27 minutes, scoring a total of 272 points. We highlight our experience-centred design methodology and key aspects of our system that contributed to our competitiveness. We believe these aspects are crucial to building robust and effective robotic systems.

show abstract

“…As recommended in [33], coefficient β 2 could be selected at 0.3. On other hand, β 2 = 0.25 is also frequently used to evaluate the quality of image processing [34]. However, no strong evidence is demonstrated in the literature to prove the priority of 0.25 or 0.3 among other adjacent values.…”

Section: A Probability Map Thresholding Criteriamentioning

confidence: 99%

Hierarchical Convolutional Neural Network With Feature Preservation and Autotuned Thresholding for Crack Detection

et al. 2021

View full text Add to dashboard Cite

Drone imagery is increasingly used in automated inspection for infrastructure surface defects, especially in hazardous or unreachable environments. In machine vision, the key to crack detection rests with robust and accurate algorithms for image processing. To this end, this paper proposes a deep learning approach using hierarchical convolutional neural networks with feature preservation (HCNNFP) and an intercontrast iterative thresholding algorithm for image binarization. First, a set of branch networks is proposed, wherein the output of previous convolutional blocks is half-sizedly concatenated to the current ones to reduce the obscuration in the down-sampling stage taking into account the overall information loss. Next, to extract the feature map generated from the enhanced HCNN, a binary contrast-based autotuned thresholding (CBAT) approach is developed at the post-processing step, where patterns of interest are clustered within the probability map of the identified features. The proposed technique is then applied to identify surface cracks on the surface of roads, bridges or pavements. An extensive comparison with existing techniques is conducted on various datasets and subject to a number of evaluation criteria including the average F-measure (AF β ) introduced here for dynamic quantification of the performance. Experiments on crack images, including those captured by unmanned aerial vehicles inspecting a monorail bridge. The proposed technique outperforms the existing methods on various tested datasets especially for GAPs dataset with an increase of about 1.4% in terms of AF β while the mean percentage error drops by 2.2%. Such performance demonstrates the merits of the proposed HCNNFP architecture for surface defect inspection.INDEX TERMS deep learning, crack detection, hierarchical convolutional neural network, feature preserving, thresholding.

show abstract

Semantic Segmentation from Limited Training Data

Cited by 45 publications

References 38 publications

Segmenting Unknown 3D Objects from Real Depth Images using Mask R-CNN Trained on Synthetic Data

Segmenting Unknown 3D Objects from Real Depth Images using Mask R-CNN Trained on Synthetic Data

Cartman: The Low-Cost Cartesian Manipulator that Won the Amazon Robotics Challenge

Hierarchical Convolutional Neural Network With Feature Preservation and Autotuned Thresholding for Crack Detection

Contact Info

Product

Resources

About