Cut, Paste and Learn: Surprisingly Easy Synthesis for Instance Detection

Abstract

A major impediment in rapidly deploying object detection models for instance detection is the lack of large annotated datasets. For example, finding a large labeled dataset containing instances in a particular kitchen is unlikely. Each new environment with new instances requires expensive data collection and annotation. In this paper, we propose a simple approach to generate large annotated instance datasets with minimal effort. Our key insight is that ensuring only patch-level realism provides enough training signal for current object detector models. We automatically 'cut' object instances and 'paste' them on random backgrounds. A naive way to do this results in pixel artifacts which result in poor performance for trained models. We show how to make detectors ignore these artifacts during training and generate data that gives competitive performance on real data. Our method outperforms existing synthesis approaches and when combined with real images improves relative performance by more than 21% on benchmark datasets. In a cross-domain setting, our synthetic data combined with just 10% real data outperforms models trained on all real data.

Instance Segmentation of Visible and Occluded Regions for Finding and Picking Target from a Pile of Objects

“…A similar problem is also pointed out in previous work [10]: they had objects for which it is difficult to extract foreground mask by thresholding of the depth image, in particular with transparent objects such as Cola Bottle. To overcome this difficulty, they trained a small convolutional neural network (ConvNet) model [21] using the mask acquired from thresholding of the depth image as the ground truth.…”

“…Above work focus on developing ways to make synthetic images closer to real images. On the other hand, it has recently been found that synthesizing only 2D instance images of objects is also effective to train detection models of object bounding boxes [9,10]. The base idea for this is that if we could generate infinite synthetic images at random and train learning model with it, the model would generalize to real images.…”

“…Blending is necessary for 2D image synthesis to remove boundary artifacts when we put the instance images onto the background image. We apply gaussian blurring following [10] with a random sigma which ranges from 0 to 1.…”

“…Data augmentation is crucial for generating synthetic data to train detection models that will generalize to real images, especially when we only have a few instance images (4 − 6), compared to 600 images found in [10]. We applied color data augmentation in addition to the geometric augmentation present in past work [10]. To make the system more robust to changes in brightness and light reflection, we also applied multiplication to S and V channels after converting the RGB image into HSV color space with random selection of scale between 0.5 to 2.0.…”

“…• Image synthesis framework for learning instance segmentation including occlusion, which is a straightforward extension to recent works [9,10]; • A novel instance segmentation network model that uses the instance density to segment multi-class masks by extending recent works [7,8]; • A new metric for instance segmentation of multi-class masks extending recent work of instance segmentation of a single mask [11]; • The integrated system of the above components and demonstration in the picking task from a pile of objects.…”