Efficient Interactive Annotation of Segmentation Datasets with Polygon-RNN++

Abstract

Manually labeling datasets with object masks is extremely time consuming. In this work, we follow the idea of Polygon-RNN [4] to produce polygonal annotations of objects interactively using humans-in-the-loop. We introduce several important improvements to the model: 1) we design a new CNN encoder architecture, 2) show how to effectively train the model with Reinforcement Learning, and 3) significantly increase the output resolution using a Graph Neural Network, allowing the model to accurately annotate highresolution objects in images. Extensive evaluation on the Cityscapes dataset [8] shows that our model, which we refer to as Polygon-RNN++, significantly outperforms the original model in both automatic (10% absolute and 16% relative improvement in mean IoU) and interactive modes (requiring 50% fewer clicks by annotators). We further analyze the cross-domain scenario in which our model is trained on one dataset, and used out of the box on datasets from varying domains. The results show that Polygon-RNN++ exhibits powerful generalization capabilities, achieving significant improvements over existing pixel-wise methods. Using simple online fine-tuning we further achieve a high reduction in annotation time for new datasets, moving a step closer towards an interactive annotation tool to be used in practice.

FastDraw: Addressing the Long Tail of Lane Detection by Adapting a Sequential Prediction Network

“…One way to combat this issue is to allow the network to act on its own proposed decoding and use reinforcement learning to train. The authors of [1] take this approach using self-critical sequence training [33] and achieve good results.…”

“…A variety of applications benefit from robust lane detection algorithms that can perform in the wild. If the detector is iterative, the detector can be used as an interactive annotation tool which can be used to decrease the cost of building high definition maps [16,1]. For level 5 systems that depend on high definition maps, online lane de- Figure 2.…”

Computer-aided Assessment of Catheters and Tubes on Radiographs: How Good Is Artificial Intelligence for Assessment?

“…Besides pixel maps, there are other ways of annotation that require less time and effort, such as drawing polygonal bounding boxes. There are some tools designed for this purpose, such as PolygonRNN++ (42). However, because catheters are generally not localized and can span the whole radiographic image, bounding boxes are not as useful as they are in other general computer vision problems.…”

Neural Turtle Graphics for Modeling City Road Layouts

“…Since our approach is local, it is able to grow large graphs which is typically harder to handle with a single RNN. Related to our work, [29,12,27,4] annotate building footprints with a graph generating neural network. However, these works are only able to handle single cycle polygons.…”