Semantic Segmentation for Line Drawing Vectorization Using Neural Networks

Abstract

In this work, we present a method to vectorize raster images of line art. Inverting the rasterization procedure is inherently ill‐conditioned, as there exist many possible vector images that could yield the same raster image. However, not all of these vector images are equally useful to the user, especially if performing further edits is desired. We therefore define the problem of computing an instance segmentation of the most likely set of paths that could have created the raster image. Once the segmentation is computed, we use existing vectorization approaches to vectorize each path, and then combine all paths into the final output vector image. To determine which set of paths is most likely, we train a pair of neural networks to provide semantic clues that help resolve ambiguities at intersection and overlap regions. These predictions are made considering the full context of the image, and are then globally combined by solving a Markov Random Field (MRF). We demonstrate the flexibility of our method by generating results on character datasets, a synthetic random line dataset, and a dataset composed of human drawn sketches. For all cases, our system accurately recovers paths that adhere to the semantics of the drawings.

A benchmark for rough sketch cleanup

“…Numerous later works have been proposed following this same basic cluster-and-replace framework for vector graphics [Liu et al 2018[Liu et al , 2015Ogawa et al 2016;Orbay and Kara 2011;Shesh and Chen 2008]. Another line of work simultaneously vectorizes and simplifies a raster image of a sketch [Bessmeltsev and Solomon 2019;Donati et al 2019;Favreau et al 2016;Kim et al 2018;Noris et al 2013;Parakkat et al 2018]. This is a more challenging problem, as parametric data is unavailable for the input curves.…”

Deep Line Drawing Vectorization via Line Subdivision and Topology Reconstruction

“…Favreau's method [FLB16] even needs several minutes to vectorize a single 1024 × 1024 image. In the segmentation step, Kim's method [KWÖG18] takes around 5 minutes for an 128 × 128 image, and it fails to process high‐resolution images.…”

Integer‐Grid Sketch Simplification and Vectorization

“…Kim et al [KWÖG18] propose a deep-learning algorithm to segment a line drawing into individual curves, effectively bypassing Starting with a bitmap line drawing, we first compute a frame field to recover the two local dominant directions at each pixel (a). We use this field to guide a grid-based parametrization whose isolines snap to strokes, and nodes snap to junctions (b).…”