From image parsing to painterly rendering

Abstract

We present a semantics-driven approach for stroke-based painterly rendering, based on recent image parsing techniques [Tu et al. 2005; Tu and Zhu 2006] in computer vision. Image parsing integrates segmentation for regions, sketching for curves, and recognition for object categories. In an interactive manner, we decompose an input image into a hierarchy of its constituent components in a parse tree representation with occlusion relations among the nodes in the tree. To paint the image, we build a brush dictionary containing a large set (760) of brush examples of four shape/appearance categories, which are collected from professional artists, then we select appropriate brushes from the dictionary and place them on the canvas guided by the image semantics included in the parse tree, with each image component and layer painted in various styles. During this process, the scene and object categories also determine the color blending and shading strategies for inhomogeneous synthesis of image details. Compared with previous methods, this approach benefits from richer meaningful image semantic information, which leads to better simulation of painting techniques of artists using the high-quality brush dictionary. We have tested our approach on a large number (hundreds) of images and it produced satisfactory painterly effects.

Stroke Based Painterly Rendering

“…Zeng et al [25] presented a region-based painterly rendering algorithm that splits the scene into a tree of regions, each region being classified by its semantic content. This allowed the users to select different styles for human skin, buildings, vegetation or sky.…”

Texture Transfer Based on Energy Minimization for Painterly Rendering

“…A number of filter-based methods have been proposed so far [10], [11], [12], [15], [22]. For example, Haeberli [10] proposed a method which selectively converts a part in a target image into a painterly style until the entire image is converted.…”

“…Hertzmann [11] proposed a method that draws strokes based on an actual drawing manner that first draws a rough sketch with a large brush and then draws details with smaller brushes. Among relatively recent works, Zeng et al [22] proposed a semantics-driven method. This method first parses a target image using methods yokoya@is.naist.jp in Refs.…”

Learnable Stroke Models for Example-based Portrait Painting

“…In recent years, sophisticated painterly rendering algorithms have been proposed that rely increasingly upon Vision to interpret structure and drive the rendering process [4,26]. Although such algorithms generate a pleasing aesthetic for many image classes e.g.…”

“…SBR algorithms create paintings by compositing a sequence of curved spline strokes on a 2D canvas. In contrast to SBR algorithms that encode various rendering heuristics to target a particular artistic style [14,26], our algorithm learns the style of a human artist by example. Given a photograph, and an ordered list of strokes (and related attributes) captured from a training session in which an artist paints that photograph, we are able to learn the artist's style and render previously unseen photographs of faces into portraits with a similar aesthetic.…”