Facial-sketch Synthesis: A New Challenge

Abstract

This paper aims to conduct a comprehensive study on facial-sketch synthesis (FSS). However, due to the high cost of obtaining hand-drawn sketch datasets, there is a lack of a complete benchmark for assessing the development of FSS algorithms over the last decade. We first introduce a high-quality dataset for FSS, named FS2K, which consists of 2 104 image-sketch pairs spanning three types of sketch styles, image backgrounds, lighting conditions, skin colors, and facial attributes. FS2K differs from previous FSS datasets in difficulty, diversity, and scalability and should thus facilitate the progress of FSS research. Second, we present the largest-scale FSS investigation by reviewing 89 classic methods, including 25 handcrafted feature-based facial-sketch synthesis approaches, 29 general translation methods, and 35 image-to-sketch approaches. In addition, we elaborate comprehensive experiments on the existing 19 cutting-edge models. Third, we present a simple baseline for FSS, named FSGAN. With only two straightforward components, i.e., facial-aware masking and style-vector expansion, our FSGAN surpasses the performance of all previous state-of-the-art models on the proposed FS2K dataset by a large margin. Finally, we conclude with lessons learned over the past years and point out several unsolved challenges. Our code is available at https://github.com/DengPingFan/FSGAN.

Stylized Face Sketch Extraction via Generative Prior with Limited Data

“…Different from the aforementioned datasets [WT08, ZWT11, YLLR19], Disney portrait dataset [BSM*13] and UPDG do not provide corresponding photo images, making it challenging to compare the images generated from image‐to‐image translation models with their corresponding ground truths. The most recent and largest FS2K dataset [FHZ*22] features 2,104 pairs of photos and sketches in three different styles created by different artists. The sketches underwent slight changes for artistic expression.…”

“…There exist several face image datasets that include the corresponding sketch drawings. We briefly describe the details of these datasets based on the descriptions of the FS2K paper [FHZ*22]. See Table 1 for the summarized details.…”