Faceshop

Abstract

We present a novel system for sketch-based face image editing, enabling users to edit images intuitively by sketching a few strokes on a region of interest. Our interface features tools to express a desired image manipulation by providing both geometry and color constraints as user-drawn strokes. As an alternative to the direct user input, our proposed system naturally supports a copy-paste mode, which allows users to edit a given image region by using parts of another exemplar image without the need of hand-drawn sketching at all. The proposed interface runs in real-time and facilitates an interactive and iterative workflow to quickly express the intended edits. Our system is based on a novel sketch domain and a convolutional neural network trained end-to-end to automatically learn to render image regions corresponding to the input strokes. To achieve high quality and semantically consistent results we train our neural network on two simultaneous tasks, namely image completion and image translation. To the best of our knowledge, we are the first to combine these two tasks in a unified framework for interactive image editing. Our results show that the proposed sketch domain, network architecture, and training procedure generalize well to real user input and enable high quality synthesis results without additional post-processing.

LC-NeRF: Local Controllable Face Generation in Neural Randiance Field

“…With the high-quality generation of images, many portraits are generated by the generation models, as described above. Meanwhile, some methods [19] take the editing as an application. Editing tasks are no longer limited to the 2D domain, and research on how to perform editing and control on 3D faces becomes pupolar.…”

“…Editing tasks are no longer limited to the 2D domain, and research on how to perform editing and control on 3D faces becomes pupolar. In the image domain, Faceshop [19] treats the face editing task as sketch-based image completion that can only edit the facial geometry. The demands for face editing are no longer editing geometry but also modifying texture, such as editing hair colors.…”

Barbershop: GAN-based Image Compositing using Segmentation Masks

“…Particularly on faces, StarGANs1&2 [Choi et al 2018[Choi et al , 2020 are able to modify multiple attributes. Other notable works, FaceShop [Portenier et al 2018], Deep plastic surgery , Interactive hair and beard synthesis [Olszewski et al 2020] and SC-FEGAN [Jo and Park 2019] can modify the images using the strokes or scribbles on the semantic regions. For the hairstyle and appearance editing, we identified two notable relevant works.…”

Modeling in the Time of COVID-19: Statistical and Rule-based Mesoscale Models

“…Here the user only specifies a 2D contour of an object and a 3D geometry is generated using the contour inflation approach [76], which we also utilize. An interesting recent trend is to control a deep learning model using sketches, e.g., for modeling terrains [23], faces [58], or buildings [52]. Utilization in the sciences can be exemplified through modeling advanced geological concepts and phenomena [41,48] or for creating quick molecular landscapes for communicating to peers or a broader audience.…”