BareSkinNet: De‐makeup and De‐lighting via 3D Face Reconstruction

Yang, Xingchao; Taketomi, Takafumi

doi:10.1111/cgf.14706

Cited by 4 publications

(1 citation statement)

References 80 publications

(131 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, it is challenging to collect a large-scale makeup dataset by following the existing methods. Recently, we proposed BareSkinNet [YT22], which can remove makeup and lighting effects from input face images. However, BareSkinNet cannot be used for subsequent makeup applications because it discards the makeup patterns and specular reflection.…”

Section: Facial Makeupmentioning

confidence: 99%

Makeup Extraction of 3D Representation via Illumination‐Aware Image Decomposition

Yang

Taketomi

Kanamori

2023

Computer Graphics Forum

Self Cite

View full text Add to dashboard Cite

Facial makeup enriches the beauty of not only real humans but also virtual characters; therefore, makeup for 3D facial models is highly in demand in productions. However, painting directly on 3D faces and capturing real‐world makeup are costly, and extracting makeup from 2D images often struggles with shading effects and occlusions. This paper presents the first method for extracting makeup for 3D facial models from a single makeup portrait. Our method consists of the following three steps. First, we exploit the strong prior of 3D morphable models via regression‐based inverse rendering to extract coarse materials such as geometry and diffuse/specular albedos that are represented in the UV space. Second, we refine the coarse materials, which may have missing pixels due to occlusions. We apply inpainting and optimization. Finally, we extract the bare skin, makeup, and an alpha matte from the diffuse albedo. Our method offers various applications for not only 3D facial models but also 2D portrait images. The extracted makeup is well‐aligned in the UV space, from which we build a large‐scale makeup dataset and a parametric makeup model for 3D faces. Our disentangled materials also yield robust makeup transfer and illumination‐aware makeup interpolation/removal without a reference image.

show abstract

Section: Facial Makeupmentioning

confidence: 99%