S2F2: Self-Supervised High Fidelity Face Reconstruction from Monocular Image

Dib, Abdallah; Ahn, Jeong Joon; Thébault, Cédric; Gosselin, Philippe-Henri; Chevallier, Louis

doi:10.1109/fg57933.2023.10042713

Cited by 5 publications

(1 citation statement)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There are two types of occlusion: Self-Occlusion (SOP) and External-Occlusion Problem (EOP). SOP means facial organ occluded by itself, features are not fully visible in a 2D image when the head undergoes a significant angular deviation, such as over 90 degrees [10][11][12] . EOP means occluded by other materials not belonging the head, such as the face occluded by hair, glasses, mask or other entities.…”

Section: Introductionmentioning

confidence: 99%

M4Net: Towards Occlusion-Robust and Accurate 3D Dense Face Alignment with 3DMM and Mask Augmentation

ZHAO,

Sulaiman,

LENG

2024

Preprint

View full text Add to dashboard Cite

Self-occlusion and External-occlusion significantly impact the accuracy of monocular 3D dense face alignment, some approaches based on 3D Morphable Models(3DMM) make much progress on it, although 3DMM is a simple and effective facialprior provider, but it can’t solve the problem of robustness caused by the lack of facial occlusion datasets. In this work, wepresent M4Net (Multi-scale features, Multi-head outputs, Mask Augmentation and based on 3D Morphable Model), a 3D denseface alignment method based on 3DMM with multi-scale features and a novel mask-based augmentation strategy, withoutincreasing training cost but significantly improving the model’s robustness on occlude situations. Moreover, the representationof pose, shape, and expression from the encoder has been decoupled, so we can transfer the target expression to the sourceimage by simply swapping the shape and/or expression represent vector between the target and the source. The quantitativeevaluation results demonstrate that our approach achieves SOTA results on AFLW2000-3D(68pts) and AFLW(21pts), and thequalitative results on three different datasets indicating our method has good occlusion robustness.

show abstract

Section: Introductionmentioning

confidence: 99%