Fast 3D Face Reconstruction from a Single Image Using Different Deep Learning Approaches for Facial Palsy Patients

Nguyen, Duc-Phong; Nguyen, Tan-Nhu; Dakpe, Stéphanie; Tho, Marie-Christine Ho Ba Ho Ba; Dao, Tien Tuan

doi:10.3390/bioengineering9110619

Cited by 2 publications

(1 citation statement)

References 69 publications

(75 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The generated dataset by the drone is going to train a PIX2PIX GAN for 3D avatar reconstruction [72]. The GAN would then be trained to learn the relationship between the 2D images and the 3D avatars so that it can generate a 3D avatar that is a reconstruction of the person in a given 2D image.…”

Section: D Avatar Of Human Reconstructionmentioning

confidence: 99%

PIXGAN-Drone: 3D Avatar of Human Body Reconstruction From Multi-View 2D Images

Salim Rasheed,

Jabberi,

Hamdani

et al. 2024

IEEE Access

View full text Add to dashboard Cite

Research on training Generative Adversarial Networks (GANs) to create 3D human body avatars from 2D datasets is underway. Research done in this field shows promise and has paved the way for significant advancements in a variety of applications, including virtual reality, sports analysis, cinematography, surveillance, and cinematography. By avoiding obstacles and producing high-resolution, rich-information multi-view (RGB) images, drone active tracking combined with aerial photography sensors can eliminate occlusions and enable 3D avatar body reconstruction. Due to several issues, such as restricted perspective coverage, obvious occlusions, and texture disappearance, 3D avatar reconstruction techniques encounter training failures that cause distortions and feature loss in 3D reconstructed models. The new endto-end trainable deep neural network methodology PIXGAN-Drone is presented in this paper for the photorealistic 3D avatar of the human body reconstruction from multi-view images. It is based on the integration of active tracking drones equipped with aerial photography sensors (stable automatic circular motion system) into the Pix2Pix GANs training framework to generate high-resolution 2D models. Conditional image-toimage translation and dynamic aerial perspectives can be used to develop realistic and accurate 3D models. This research conducted experiments on multiple datasets to demonstrate the improved performance of our method over state-of-the-art methods for various metrics (Chamfer, P2S, and CED). The results showed that our 3D reconstructed human avatars were 0.0293, 0.0271, and 0.0232 on RenderPeople, 0.0133, 0.0136, 0.0050 on People Snapshot (indoor), 0.0154, 0.0101, 0.0063 on People Snapshot (outdoor), and 0.0316, 0.0275, 0.0216 on Custom data-drone (collected dataset).INDEX TERMS 3D human avatar, 3D reconstruction, PIX2PIXGAN, body model rendering, drone active tracking.

show abstract

Section: D Avatar Of Human Reconstructionmentioning

confidence: 99%

PIXGAN-Drone: 3D Avatar of Human Body Reconstruction From Multi-View 2D Images

Salim Rasheed,

Jabberi,

Hamdani

et al. 2024

IEEE Access

View full text Add to dashboard Cite

show abstract

A survey on deep learning based reenactment methods for deepfake applications

Dhanyalakshmi,

Popirlan,

Hemanth

2024

IET Image Processing

View full text Add to dashboard Cite

Among the sectors that deep learning has transformed, deepfake, a novel method of manipulating multimedia, deserves particular attention. The long‐term objective of many researchers is to seamlessly mimic human facial movement or whole‐body activity, referred to as reenactment. Deepfake progress has made this goal much more feasible in recent years. Yet, achieving more realistic facial and body reenactment remains a challenging task. The primary focus of this study is to explore the current capability of the reenactment techniques and expand them further to attain greater results. The analysis offers a thorough overview of the various techniques involved, the challenges addressed, the datasets utilized, and the metrics employed by the underlying methods of reenactment technologies. The study also addresses the potential risks and their mitigating strategies to ensure responsible reenactment techniques. To the best of the authors' knowledge, this is the first survey paper that delves deeper into the topic of deepfake reenactment.

show abstract

Fast 3D Face Reconstruction from a Single Image Using Different Deep Learning Approaches for Facial Palsy Patients

Cited by 2 publications

References 69 publications

PIXGAN-Drone: 3D Avatar of Human Body Reconstruction From Multi-View 2D Images

PIXGAN-Drone: 3D Avatar of Human Body Reconstruction From Multi-View 2D Images

A survey on deep learning based reenactment methods for deepfake applications

Contact Info

Product

Resources

About