Efficient Geometry-aware 3D Generative Adversarial Networks

Chan, Eric R.; Lin, Connor Z.; Chan, Matthew A.; Nagano, Koki; Pan, Boxiao; Mello, Shalini De; Gallo, Orazio; Guibas, Leonidas J.; Tremblay, Jonathan; Khamis, Sameh; Karras, Tero; Wetzstein, Gordon

doi:10.1109/cvpr52688.2022.01565

Cited by 537 publications

(435 citation statements)

References 41 publications

Supporting

Mentioning

373

Contrasting

Order By: Relevance

“…An example of synthetic data rendering process is to center a scan (e.g., objects from ScanNet [66], ShapeNet [67], or DeepVoxels [14]) at the origin, scale it to lie within the unit cube, and render images at sampled viewpoints. The training set can be obtained by 2019 2020 2021 2022 2023 S 2 -GAN [12] PrGAN [13] DeepVoxel [14] VON [15] HoloGAN [16] SRN [17] GANSteerability [18] RGBD-GAN [19] DVR [7] BlockGAN [20] GANLatentDiscovery [21] NeRF [5] StyleRig [22] CONFIG [23] GANSpace [24]) InterFaceGAN [25]) NGP [26] SeFa [27] GRAF [28] NeRF-W [29] PIE [30] NeRF++ [31] GIRAFFE [32] pi-GAN [2] PixelNeRF [33] GAN-Control [34] NeRF- [35] KiloNeRF [36] Mip-NeRF [37] FastNeRF [38] CAMPARI [39] BARF [40] VariTex [41] Liao et al [42] ShadeGAN [43] CIPS-3D [44] StyleNeRF [1] GOF [45] LOLNeRF [46] URF [47] EG3D [48] GRAM [49] StyleSDF ...…”

Section: Multiple-view Image Collectionsmentioning

confidence: 99%

“…Shape Quality is evaluated mostly by calculating differences between the rendered depth map and the pseudoground-truth depth, e.g. using MSE [48] or a modified Chamfer distance [50]. For example, given two generated images from two sampled angles of the same scene, Shi et al [52] uses rotation precision and rotation consistency to evaluate the quality of the depth maps (point cloud).…”

Section: Multi-view 3d Consistencymentioning

confidence: 99%

“…Several methods use FID to evaluate image quality at different camera poses as part of the multi-view texture quality. A more direct way is to assess multi-view facial identity consistency (ID) [48] by calculating the mean Arcface [91] cosine similarity score between pairs of the same face rendered from random camera poses.…”

Section: Multi-view 3d Consistencymentioning

confidence: 99%

“…Apart from the aforementioned performance measures, pose accuracy [48] is also considered as an important indicator of shape quality and controllability. Poses (pitch, yaw, and roll) are detected with the help of pre-trained face reconstruction models from the generated images and then its L2 errors against the ground-truth poses is computed to determine each model's pose drift.…”

Section: Multi-view 3d Consistencymentioning

confidence: 99%

“…Due to the prevalence of 2D generative models, there have been studies aiming to make these pretrained models, especially GANs, 3D aware. These works, mostly built on the [48], [50], [51] single simple-shape AFHQ [82] CVPR 2020 Cat, Dog, and Wildlife 15k 512 × 512 [1], [48], [50], [51] single simple-shape CompCars [72] CVPR 2015…”

Section: D Control Of 2d Generative Modelsmentioning

confidence: 99%

See 4 more Smart Citations

A Survey on 3D-aware Image Synthesis

Xia¹,

Xue²

2022

Preprint

View full text Add to dashboard Cite

Recent years have seen remarkable progress in deep learning powered visual content creation. This includes 3D-aware generative image synthesis, which produces high-fidelity images in a 3D-consistent manner while simultaneously capturing compact surfaces of objects from pure image collections without the need for any 3D supervision, thus bridging the gap between 2D imagery and 3D reality. The 3D-aware generative models have shown that the introduction of 3D information can lead to more controllable image generation. The task of 3D-aware image synthesis has taken the field of computer vision by storm, with hundreds of papers accepted to top-tier journals and conferences in recent year (mainly the past two years), but there lacks a comprehensive survey of this remarkable and swift progress. Our survey aims to introduce new researchers to this topic, provide a useful reference for related works, and stimulate future research directions through our discussion section. Apart from the presented papers, we aim to constantly update the latest relevant papers along with corresponding implementations at https://weihaox.github.io/awesome-3D-aware-synthesis.

show abstract

Section: Multiple-view Image Collectionsmentioning

confidence: 99%

Section: Multi-view 3d Consistencymentioning

confidence: 99%

Section: Multi-view 3d Consistencymentioning

confidence: 99%

Section: Multi-view 3d Consistencymentioning

confidence: 99%

Section: D Control Of 2d Generative Modelsmentioning

confidence: 99%

See 3 more Smart Citations

A Survey on 3D-aware Image Synthesis

Xia¹,

Xue²

2022

Preprint

View full text Add to dashboard Cite

show abstract

Understanding the brain with attention: A survey of transformers in brain sciences

Chen,

Wang,

Chen

et al. 2023

Brain-X

View full text Add to dashboard Cite

Owing to their superior capabilities and advanced achievements, Transformers have gradually attracted attention with regard to understanding complex brain processing mechanisms. This study aims to comprehensively review and discuss the applications of Transformers in brain sciences. First, we present a brief introduction of the critical architecture of Transformers. Then, we overview and analyze their most relevant applications in brain sciences, including brain disease diagnosis, brain age prediction, brain anomaly detection, semantic segmentation, multi‐modal registration, functional Magnetic Resonance Imaging (fMRI) modeling, Electroencephalogram (EEG) processing, and multi‐task collaboration. We organize the model details and open sources for reference and replication. In addition, we discuss the quantitative assessments, model complexity, and optimization of Transformers, which are topics of great concern in the field. Finally, we explore possible future challenges and opportunities, exploiting some concrete and recent cases to provoke discussion and innovation. We hope that this review will stimulate interest in further research on Transformers in the context of brain sciences.

show abstract

InfiniteNature-Zero: Learning Perpetual View Generation of Natural Scenes from Single Images

Wang

Snavely

et al. 2022

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Efficient Geometry-aware 3D Generative Adversarial Networks

Cited by 537 publications

References 41 publications

A Survey on 3D-aware Image Synthesis

A Survey on 3D-aware Image Synthesis

Understanding the brain with attention: A survey of transformers in brain sciences

InfiniteNature-Zero: Learning Perpetual View Generation of Natural Scenes from Single Images

Contact Info

Product

Resources

About