Context-Aware Content Generation for Virtual Environments

Brock, Andrew; Lim, Theodore; Ritchie, James Millar; Weston, Nick

doi:10.1115/detc2016-59997

Cited by 25 publications

(28 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While the G network is trained to produce realistic images G ( z ) from a random vector z , the D network is trained to discriminate between real and generated images [32], [33]. GANs have been used in many applications such as single image super-resolution [29], art creation [34], [35], and image transformation [36]. In the field of medical imaging, Nie et al [37] proposed to use GAN to estimate CT image from its corresponding MR image.…”

Section: Introductionmentioning

confidence: 99%

Low-Dose CT Image Denoising Using a Generative Adversarial Network With Wasserstein Distance and Perceptual Loss

Yang¹,

Yan²,

Zhang³

et al. 2018

IEEE Trans. Med. Imaging

1,245

890

View full text Add to dashboard Cite

The continuous development and extensive use of computed tomography (CT) in medical practice has raised a public concern over the associated radiation dose to the patient. Reducing the radiation dose may lead to increased noise and artifacts, which can adversely affect the radiologists' judgment and confidence. Hence, advanced image reconstruction from low-dose CT data is needed to improve the diagnostic performance, which is a challenging problem due to its ill-posed nature. Over the past years, various low-dose CT methods have produced impressive results. However, most of the algorithms developed for this application, including the recently popularized deep learning techniques, aim for minimizing the mean-squared error (MSE) between a denoised CT image and the ground truth under generic penalties. Although the peak signal-to-noise ratio is improved, MSE- or weighted-MSE-based methods can compromise the visibility of important structural details after aggressive denoising. This paper introduces a new CT image denoising method based on the generative adversarial network (GAN) with Wasserstein distance and perceptual similarity. The Wasserstein distance is a key concept of the optimal transport theory and promises to improve the performance of GAN. The perceptual loss suppresses noise by comparing the perceptual features of a denoised output against those of the ground truth in an established feature space, while the GAN focuses more on migrating the data noise distribution from strong to weak statistically. Therefore, our proposed method transfers our knowledge of visual perception to the image denoising task and is capable of not only reducing the image noise level but also trying to keep the critical information at the same time. Promising results have been obtained in our experiments with clinical CT images.

show abstract

Section: Introductionmentioning

confidence: 99%

Low-Dose CT Image Denoising Using a Generative Adversarial Network With Wasserstein Distance and Perceptual Loss

Yang¹,

Yan²,

Zhang³

et al. 2018

IEEE Trans. Med. Imaging

1,245

890

View full text Add to dashboard Cite

show abstract

“…Despite the prominence of 3D Object Generation papers in the computer graphics community, a few have arisen within the engineering design community as well. In an early work, for example, Brock et al [87], implement a 3-Dimensional cVAE to reconstruct and interpolate between voxelized models from the Modelnet-10 Dataset.…”

Section: D Shape Synthesismentioning

confidence: 99%

Deep Generative Models in Engineering Design: A Review

Regenwetter¹,

Nobari²,

Ahmed³

2021

Preprint

View full text Add to dashboard Cite

Automated design synthesis has the potential to revolutionize the modern human design process and improve access to highly optimized and customized products across countless industries. Successfully adapting generative Machine Learning to design engineering may be the key to such automated design synthesis and is a research subject of great importance. We present a review and analysis of Deep Generative Learning models in engineering design. Deep Generative Models (DGMs) typically leverage deep networks to learn from an input dataset and learn to synthesize new designs. Recently, DGMs such as Generative Adversarial Networks (GANs), Variational Autoencoders (VAEs), feedforward Neural Networks (NNs) and certain Deep Reinforcement Learning (DRL) frameworks have shown promising results in design applications like structural optimization, materials design, and shape synthesis. The prevalence of DGMs in Engineering Design has skyrocketed since 2016. Anticipating continued growth, we conduct a review of recent advances with the hope of benefitting researchers interested in DGMs for design. We structure our review as an exposition of the algorithms, datasets, representation methods, and applications commonly used in the current literature. In particular, we discuss key works that have introduced new techniques and methods in DGMs, successfully applied DGMs to a design-related domain, or directly supported development of DGMs through datasets or auxiliary methods. We further identify key challenges and limitations currently seen in DGMs across design fields, such as design creativity, handling complex constraints and objectives, and modeling both form and functional performance simultaneously. In our discussion we identify possible solution pathways as key areas on which to target future work.

show abstract

“…Transformations can then be expressed as vector operations over embeddings [66,82] or manifold traversals [27,65]. Alternative approaches rely on training conditional generative models [2,11,13,37,97] that learn a mapping between two or more image distributions. Much of this prior work is motivated by the need for sophisticated tools for image editing [42,82] e.g.…”

Section: B2 Augmentation Primitives and Pipelinesmentioning

confidence: 99%

“…for any sample x and where [x] denotes the generated coupled set {F1(x), F2(x)} as usual. Denoting the right hand side Ls(x; θ) for shorthand, summing equations ( 10) and (11), and using the metric property of the JSD (Proposition 1) gives…”

Section: Augmentedmentioning

confidence: 99%

Model Patching: Closing the Subgroup Performance Gap with Data Augmentation

Goel,

Gu,

et al. 2020

Preprint

View full text Add to dashboard Cite

Classifiers in machine learning are often brittle when deployed. Particularly concerning are models with inconsistent performance on specific subgroups of a class, e.g., exhibiting disparities in skin cancer classification in the presence or absence of a spurious bandage. To mitigate these performance differences, we introduce model patching, a two-stage framework for improving robustness that encourages the model to be invariant to subgroup differences, and focus on class information shared by subgroups. Model patching first models subgroup features within a class and learns semantic transformations between them, and then trains a classifier with data augmentations that deliberately manipulate subgroup features. We instantiate model patching with CAMEL, which (1) uses a CycleGAN to learn the intra-class, intersubgroup augmentations, and (2) balances subgroup performance using a theoretically-motivated subgroup consistency regularizer, accompanied by a new robust objective. We demonstrate CAMEL's effectiveness on 3 benchmark datasets, with reductions in robust error of up to 33% relative to the best baseline. Lastly, CAMEL successfully patches a model that fails due to spurious features on a real-world skin cancer dataset.

show abstract

Context-Aware Content Generation for Virtual Environments

Cited by 25 publications

References 7 publications

Low-Dose CT Image Denoising Using a Generative Adversarial Network With Wasserstein Distance and Perceptual Loss

Low-Dose CT Image Denoising Using a Generative Adversarial Network With Wasserstein Distance and Perceptual Loss

Deep Generative Models in Engineering Design: A Review

Model Patching: Closing the Subgroup Performance Gap with Data Augmentation

Contact Info

Product

Resources

About