Semantic Image Inpainting with Deep Generative Models

Yeh, Raymond A.; Chen, Chen; Lim, Teck Yian; Schwing, Alexander G.; Hasegawa‐Johnson, Mark; Minh, N.

doi:10.48550/arxiv.1607.07539

Cited by 105 publications

(194 citation statements)

References 0 publications

Supporting

Mentioning

194

Contrasting

Order By: Relevance

“…However, this is already feasible for many types of distributions. For example, image inpainting is a subfield of computer vision that has a long history [13] and much recent work (e.g., [32,31,30,28,23,33]), with plausible in-fill models available for many domains. As these generative models improve, so will the framework we proposed.…”

Section: Discussionmentioning

confidence: 99%

Interpreting Black Box Models via Hypothesis Testing

Burns,

Thomason,

Tansey

2019

Preprint

View full text Add to dashboard Cite

While many methods for interpreting machine learning models have been proposed, they are often ad hoc, difficult to interpret, and come with limited guarantees. This is especially problematic in science and medicine, where model interpretations may be reported as discoveries or guide patient treatments. As a step toward more principled and reliable interpretations, in this paper we reframe black box model interpretability as a multiple hypothesis testing problem. The task is to discover "important" features by testing whether the model prediction is significantly different from what would be expected if the features were replaced with uninformative counterfactuals. We propose two testing methods: one that provably controls the false discovery rate but which is not yet feasible for large-scale applications, and an approximate testing method which can be applied to real-world data sets. In simulation, both tests have high power relative to existing interpretability methods. When applied to state-of-the-art vision and language models, the framework selects features that intuitively explain model predictions. The resulting explanations have the additional advantage that they are themselves easy to interpret.Preprint. Under review.

show abstract

Section: Discussionmentioning

confidence: 99%

Interpreting Black Box Models via Hypothesis Testing

Burns,

Thomason,

Tansey

2019

Preprint

View full text Add to dashboard Cite

show abstract

“…The PICS method was implemented using the BART Toolbox 32 with wavelets as the sparse transform. In order to further demonstrate the performance of our UFLoss, MoDL with 2 + perceptual VGG loss 37 was also included in our comparisons.…”

Section: Unrolled Reconstructions With Uflossmentioning

confidence: 99%

High Fidelity Deep Learning-based MRI Reconstruction with Instance-wise Discriminative Feature Matching Loss

Wang¹,

Tamir²,

Goyeneche³

et al. 2021

Preprint

View full text Add to dashboard Cite

Purpose: To improve reconstruction fidelity of fine structures and textures in deep learning (DL) based reconstructions. Methods: A novel patch-based Unsupervised Feature Loss (UFLoss) is proposed and incorporated into the training of DL-based reconstruction frameworks in order to preserve perceptual similarity and high-order statistics. The UFLoss provides instance-level discrimination by mapping similar instances to similar low-dimensional feature vectors and is trained without any human annotation. By adding an additional loss function on the low-dimensional feature space during training, the reconstruction frameworks from under-sampled or corrupted data can reproduce more realistic images that are closer to the original with finer textures, sharper edges, and improved overall image quality. The performance of the proposed UFLoss is demonstrated on unrolled networks for accelerated 2D and 3D knee MRI reconstruction with retrospective under-sampling. Quantitative metrics including NRMSE, SSIM, and our proposed UFLoss were used to evaluate the performance of the proposed method and compare it with others. Results: In-vivo experiments indicate that adding the UFLoss encourages sharper edges and more faithful contrasts compared to traditional and learning-based methods with pure 2 loss. More detailed textures can be seen in both 2D and 3D knee MR images. Quantitative results indicate that reconstruction with UFLoss can provide comparable NRMSE and a higher SSIM while achieving a much lower UFLoss value. Conclusion: We present UFLoss, a patch-based unsupervised learned feature loss, which allows the training of DL-based reconstruction to obtain more detailed texture, finer features, and sharper edges with higher overall image quality under DL-based reconstruction frameworks.1

show abstract

“…Since its introduction, GANs have enjoyed great empirical success, with a wide range of applications especially in image generation and natural language processing, including high resolution image generation [Denton et al, 2015, Radford et al, 2015, image inpainting [Yeh et al, 2016], image super-resolution [Ledig et al, 2017], visual manipulation [Zhu et al, 2016], text-to-image synthesis [Reed et al, 2016], video generation [Vondrick et al, 2016], semantic segmentation [Luc et al, 2016], and abstract reasoning diagram generation [Kulharia et al, 2017].…”

Section: Introductionmentioning

confidence: 99%

Generative Adversarial Network: Some Analytical Perspectives

Cao¹,

Guo²

2021

Preprint

View full text Add to dashboard Cite

Ever since its debut, generative adversarial networks (GANs) have attracted tremendous amount of attention. Over the past years, different variations of GANs models have been developed and tailored to different applications in practice. Meanwhile, some issues regarding the performance and training of GANs have been noticed and investigated from various theoretical perspectives. This subchapter will start from an introduction of GANs from an analytical perspective, then move on the training of GANs via SDE approximations and finally discuss some applications of GANs in computing high dimensional MFGs as well as tackling mathematical finance problems.

show abstract

Semantic Image Inpainting with Deep Generative Models

Cited by 105 publications

References 0 publications

Interpreting Black Box Models via Hypothesis Testing

Interpreting Black Box Models via Hypothesis Testing

High Fidelity Deep Learning-based MRI Reconstruction with Instance-wise Discriminative Feature Matching Loss

Generative Adversarial Network: Some Analytical Perspectives

Contact Info

Product

Resources

About