Image Inpainting for Irregular Holes Using Partial Convolutions

Liu, Guilin; Reda, Fitsum A.; Shih, Kevin J.; Wang, Ting-Chun; Tao, Andrew; Catanzaro, Bryan

doi:10.1007/978-3-030-01252-6_6

Cited by 1,548 publications

(1,814 citation statements)

References 39 publications

Supporting

Mentioning

1,796

Contrasting

Unclassified

Order By: Relevance

“…Deep learning methods present as a powerful tool to address these limitations; deep learning models have been widely successful for denoising speech 16 , and for inpainting (filling missing data) in images 17,18 . An earlier study 19 demonstrated that simple convolutional neural networks are successful at denoising or peak calling from ChIP-seq data.…”

Section: Discussionmentioning

confidence: 99%

AtacWorks: A deep convolutional neural network toolkit for epigenomics

Chiang

Yakovenko

et al. 2019

Preprint

View full text Add to dashboard Cite

We introduce AtacWorks ( https://github.com/clara-genomics/AtacWorks ), a method to denoise and identify accessible chromatin regions from low-coverage or low-quality ATAC-seq data. AtacWorks uses a deep neural network to learn a mapping between noisy ATAC-seq data and corresponding higher-coverage or higher-quality data. To demonstrate the utility of AtacWorks, we train a model on data from four blood cell types and show that this model accurately denoises and identifies peaks from low-coverage bulk sequencing of different individuals, cell types, and experimental conditions. Further, we show that the deep learning model can be generalized to denoise low-quality data, aggregate single-cell ATAC-seq profiles, and Tn5 insertion sites for transcription factor footprinting. Finally, we apply our deep learning approach to denoise single-cell ATAC-seq data from hematopoietic stem cells to identify differentially-accessible regulatory elements between rare lineage-primed cell subpopulations.

show abstract

Section: Discussionmentioning

confidence: 99%

AtacWorks: A deep convolutional neural network toolkit for epigenomics

Chiang

Yakovenko

et al. 2019

Preprint

View full text Add to dashboard Cite

show abstract

“…We only take the feature map from the first convolutional layer for perceptual loss and style loss, since the network is trained for natural images and the higher-level features are not applicable to our context. The same weighting for loss terms is used as in [7]. The baseline MRI model was trained for 4,000 epochs using a mini-batch of eight 128 × 128 training images.…”

Section: Baseline Mri Modelmentioning

confidence: 99%

“…turned off as suggested in [7]. Common image augmentations, including shifting, left-right flipping, and gray value variations [10], were applied.…”

Section: Baseline Mri Modelmentioning

confidence: 99%

Prostate Cancer Inference via Weakly-Supervised Learning using a Large Collection of Negative MRI

Cao

Zhong

Scalzo

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)

View full text Add to dashboard Cite

Recent advances in medical imaging techniques have led to significant improvements in the management of prostate cancer (PCa). In particular, multi-parametric MRI (mp-MRI) continues to gain clinical acceptance as the preferred imaging technique for non-invasive detection and grading of PCa. However, the machine learning-based diagnosis systems for PCa are often constrained by the limited access to accurate lesion ground truth annotations for training. The performance of the machine learning system is highly dependable on both quality and quantity of lesion annotations associated with histopathologic findings, resulting in limited scalability and clinical validation. Here, we propose the baseline MRI model to alternatively learn the appearance of mp-MRI using radiology-confirmed negative MRI cases via weakly supervised learning. Since PCa lesions are case-specific and highly heterogeneous, it is assumed to be challenging to synthesize PCa lesions using the baseline MRI model, while it would be relatively easier to synthesize the normal appearance in mp-MRI. We then utilize the baseline MRI model to infer the pixel-wise suspiciousness of PCa by comparing the original and synthesized MRI with two distance functions. We trained and validated the baseline MRI model using 1,145 negative prostate mp-MRI scans. For evaluation, we used separated 232 mp-MRI scans, consisting of both positive and negative MRI cases. The 116 positive MRI scans were annotated by radiologists, confirmed with post-surgical whole-gland specimens. The suspiciousness map was evaluated by receiver operating characteristic (ROC) analysis for PCa lesions versus non-PCa regions classification and free-response receiver oper-ating characteristic (FROC) analysis for PCa localization. Our proposed method achieved 0.84 area under the ROC curve and 77.0% sensitivity at one false positive per patient in FROC analysis.

show abstract

“…There is likely room to improve our work here, particularly in exploring further the potential of batch augmentation [5], in developing better saliency-based approaches to occlusion augmentation, and in elucidating further the interaction between and impact of dataset and model complexity for effective occlusion augmentation. Further research could also be done on other kinds of occlusions, such as blur or random noise or even ignoring regions [9]. In conclusion, in contrast to other regularization techniques that require architectural changes, we present a simple paradigm for making occlusions effective on ImageNet for sufficiently capable models (e.g., ResNet50) that can be easily added into existing training paradigms.…”

Section: Resultsmentioning

confidence: 99%

Occlusions for Effective Data Augmentation in Image Classification

Fong

Vedaldi

2019

2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)

View full text Add to dashboard Cite

Deep networks for visual recognition are known to leverage "easy to recognise" portions of objects such as faces and distinctive texture patterns. The lack of a holistic understanding of objects may increase fragility and overfitting. In recent years, several papers have proposed to address this issue by means of occlusions as a form of data augmentation. However, successes have been limited to tasks such as weak localization and model interpretation, but no benefit was demonstrated on image classification on large-scale datasets. In this paper, we show that, by using a simple technique based on batch augmentation, occlusions as data augmentation can result in better performance on ImageNet for high-capacity models (e.g., ResNet50). We also show that varying amounts of occlusions used during training can be used to study the robustness of different neural network architectures.

show abstract

Image Inpainting for Irregular Holes Using Partial Convolutions

Cited by 1,548 publications

References 39 publications

AtacWorks: A deep convolutional neural network toolkit for epigenomics

AtacWorks: A deep convolutional neural network toolkit for epigenomics

Prostate Cancer Inference via Weakly-Supervised Learning using a Large Collection of Negative MRI

Occlusions for Effective Data Augmentation in Image Classification

Contact Info

Product

Resources

About