Pixel-Level Domain Transfer

Yoo, Donggeun; Kim, Namil; Park, Sunggyun; Paek, Anthony S.; Kweon, In So

doi:10.1007/978-3-319-46484-8_31

Cited by 255 publications

(197 citation statements)

References 21 publications

Supporting

Mentioning

188

Contrasting

Order By: Relevance

“…Typically used fine-tuning methods require prohibitively large labeled data in the target domain. As an alternative, domain adaptation methods attempt to minimize domain shift either by feature sharing[2] or by learning to reconstruct the target from source domain[3, 4]. In essence, domain adaptation methods learn the marginal distributions [5] to transform source to target domain.…”

Section: Introductionmentioning

confidence: 99%

Tumor-Aware, Adversarial Domain Adaptation from CT to MRI for Lung Cancer Segmentation

Jiang

Tyagi

et al. 2018

Lecture Notes in Computer Science

149

117

View full text Add to dashboard Cite

We present an adversarial domain adaptation based deep learning approach for automatic tumor segmentation from T2-weighted MRI. Our approach is composed of two steps: (i) a tumor-aware unsupervised cross-domain adaptation (CT to MRI), followed by (ii) semi-supervised tumor segmentation using Unet trained with synthesized and limited number of original MRIs. We introduced a novel target specific loss, called tumor-aware loss, for unsupervised cross-domain adaptation that helps to preserve tumors on synthesized MRIs produced from CT images. In comparison, state-of-the art adversarial networks trained without our tumor-aware loss produced MRIs with ill-preserved or missing tumors. All networks were trained using labeled CT images from 377 patients with non-small cell lung cancer obtained from the Cancer Imaging Archive and unlabeled T2w MRIs from a completely unrelated cohort of 6 patients with pre-treatment and 36 on-treatment scans. Next, we combined 6 labeled pre-treatment MRI scans with the synthesized MRIs to boost tumor segmentation accuracy through semi-supervised learning. Semi-supervised training of cycle-GAN produced a segmentation accuracy of 0.66 computed using Dice Score Coefficient (DSC). Our method trained with only synthesized MRIs produced an accuracy of 0.74 while the same method trained in semi-supervised setting produced the best accuracy of 0.80 on test. Our results show that tumor-aware adversarial domain adaptation helps to achieve reasonably accurate cancer segmentation from limited MRI data by leveraging large CT datasets.

show abstract

Section: Introductionmentioning

confidence: 99%

Tumor-Aware, Adversarial Domain Adaptation from CT to MRI for Lung Cancer Segmentation

Jiang

Tyagi

et al. 2018

Lecture Notes in Computer Science

149

117

View full text Add to dashboard Cite

show abstract

“…Yoo te al. [36] generated in shop clothes conditioned on a person in clothing, rather than the reverse.…”

Section: Person Image Generationmentioning

confidence: 99%

Toward Characteristic-Preserving Image-Based Virtual Try-On Network

Wang

Zheng

Liang

et al. 2018

Lecture Notes in Computer Science

291

433

View full text Add to dashboard Cite

Image-based virtual try-on systems for fitting a new in-shop clothes into a person image have attracted increasing research attention, yet is still challenging. A desirable pipeline should not only transform the target clothes into the most fitting shape seamlessly but also preserve well the clothes identity in the generated image, that is, the key characteristics (e.g. texture, logo, embroidery) that depict the original clothes. However, previous image-conditioned generation works fail to meet these critical requirements towards the plausible virtual try-on performance since they fail to handle large spatial misalignment between the input image and target clothes. Prior work explicitly tackled spatial deformation using shape context matching, but failed to preserve clothing details due to its coarse-to-fine strategy. In this work, we propose a new fully-learnable Characteristic-Preserving Virtual Try-On Network (CP-VTON) for addressing all real-world challenges in this task. First, CP-VTON learns a thin-plate spline transformation for transforming the in-shop clothes into fitting the body shape of the target person via a new Geometric Matching Module (GMM) rather than computing correspondences of interest points as prior works did. Second, to alleviate boundary artifacts of warped clothes and make the results more realistic, we employ a Try-On Module that learns a composition mask to integrate the warped clothes and the rendered image to ensure smoothness. Extensive experiments on a fashion dataset demonstrate our CP-VTON achieves the state-ofthe-art virtual try-on performance both qualitatively and quantitatively. Code is available at https://github.com/sergeywong/cp-vton.

show abstract

“…However, performance is limited due to the hand-crafted loss function. A GAN based model can overcome this limitation as it can automatically learn a loss function and has shown promising performance in recent research [35,38,54].…”

Section: Previous Workmentioning

confidence: 99%

Predicting the Future: A Jointly Learnt Model for Action Anticipation

Gammulle

Denman

Sridharan³

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

View full text Add to dashboard Cite

Inspired by human neurological structures for action anticipation, we present an action anticipation model that enables the prediction of plausible future actions by forecasting both the visual and temporal future. In contrast to current state-of-the-art methods which first learn a model to predict future video features and then perform action anticipation using these features, the proposed framework jointly learns to perform the two tasks, future visual and temporal representation synthesis, and early action anticipation. The joint learning framework ensures that the predicted future embeddings are informative to the action anticipation task. Furthermore, through extensive experimental evaluations we demonstrate the utility of using both visual and temporal semantics of the scene, and illustrate how this representation synthesis could be achieved through a recurrent Generative Adversarial Network (GAN) framework. Our model outperforms the current state-of-the-art methods on multiple datasets: UCF101, UCF101-24, UT-Interaction and TV Human Interaction. 1

show abstract

Pixel-Level Domain Transfer

Cited by 255 publications

References 21 publications

Tumor-Aware, Adversarial Domain Adaptation from CT to MRI for Lung Cancer Segmentation

Tumor-Aware, Adversarial Domain Adaptation from CT to MRI for Lung Cancer Segmentation

Toward Characteristic-Preserving Image-Based Virtual Try-On Network

Predicting the Future: A Jointly Learnt Model for Action Anticipation

Contact Info

Product

Resources

About