BranchGAN: Unsupervised Mutual Image-to-Image Transfer With A Single Encoder and Dual Decoders

Zhou, Yifan; Jiang, Runhao; Wu, Xiao; He, Jun-Yan; Weng, Shuang; Peng, Qiang

doi:10.1109/tmm.2019.2920613

Cited by 47 publications

(12 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In this paper, we considerably improve the performance of the attention-based encoder-decoder method on image-based table recognition with a novel EDD architecture. Our model differs from other existing EDD architectures [32], [33], where the dual decoders are independent from each other. In our model, the cell decoder is triggered only when the structure decoder generates a new cell.…”

Section: Related Workmentioning

confidence: 99%

Image-Based Table Recognition: Data, Model, and Evaluation

Zhong

ShafieiBavani

Jimeno-Yepes

2020

Lecture Notes in Computer Science

114

118

View full text Add to dashboard Cite

Important information that relates to a specific topic in a document is often organized in tabular format to assist readers with information retrieval and comparison, which may be difficult to provide in natural language. However, tabular data in unstructured digital documents, e.g. Portable Document Format (PDF) and images, are difficult to parse into structured machine-readable format, due to complexity and diversity in their structure and style. To facilitate image-based table recognition with deep learning, we develop the largest publicly available table recognition dataset PubTabNet 1 , containing 568k table images with corresponding structured HTML representation. PubTabNet is automatically generated by matching the XML and PDF representations of the scientific articles in PubMed Central TM Open Access Subset (PMCOA). We also propose a novel attention-based encoder-dual-decoder (EDD) architecture that converts images of tables into HTML code. The model has a structure decoder which reconstructs the table structure and helps the cell decoder to recognize cell content. In addition, we propose a new Tree-Edit-Distance-based Similarity (TEDS) metric for table recognition. The experiments demonstrate that the EDD model can accurately recognize complex tables solely relying on the image representation, outperforming the state-ofthe-art by 9.7% absolute TEDS score.

show abstract

Section: Related Workmentioning

confidence: 99%

Image-Based Table Recognition: Data, Model, and Evaluation

Zhong

ShafieiBavani

Jimeno-Yepes

2020

Lecture Notes in Computer Science

114

118

View full text Add to dashboard Cite

show abstract

“…Some methods , e.g. Pix2Pix, 36 AutoGAN 37 and others 38,39 have reported remarkably good style transfer results.…”

Section: Deep Learning-based Photorealistic Style Transfer Methodsmentioning

confidence: 99%

UMFA: A photorealistic style transfer method based on U-Net and multi-layer feature aggregation

Rao,

Wu,

et al. 2021

Preprint

View full text Add to dashboard Cite

In this paper, we propose a photorealistic style transfer network to emphasize the natural effect of photorealistic image stylization. In general, distortion of the image content and lacking of details are two typical issues in the style transfer field. To this end, we design a novel framework employing the U-Net structure to maintain the rich spatial clues, with a multi-layer feature aggregation (MFA) method to simultaneously provide the details obtained by the shallow layers in the stylization processing. In particular, an encoder based on the dense block and a decoder form a symmetrical structure of U-Net are jointly staked to realize an effective feature extraction and image reconstruction. Besides, a transfer module based on MFA and "adaptive instance normalization" (AdaIN) is inserted in the skip connection positions to achieve the stylization. Accordingly, the stylized image possesses the texture of a real photo and preserves rich content details without introducing any mask or post-processing steps. The experimental results on public datasets demonstrate that our method achieves a more faithful structural similarity with a lower style loss, reflecting the effectiveness and merit of our approach.

show abstract

“…Unlike DIRT++, GMM-UNIT entailed a proposal of a continuous domain encoding that allowed generating images with zero-or few-shot generation. Zhou et al [91] proposed an unsupervised mutual image-to-image translation model, called BranchGAN, based on a single-encoder-dual-decoder architecture for two domains. BranchGAN transfers one image from one domain to another domain by exploiting the shared distribution of the two domains with the same encoder.…”

Section: Unsupervised Translation With Autoencoder-based Modelsmentioning

confidence: 99%

Deep Generative Adversarial Networks for Image-to-Image Translation: A Review

Alotaibi

2020

Symmetry

View full text Add to dashboard Cite

Many image processing, computer graphics, and computer vision problems can be treated as image-to-image translation tasks. Such translation entails learning to map one visual representation of a given input to another representation. Image-to-image translation with generative adversarial networks (GANs) has been intensively studied and applied to various tasks, such as multimodal image-to-image translation, super-resolution translation, object transfiguration-related translation, etc. However, image-to-image translation techniques suffer from some problems, such as mode collapse, instability, and a lack of diversity. This article provides a comprehensive overview of image-to-image translation based on GAN algorithms and its variants. It also discusses and analyzes current state-of-the-art image-to-image translation techniques that are based on multimodal and multidomain representations. Finally, open issues and future research directions utilizing reinforcement learning and three-dimensional (3D) modal translation are summarized and discussed.

show abstract

BranchGAN: Unsupervised Mutual Image-to-Image Transfer With A Single Encoder and Dual Decoders

Cited by 47 publications

References 26 publications

Image-Based Table Recognition: Data, Model, and Evaluation

Image-Based Table Recognition: Data, Model, and Evaluation

UMFA: A photorealistic style transfer method based on U-Net and multi-layer feature aggregation

Deep Generative Adversarial Networks for Image-to-Image Translation: A Review

Contact Info

Product

Resources

About