COCO-GAN: Generation by Parts via Conditional Coordinating

Abstract

Humans can only interact with part of the surrounding environment due to biological restrictions. Therefore, we learn to reason the spatial relationships across a series of observations to piece together the surrounding environment. Inspired by such behavior and the fact that machines also have computational constraints, we propose COnditional COordinate GAN (COCO-GAN) of which the generator generates images by parts based on their spatial coordinates as the condition. On the other hand, the discriminator learns to justify realism across multiple assembled patches by global coherence, local appearance, and edge-crossing continuity. Despite the full images are never generated during training, we show that COCO-GAN can produce state-of-the-art-quality full images during inference. We further demonstrate a variety of novel applications enabled by teaching the network to be aware of coordinates. First, we perform extrapolation to the learned coordinate manifold and generate off-the-boundary patches. Combining with the originally generated full image, COCO-GAN can produce images that are larger than training samples, which we called "beyond-boundary generation". We then showcase panorama generation within a cylindrical coordinate system that inherently preserves horizontally cyclic topology. On the computation side, COCO-GAN has a built-in divide-and-conquer paradigm that reduces memory requisition during training and inference, provides highparallelism, and can generate parts of images on-demand.

A lightweight GAN network for large scale fingerprint generation

“…AE-GAN [2] combines WGAN and VAE to create stable, high-resolution photos. COCO-GAN [37] generates state-of-the-art images by utilizing spatial information as the constraint for the generator. The semantic bottleneck network combines progressive semantic generation network and segmentation-to-image synthesis network to produce 5k images.…”

A State-of-the-Art Review on Image Synthesis With Generative Adversarial Networks

“…Other directions that are equally noteworthy, such as in the modular [138] and game areas [139], have rarely been studied. Recently, Lin et al [140] proposed a novel method called Conditional Coordinate GAN (COCO-GAN), which uses the spatial coordinates as the condition to generate images by parts, and it achieves a high generation quality. Particularly, this approach can generate images larger than any training sample and can be used for large field-of-view generation.…”

Data Enhancement for Plant Disease Classification Using Generated Lesions

“…Tran et al [46] applied self-supervised learning via the geometric transformation on input images and assigned the pseudo-labels to these transformed images to improve an unconditional GAN. Lin et al [47] produced images that were larger than training samples by combining them with the originally generated full image. Takano et al [48] explored how selecting a dataset affects the outcome by using three different datasets to see that a Super-Resolution GAN (SRGAN) fundamentally learns objects, and using their shape, color, and texture, redraws them in the output rather than merely attempting to sharpen edges.…”