SSD-GAN: Measuring the Realness in the Spatial and Spectral Domains

Chen, Yuanqi; Li, Ge; Jin, Cece; Liu, Shan; Li, Thomas

doi:10.1609/aaai.v35i2.16196

Cited by 19 publications

(7 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As proved in the observations [28,30,35] that down-sampling techniques would lead to high frequencies missed in the discriminator. Meanwhile, the generator, which is implicated in the lacking of incentives from the discriminator, fails to capture high frequencies of clear underwater images y.…”

Section: Frequency-aware Constraintmentioning

confidence: 92%

See 1 more Smart Citation

SFA-GAN: Structure-frequency-aware generative adversarial network for underwater image enhancement

Zhang

Liu

Zhao

et al. 2023

Preprint

View full text Add to dashboard Cite

Influenced by light scattering, absorption and water impurities, the quality of underwater image is so poor that it poses a great challenge to underwater target detection, marine biological research and marine exploration. Thus, significant attention on underwater image enhancement (UIE) has been attracted for producing high quality visuality as if the underwater image was taken in-air without any structure, texture and color loss. To solve this issue, previous work mainly focus on supervised-learning with large amount of paired data, which is more demanding in practical application. Recent Cycle-GAN based UIE break through the dependence on paired data but easily trap in mapping ambiguity. Essentially, two-sided cycle-consistency is a bijection and only focuses on the pixel level, which is too restrictive and can not accurately express underwater scene structure. Besides, high frequencies in reference images tend to be eschewed by generator, making it difficult to synthesize authentic textures and colors of underwater images. We therefore propose a novel unconstrained UIE framework, structure-frequency-aware generative adversarial network (SFA-GAN), which not only accurately preserves the structure of low quality underwater images, but also captures the high frequencies of the reference images under unconstrained settings. Extensive experiments on datasets EUVP, UFO-120 and UIEB demonstrate that the proposed SFA-GAN can achieves state-of-art results on some metrics and produce more clear underwater images without sacrificing model complexity.

show abstract

Section: Frequency-aware Constraintmentioning

confidence: 92%

“…As demonstrated in [28,30,36], a 1D representation of the Fourier power spectrum from polar coordinate of ℱ(m, n) by azimuthally integrating over polar angle θ, can be used to highlight spectral differences without significant loss in information.…”

Section: Frequency-aware Constraintmentioning

confidence: 99%

SFA-GAN: Structure-frequency-aware generative adversarial network for underwater image enhancement

Zhang

Liu

Zhao

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…In recent research, frequency domain analysis for deep neural networks, especially GANs, has been proved an effective tool for network diagnose [9,27], GAN generated image detection [10,12,43], and improving the capability of GANs for image generation [6,33]. For example, Rahaman et al [27] used Fourier analysis to highlight the bias of deep networks towards low-frequency functions.…”

Section: Frequency Domain Analysis For Gansmentioning

confidence: 99%

“…Dzanic et al [10] explored the Fourier spectrum discrepancies of high-frequency components between natural and generated images for fake image detection. Chen et al [6] observed missing of high-frequency information in discriminators of GANs. Nevertheless, there is no prior work to study whether the temporal inconsistency of synthetic videos is caused by the frequency discrepancies between natural videos and synthetic videos.…”

Section: Frequency Domain Analysis For Gansmentioning

confidence: 99%

Delving into the Frequency: Temporally Consistent Human Motion Transfer in the Fourier Space

Yang

Liu

et al. 2022

Proceedings of the 30th ACM International Conference on Multimedia

View full text Add to dashboard Cite

Human motion transfer refers to synthesizing photo-realistic and temporally coherent videos that enable one person to imitate the motion of others. However, current synthetic videos suffer from the temporal inconsistency in sequential frames that significantly degrades the video quality, yet is far from solved by existing methods in the pixel domain. Recently, some works on DeepFake detection try to distinguish the natural and synthetic images in the frequency domain because of the frequency insufficiency of image synthesizing methods. Nonetheless, there is no work to study the temporal inconsistency of synthetic videos from the aspects of the frequencydomain gap between natural and synthetic videos. Therefore, in this paper, we propose to delve into the frequency space for temporally consistent human motion transfer. First of all, we make the first comprehensive analysis of natural and synthetic videos in the frequency domain to reveal the frequency gap in both the spatial dimension of individual frames and the temporal dimension of the video. To close the frequency gap between the natural and synthetic videos, we propose a novel Frequency-based human MOtion TRansfer framework, named FreMOTR, which can effectively mitigate the spatial artifacts and the temporal inconsistency of the synthesized videos. FreMOTR explores two novel frequency-based regularization modules: 1) the Frequency-domain Appearance Regularization (FAR) to improve the appearance of the person in individual frames and 2) Temporal Frequency Regularization (TFR) to guarantee the temporal consistency between adjacent frames. Finally, comprehensive experiments demonstrate that the FreMOTR not only yields superior performance in temporal consistency metrics but also improves the frame-level visual quality *This work was done when Guang Yang was an intern at JD Explore Academy. †Wu Liu and Juan Cao are corresponding authors.

show abstract

“…As these regions belong to high-frequency information, the generator fails to produce high-frequency details. Previous researches [46][47][48] have shown that high-frequency information has a certain influence on the training of GANs. Thus we introduce a Gaussian high-pass filter in the discriminator (Figure 4) to guide the discriminator to process high-frequency features and low-frequency features separately.…”

Section: High-frequency Separationmentioning

confidence: 99%

GSGAN: Learning controllable geospatial images generation

Xingzhe

Lin

Zheng

et al. 2022

IET Image Processing

View full text Add to dashboard Cite

Compared with natural images, geospatial images cover larger area and have more complex image contents. There are few algorithms for generating controllable geospatial images, and their results are of low quality. In response to this problem, this paper proposes Geospatial Style Generative Adversarial Network to generate controllable and high‐quality geospatial images. Current conditional generators suffer the mode collapse problem in geospatial field. The problem is addressed via a modified mode seeking regularization term with contrastive learning theory. Besides, the discriminator network architecture is modified to process global feature information and texture information of geospatial images. Feature loss in the generator is introduced to stabilize the training process and improve generated image quality. Comprehensive experiments are conducted on UC Merced Land Use Dataset, NWPU‐RESISC45 Dataset, and AID Dataset to evaluate all compared methods. Experiment results show our method outperforms state‐of‐the‐art models. Our method not only generates high‐quality and controllable geospatial images, but also enhances the discriminator to learn better representations.

show abstract

SSD-GAN: Measuring the Realness in the Spatial and Spectral Domains

Cited by 19 publications

References 20 publications

SFA-GAN: Structure-frequency-aware generative adversarial network for underwater image enhancement

SFA-GAN: Structure-frequency-aware generative adversarial network for underwater image enhancement

Delving into the Frequency: Temporally Consistent Human Motion Transfer in the Fourier Space

GSGAN: Learning controllable geospatial images generation

Contact Info

Product

Resources

About