Learning frequency‐aware convolutional neural network for spatio‐temporal super‐resolution water surface waves

Abstract

As a usual component in virtual scenes, water surface plays an important role in various graphical applications, including special effects, video games, and virtual reality. Although recent years have witnessed significant progress based on Navier-Stokes equations and simplified water models, large-scale water surface waves with high-frequency visual details remain computationally expensive for interactive applications. This article proposes a novel frequency-aware neural network to synthesize consistent and detailed water surface waves from low-resolution input. At its core, our approach leverage the wavelet transformation theory over space, frequency and direction, and incremental supervision to decompose the 4D amplitude function into multiple smaller subproblems. Specifically, we first customize four subnetworks and corresponding loss functions for super-resolution of spatial resolution, temporal evolution, wave direction subdivision, and wave number, respectively. Then, to enforce the upsampling along each dimension orthogonal to each other, we introduce a cooperative training scheme to fine-tune and integrate the proposed subnetworks with carefully designed training dataset. Our method can visually enhance high-resolution spatial details, temporal coherence, interactions with complex boundaries, and various wave patterns with flexible control along multiple dimensions. Through extensive experiments, our method arrives at 13× speedup for 32× upsampling of various simulation scenarios. We also validate the effectiveness and robustness of our method to produce realistic water surface waves toward artistic innovation.

A novel image restoration solution for cross-resolution person re-identification

“…One line of research focuses on learning resolution invariant features for person representation. For example, Jing et al [29] proposed the SLD 2 L method, which leverages dictionary learning to construct a shared feature space for LR and HR images [30]. This enables the extraction of discriminative features that are invariant to resolution changes.…”