Multitask learning on monocular water images: Surface reconstruction and image synthesis

Abstract

In this paper, we present a new strategy, a joint deep learning architecture, for two classic tasks in computer graphics: water surface reconstruction and water image synthesis. Modeling water surfaces from single images can be regarded as the inverse of image rendering, which converts surface geometries into photorealistic images. On the basis of this fact, we therefore consider these two problems as a cycle image‐to‐image translation and propose to tackle them together using a pair of neural networks, with the three‐dimensional surface geometries being represented as two‐dimensional surface normal maps. Furthermore, we also estimate the imaging parameters from the existing water images with a subnetwork to reuse the lighting conditions when synthesizing new images. Experiments demonstrate that our method achieves an accurate reconstruction of surfaces from monocular images efficiently and produces visually plausible new images under variable lighting conditions.

Learning frequency‐aware convolutional neural network for spatio‐temporal super‐resolution water surface waves

“…Deep learning in animations. Deep learning technologies have also been successfully applied in computer animations, benefiting from the powerful mining and fitting ability for the relationships between spatial and temporal changes, including character motion, [30][31][32] elasticity deformation, 33 latent-space physics, 34,35 cloth, 36 fluid acceleration 37 and detail enhancement, 38 limited view reconstruction, 39 water surface reconstruction, 40 and so forth.…”