Variational Models for Joint Subsampling and Reconstruction of Turbulence-Degraded Images

Abstract

Turbulence-degraded image frames are distorted by both turbulent deformations and space-timevarying blurs. To suppress these effects, we propose a multi-frame reconstruction scheme to recover a latent image from the observed image sequence. Recent approaches are commonly based on registering each frame to a reference image, by which geometric turbulent deformations can be estimated and a sharp image can be restored. A major challenge is that a fine reference image is usually unavailable, as every turbulence-degraded frame is distorted. A high-quality reference image is crucial for the accurate estimation of geometric deformations and fusion of frames. Besides, it is unlikely that all frames from the image sequence are useful, and thus frame selection is necessary and highly beneficial. In this work, we propose a variational model for joint subsampling of frames and extraction of a clear image. A fine image and a suitable choice of subsample are simultaneously obtained by iteratively reducing an energy functional. The energy consists of a fidelity term measuring the discrepancy between the extracted image and the subsampled frames, as well as regularization terms on the extracted image and the subsample. Different choices of fidelity and regularization terms are explored. By carefully selecting suitable frames and extracting the image, the quality of the reconstructed image can be significantly improved. Extensive experiments have been carried out, which demonstrate the efficacy of our proposed model. In addition, the extracted subsamples and images can be put in existing algorithms to produce improved results.

Subsampled turbulence removal network

“…Another recent approach is the joint subsampling and reconstruction variational model proposed by Lau et al [16]. An advantage of the model is that there is no registration involved during the subsampling and reconstruction processes, and hence it is computationally efficient.…”

“…Instead of using the whole sequence of frames as the input, we randomly select m = 20 frames from the whole video in the training stage as the input for the generator in the GAN model. In the testing stage, we incorporate a subsampling method [16] to select the most useful frames as the input. The incorporating of the subsampling method into the network is shown to be effective in obtaining a significantly better restored image.…”

“…The function is chosen in order to obtain more information from additional frames, whereas the effect on the quality of the reference image I R is reduced with a marginal increase in the subsample size. The detailed formulation of the variational model is described in [16]. An alternating minimization strategy can be used to solve the model, which is described in Algorithm 2.…”

Blind atmospheric turbulence deconvolution

“…Finally, a deformation guided fusion combined with a semi‐blind deconvolution provides the restored image. In [21], Lau et al proposed to minimise a functional, using different regularisation terms, to perform specific subsampling of frames without any registration. A procedure based on optimal transport has been proposed in [22].…”

Analysis of deep learning architectures for turbulence mitigation in long-range imagery

“…The second being to use post-acquisition processing, which aims to recover the lost high frequency details of the captured image. The field of turbulence mitigation in imagery is a well researched field [2][3][4][5][6][7][8] , where common techniques make use of temporally varying video sequences to obtain either a single high resolution image or a sequence of clean images.…”