In-Situ Joint Light and Medium Estimation for Underwater Color Restoration

Abstract

The majority of Earth's surface is situated in the deep sea and thus remains deprived of natural light. Such adverse underwater environments have to be explored with powerful camera-light systems. In order to restore the colors in images taken by such systems, we need to jointly estimate physically-meaningful optical parameters of the light as well as the water column. We thus propose an integrated in-situ estimation approach and a complementary surface texture recovery strategy, which also removes shadows as a by-product. As we operate in a scattering medium under inhomogeneous lighting conditions, the volumetric effects are difficult to capture in closed-form solutions. Hence, we leverage the latest progress in Monte Carlo-based differentiable ray tracing that becomes tractable through recent GPU RTX-hardware acceleration. Evaluations on synthetic data and in a water tank show that we can estimate physically meaningful parameters, which enables color restoration. The approaches could also be employed to other camera-light systems (AUV, robot, car, endoscope) operating either in the dark, in fog -or -underwater.

Investigation of the Challenges of Underwater-Visual-Monocular-Slam

“…However, to make this approach work, the artificial illumination has to be locally homogeneous. Further approaches, which can be applied to true heterogeneous underwater artificial scenarios, i.e., with artificial illumination, are presented in (Bryson et al, 2016) and (Nakath et al, 2021). While physically-based methods have the advantage of being based on well-established principles, they can be limited by the availability and accuracy of the precise parameters needed for the models.…”

Optical Imaging and Image Restoration Techniques for Deep Ocean Mapping: A Comprehensive Survey

“…Some researchers assume the proportion of scattered light has a uniform directional distribution, such that the corresponding VSF becomes constant (Bryson et al 2016) and might be negligible during the restoration. Only some works actually attempt to estimate the VSF parameters from images: Tsiotsios et al 2014) use the phase function from Chandrasekhar (2013), (Murez et al 2015;Nakath et al 2021;Narasimhan et al 2006;Spier et al 2017;Tian et al 2017) utilize the HG phase function and (Pegoraro et al 2010) models a general phase function model by using Legendre polynomial basis or Taylor series.…”

“…Powered by advances in GPU technology and physics-based simulation, nowadays graphic engines are able to synthesize complex underwater effects using ray-tracing efficiently (Zwilgmeyer et al 2021). Latest approaches even employ Monte Carlobased differentiable ray-tracing to replace an explicit image formation model for image restoration, by simply characterizing the water by differential properties and then optimizing (Nakath et al 2021). Such an approach can implicitly handle multi-scattering, shadows as well as different phase functions.…”

Symptomatic vitreous opacity image enhancement using linear and nonlinear transformations

“…where the correction function H maps to its corresponding matrix M . We refer to the trained mapping model in the literature [24] to complete the mapping of the input clusters to the correction function H . The mapping is done by the mapping model in Eq.…”