Dynamic refraction stereo

Abstract

In this paper we consider the problem of reconstructing the 3D position and surface normal of points on an unknown, arbitrarily-shaped refractive surface. We show that two viewpoints are sufficient to solve this problem in the general case, even if the refractive index is unknown. The key requirements are (1) knowledge of a function that maps each point on the two image planes to a known 3D point that refracts to it, and (2) light is refracted only once. We apply this result to the problem of reconstructing the time-varying surface of a liquid from patterns placed below it. To do this, we introduce a novel "stereo matching" criterion called refractive disparity, appropriate for refractive scenes, and develop an optimization-based algorithm for individually reconstructing the position and normal of each point projecting to a pixel in the input views. Results on reconstructing a variety of complex, deforming liquid surfaces suggest that our technique can yield detailed reconstructions that capture the dynamic behavior of free-flowing liquids.

Multi-View Geometry of the Refractive Plane

“…Kutulakos et al [5] investigate the geometry of light-path triangulation, which aims to find conditions and algorithms where reconstruction of individual tracks of light is possible. The work of Morris and Kutulakos [8] looks at refractive stereo. The main idea here is to estimate the normal of the refractive surface, given 2D-3D correspondences, irrespective of the refractive index.…”

“…Until recently, the study of this change had been restricted to multimedia photogrammetry [7] and oceanic engineering [9], where the major perspectives were to either neglect the refraction [3,9], view refraction as an error or aberration to perspective imaging [12,10], or to look at its correction as an iterative optimization problem [7]. In computer vision, some of the first attempts have come in the recent past [5,8,1,15,6]. Kutulakos et al [5] investigate the geometry of light-path triangulation, which aims to find conditions and algorithms where reconstruction of individual tracks of light is possible.…”

Refraction Wiggles for Measuring Fluid Depth and Velocity from Video

“…Several authors have shown that it is possible to recover the shape of the surface of an air-water interface by filming a textured background underneath it [9,16,30]. Alterman et al [4] proposed to recover refraction location and strength from multiple cameras using tomography.…”

Detecting Motion through Dynamic Refraction

“…To recover the water geometry in real time, a theoretical work by Dolin [16] hypothesizes that the WAI slope can be inferred by its specular reflection 1 of a cloudless sky. Multicamera methods have also been applied to WAI recovery [15], [33], [37], [38], [49]. A critical review of passive approaches to recover the WAI is in [22].…”