Toward a Theory of Shape from Specular Flow

Toward robust estimation of specular flow

“…1a), the acquisition of specular image sequences using a custom made device (Fig. 1e), and the computation of ground truth specular flow data from the known shape using the generative equation of Adato et al [1] (Eq. 1).…”

“…Clearly, the inverse relationship between K and the induced specular flow u indicates that the magnitude of the specular flow grows dramatically as surface curvature decreases. Second, while specular flows exhibit magnitude singularities at (for orthographic observer [1]) or near (for perspective observer [21]) the projection of parabolic points, at these points they also exhibit orientation singularities. Formally, if m(x, y) and θ (x, y) are the magnitude and orientation components of the specular flow at each point, respectively, α(s) is a parametric representation of the singularity cirve and β (t) is an image curve that intersects α(s) non transversely at time t = 0, then the one sided limits at that point of intersection satisfy…”

“…While past studies of specular flows have not addressed their estimation from images [1,17,21] fx fxx+ fy fxy (1+ ∇ f 2 ) 2   ω α ω β (1) in which f (x, y) is the specular shape, (ω α , ω β ) T is a description of the relative objectenvironment motion, u is the specular flow, and K is the Gaussian curvature of the surface. Clearly, the inverse relationship between K and the induced specular flow u indicates that the magnitude of the specular flow grows dramatically as surface curvature decreases.…”

“…However, while all these studies clearly demonstrate that specular flows can be exploited successfully (and theoretically rigorously) in various computational ways, they do not address the problem of how specular flows can be estimated from image data. At best, it was suggested to use optical flow algorithms on specular flow image sequences [1]. Indeed, optical flow is classically defined as 'the apparent motion of brightness patterns over the image plane' [12] and in this sense specular flows are optical flows.…”

“…Most recent contributions to such problems have exploited the specular flow -the vector field that is induced on the image plane as a result of a relative motion between the camera, object, or environment -to facilitate diverse tasks such as shape inference [1,9,17,20], 3D pose estimation [10] and detection of rigid objects [23]. However, while all these studies clearly demonstrate that specular flows can be exploited successfully (and theoretically rigorously) in various computational ways, they do not address the problem of how specular flows can be estimated from image data.…”