The Joint Image Handbook

Abstract

Given multiple perspective photographs, point correspondences form the "joint image", effectively a replica of three-dimensional space distributed across its two-dimensional projections. This set can be characterized by multilinear equations over image coordinates, such as epipolar and trifocal constraints. We revisit in this paper the geometric and algebraic properties of the joint image, and address fundamental questions such as how many and which multilinearities are necessary and/or sufficient to determine camera geometry and/or image correspondences. The new theoretical results in this paper answer these questions in a very general setting and, in turn, are intended to serve as a "handbook" reference about multilinearities for practitioners.

On the Solvability of Viewing Graphs

“…In fact, it is probably quite often erroneously believed that 2n− 3 is the minimal number of fundamental matrices that are required for multi-view reconstruction. Part of the confusion may arise from the fact that the "joint image" [23,22,1], which characterizes multi-view point correspondences in (P 2 ) n , has dimension three (or codimension 2n − 3). This means means that we expect 2n − 3 conditions to be necessary to cut out generically the set of image correspondences among n views.…”

Congruences and concurrent lines in multi-view geometry

“…This reduces the minimal generators to n 2 quadrics and n 3 cubics. These are the bilinearities and trilinearities, well-known in the computer vision community [16,28], that link two and three views. For an algebraic derivation see [3,Corollary 2.7].…”

Ideals of the Multiview Variety

“…GA . Therefore, by Lemmas 3.1 and 2.5, we get [21] says that the H 2 A and H 3 A together cut out the multiview variety which implies that H 2 Theorem 3.7 shows that these polynomials also generate the multiview ideal providing the analogous ideal-theoretic statement.…”

“…, A j c i , is called the epipole in image j relative to image i. Corollary 3.9 shows that while the product of an arbitrary point in image i with all epipoles relative to image i does not appear in the image of ϕ A , these points appear in the multiview variety after taking Zariski closure. See also Proposition 1 in [21].…”

“…A : m. We now make some observations about Theorem 5.6. Theorem 6.1 in [12] observes that VpH 2 A q " VpM A q for noncoplanar A while Proposition 5 (2) in [21] further shows that VpH 2 A q " VpM A q is equivalent to the foci of A being noncoplanar. Our Theorem 5.6 proves the analogous ideal statement, namely that noncoplanarity of foci is equivalent to M A " H 2 A : m. Example 5.2 shows how Theorem 5.6 fails when A is coplanar.…”