Rank Constraints for Homographies over Two Views: Revisiting the Rank Four Constraint

Abstract

It is well known that one can collect the coefficients of five (or more) homographies between two views into a large, rank deficient matrix. In principle, this implies that one can refine the accuracy of the estimates of the homography coefficients by exploiting the rank constraint. However, the standard rank-projection approach is impractical for two different reasons: it requires many homographies to even score a modest gain; and, secondly, correlations between the errors in the coefficients will lead to poor estimates.In this paper we study these problems and provide solutions to each. Firstly, we show that the matrices of the homography coefficients can be recast into two parts, each consistent with ranks of only one. This immediately establishes the prospect of realistically (that is, with as few as only three or four homographies) exploiting the redundancies of the homographies over two views. We also tackle the remaining issue: correlated coefficients. We compare our approach with the "gold standard"; that is, non-linear bundle adjustment (initialized from the ground truth estimate-the ideal initialization). The results confirm our theory and show one can implement rank-constrained projection and come close to the gold standard in effectiveness. Indeed, our algorithm (by itself), or our algorithm further refined by a bundle adjustment stage; may be a practical algorithm: providing generally better results than the "standard" DLT (direct linear transformation) algorithm, and even better than the bundle adjustment result with the DLT result as the starting point. Our unoptimized version has roughly the same cost as bundle adjustment and yet can generally produce close to the "gold standard" estimate (as illustrated by comparison with bundle adjustment initialized from the ground truth).Independent of the merits or otherwise of our algorithm, we have illuminated why the naive approach of direct rankprojection is relatively doomed to failure. Moreover, in revealing that there are further rank constraints, not previously known; we have added to the understanding of these issues, and this may pave the way for further improvements.

Enforcing consistency constraints in uncalibrated multiple homography estimation using latent variables

“…This is in contrast with the initialisation proposed by Chen and Suter [4] which requires at least three different homographies.…”

“…. d qq , q = min(9, I), and, correspondingly, let F p (X) 4 = UD 4 V be the 4-truncated SVD of F p (X), with D 4 resulting from D by replacing the entries d 55 , . .…”

“…Adhering to this methodology, Shashua and Avidan [40] have found that homography matrices induced by four or more planes in the 3D scene between two views span a four-dimensional linear subspace. Chen and Suter [4] have derived a set of strengthened constraints for the case of three or more homographies in two views. Zelnik-Manor and Irani [46] have shown that another rank-four constraint applies to a set of so-called relative homographies generated by two planes between four or more views.…”

On the Dimension of the Set of Two-View Multi-Homography Matrices

“…This estimate can be further refined to the inequality dim H ≤ 4I + 10 [3]. Indeed, it follows from (5.1) that any multi-homography matrix H splits as the sum…”

“…This result has its origins in computer vision in the context of solving certain statistical parameter estimation problems [3][4][5]. One issue that arises naturally in connection with these problems is the question of characterising the Zariski closure of H, which is the smallest set containing H defined by finitely many polynomials with real coefficients, as a set of points satisfying explicit constraints put on the ambient Euclidean space.…”