Parametric Image Alignment Using Enhanced Correlation Coefficient Maximization

Abstract

Abstract-In this work we propose the use of a modified version of the correlation coefficient as a performance criterion for the image alignment problem. The proposed modification has the desirable characteristic of being invariant with respect to photometric distortions. Since the resulting similarity measure is a nonlinear function of the warp parameters, we develop two iterative schemes for its maximization, one based on the forward additive approach and the second on the inverse compositional method. As it is customary in iterative optimization, in each iteration the nonlinear objective function is approximated by an alternative expression for which the corresponding optimization is simple. In our case we propose an efficient approximation that leads to a closed form solution (per iteration) which is of low computational complexity, the latter property being particularly strong in our inverse version. The proposed schemes are tested against the Forward Additive Lucas-Kanade and the Simultaneous Inverse Compositional algorithm through simulations. Under noisy conditions and photometric distortions our forward version achieves more accurate alignments and exhibits faster convergence whereas our inverse version has similar performance as the Simultaneous Inverse Compositional algorithm but at a lower computational complexity.

Efficient and Robust Alignment of Unsynchronized Video Sequences

“…Ideally, however, synchronization would yield a sequence T : IN → IR + providing subframe accuracy. To further refine synchronization results and to spatially align synchronized frames, we extend a recent, robust image alignment algorithm [4].…”

“…The Enhanced Correlation Coefficient (ECC) scheme as reported in [4] supposes that I q (x, y,t 0 ) is the template image and I r (x,ŷ, T (t 0 )) is the input image that must be warped towards the alignment. If A = {x m |m = 1, 2, .…”

“…In order to solve the maximization problem, we assume similar to [4] that a nominal parameter vectorp is known, such that p =p + ∆ p. Then, using a first order Taylor expansion onī p , the ECC function amounts to…”

“…This yields the optimum parameter vector for dense spatio-temporal correspondences of subpixel and subframe accuracy. The complexity per iteration of this scheme can be shown to be O(Mη 2 ), where η is the number of parameters [4]. Figure 4 shows an example of the resulting spatio-temporal alignment.…”

“…This way we can handle the subsequent synchronization problem by means of querying the database for content that is similar to a given frame in the second video sequence. Having thus obtained a rough synchronization, we then address the spatial registration between synchronized frame and the problem of subframe correction and propose a space-time extension of the recently introduced ECC algorithm [4].…”