Estimation of 2-D Motion Fields from Image Sequences with Application to Motion-Compensated Processing

Abstract

In this chapter we are concerned with the estimation of 2-D motion from time-varying images and with the application of the computed motion to image sequence processing. Our goal for motion estimation is to propose a general formulation that incorporates object acceleration, nonlinear motion trajectories, occlusion effects and multichannel (vector) observations. To achieve this objective we use Gibbs-Markov models linked together by the Maximum A Posteriori Probability criterion which results in minimization of a multiple-term cost function. The specific applications of motion-compensated processing of image sequences are prediction, noise reduction and spatiotemporal interpolation.Estimation of motion from dynamic images is a very difficult task due to its ill-posedness [4]. Despite this difficulty, however, many approaches to the problem have been proposed in the last dozen years [27], [24], [40]. This activity can certainly be attributed in large measure to the importance of motion in the processing and coding of image sequences. Below we explain why motion is important in these tasks.Efficient encoding of time-varying images is essential to provide economical use of network or storage facilities in the provision of video services. Image sequences can be compressed by independent coding of each frame (intraframe coding) or by straightforward extension of spatial coding techniques to three dimensions (e.g., 3-D transform coding). However, such approaches ignore the fact that the majority of new information (innovations) in a time-varying image is carried by the motion. The correlation of image intensity or color is very high along the direction of motion. Thus, the knowledge of motion helps in removing significant interimage redundancy, as is the case in predictive or hybrid (predictive/transform or predictive/subband) coding compensated for motion. In fact, these techniques include most algorithms currently used in videoconferencing [41], [42] and proposed for High Definition Television (HDTV) [21], [60].Sampling structure conversion and noise reduction are two examples of image sequence processing that can benefit greatly from the knowledge of motion. In the case of conversion, two scenarios are possible. In one situation a missing image must be recovered. Again, due to the high correlation along motion trajectories, motion-compensated interpolation is the most effective tool [53], [15]. In the other

A Rehabilitation Tool for Functional Balance using Altered Gravity and Virtual Reality

“…The estimation of disparity is typically achieved by assuming an invariant property, such as brightness or color, and then establishing correspondence between images based on this property [26][27][28]. Additional work by Konrad [25,[29][30][31] has addressed these issues. Based on disparities between two calibrated views, depth (structure) of the captured 3-D scene can be computed which, in turn, permits the reconstruction of views from virtual cameras.…”

Object-based estimation of dense motion fields

“…When neglecting temporal dependencies and setting in (11) for the moment, the a priori distribution becomes Hence, the distribution of the motion field given the segmentation yields and the a priori distribution of the segmentation yields (15) In general, the normalizing factor depends on the realization of the random field Therefore, the latter is not a Gibbs distribution and in particular is different from the distribution in (12). Let us, for the moment, consider a virtual segmentation where all segments are background segments, i.e., the segmentation only includes occluded edges.…”

“…In other applications the knowledge of motion is not the ultimate goal but only a tool to improve efficiency of some subsequent operation, benefitting from the strong statistical dependency of image intensity along motion trajectories. In particular, spatiotemporal filters can be steered in these directions leading to improved performance, e.g., for noise reduction, image raster conversion [15], or for prediction in image sequence compression [40].…”

Three-dimensional motion estimation of objects for video coding

“…The velocity of any point of the rigid body can be represented by the sum of a translation velocity and a rotation velocity, namely (1) where is the vector of the angular velocities. Thus, six parameters are sufficient to characterize the motion.…”

“…A way to accomplish this is by estimating the displacement between frames of image elements, which can be individual pixels [1], picture blocks of fixed dimension (as in block matching motion compensation [2]), or groups of pixels corresponding to moving objects in the scene [3].…”