Motion estimation using physical simulation

Abstract

Abstract-We consider the task of monocular visual motion estimation from video image sequences. We hypothesise that performance on the task can be improved by incorporating an understanding of physically likely and feasible object dynamics. We test this hypothesis by incorporating a physical simulator into a least-squares estimation procedure. We initialise a full trajectory estimate using RANSAC followed by gradient descent refinement. We present results for 2D image sequences consisting of single ambiguous, visible or occluded balls, as well as results for 3D computer-generated sequences of objects in free-flight with added noise. Results suggest that restricting the estimation to allow only motions that are feasible according to the physics simulator can produce marked improvement when the observed object motion is within the limits of the physics simulator and its world model. Conversely, merely penalising deviations from feasible physical dynamics produces a consistent but incremental improvement over more common dynamics models.

Binding Computer Vision to Physics Based Simulation: The Case Study of a Bouncing Ball

“…This work is most closely related to the works in [3,15,18]. Metaxas and Terzopoulos [18] defined a continuous Kalman filter that was able to track a deformable object.…”

“…However, the shape of the object and the restriction that it is tracked while in flight does not expose the full potential of employing physics. Finally, Duff and Wyatt [15] used physical simulation and search heuristics to track a fast moving ball, despite occlusions. They reasoned upon the ball's 2D position but they did not consider the 3D case, or the hidden variables of ball orientation and angular velocity.…”

Physically Plausible 3D Scene Tracking: The Single Actor Hypothesis

“…Bhat et al [2] performed 3D tracking of an object by searching over parameterized experiments that optimally projected back to an image sequence. Duff and Wyatt [8] used physical simulation and search heuristics to track a fast moving ball, despite occlusions and for the 2D case. In previous work [13], we performed 3D motion estimation for a bouncing ball, from a single camera and despite severe occlusions by exploiting dynamics modelling.…”

Predicting the unobservable Visual 3D tracking with a probabilistic motion model

“…We know of little in the way of literature which specifically addresses the prediction problem in robotic push manipulations of real 3D objects, which are subject to complex 6-dof motions such as tipping and toppling over. It is possible to use physics simulators to predict the motions of interacting rigid bodies, however this approach is reliant on explicit knowledge of the objects, the environment and key physical parameters which can be surprisingly difficult to effectively tune in practice, [15]. Furthermore, once a physics simulator has been set up for a particular scenario, it is not generalizable to new objects or novel situations.…”