Non-rigid structure from locally-rigid motion

Abstract

The non-rigid structure from motion problem typically involves recovering the 3D trajectories of a set of scene points, from their corresponding image trajectories. In this thesis, the assumption of locally-rigid motion is used to regularize this otherwise underconstrained problem. The key idea is that even when a scene undergoes complex global deformations, the trajectories of local triplets of scene points can often be approximated by the vertices of a rigidly moving triangle. This intuition informs our bottom-up reconstruction procedure, which discovers such triplets through a hypothesis and test framework. To this end, a rigid triangle model is fit to the proposed image trajectories and evaluated using a procedure that we call 3-SFM. The recovered triangle models are then integrated into a global solution, by resolving their orthographic depth flip and translation ambiguities. Lastly, we consider using this solution to initialize an energy based model, subject to a set of soft isometric constraints, in order to allow each observation to constrain the global scene structure. Results on several sequences, both our own and from related work, suggest that these models are applicable in diverse and challenging scenes, such as those including multiple deforming bodies.

Optical techniques for 3D surface reconstruction in computer-assisted laparoscopic surgery

“…One of the unsolved problems therefore concerns the use of a more advanced model of the laparoscopic environment's Bidirectional Reflectance Distribution Function (BRDF). An interesting avenue for future research is applying template-free DSfM in the context of laparoscopy (Taylor et al, 2010;Russell et al, 2011;Collins and Bartoli, 2010;Varol et al, 2009) and studying its combination with SfS (Moreno-Noguer et al, 2010).…”

Combining Local-Physical and Global-Statistical Models for Sequential Deformable Shape from Motion

“…Recently, results from this field have significantly advanced. Stronger deformations have been tackled using piecewise models (Chhatkuli et al 2014;Fayad et al 2010;Russell et al 2011;Taylor et al 2010), even combining segmentation and reconstruction under local rigidity (Russell et al 2014), or eliminating the rank dependency by means of Procustean normal distributions (Lee et al 2013). In (Garg et al 2013), a variational approach integrating a low-rank shape model with spatial smoothness allowed per-pixel dense reconstructions.…”

“…The Tear sequence (Taylor et al 2010) contains 167 frames of a paper being split in two parts. We use the point tracks provided by (Russell et al 2011).…”

“…Since the deformation on this video is very local, it was originally tackled using piecewise techniques (Taylor et al 2010;Russell et al 2011). Our particle-based approach also handles this type of deformation without much difficulty.…”

“…Yet, while low-rank meth-ods can effectively encode global deformations, they cannot, in general, handle non-linear motion patterns and strong local deformations. Piecewise strategies (Chhatkuli et al 2014;Russell et al 2011;Taylor et al 2010;Varol et al 2009) allow recovering larger deformations, although their performance highly depends on having overlapping features in neighboring patches, or require large number of correspondences to enforce local rigidity constraints (Chhatkuli et al 2014;Taylor et al 2010;Varol et al 2009), which can be hard to obtain in practice. In any event, these previous approaches batch process all frames of the sequence at once, after video capture, preventing them from being used online and in real-time applications, where NRSfM may have an enormous potential.…”