Multimodal 3D Shape Reconstruction under Calibration Uncertainty Using Parametric Level Set Methods

Abstract

We consider the problem of 3D shape reconstruction from multi-modal data, given uncertain calibration parameters. Typically, 3D data modalities can come in diverse forms such as sparse point sets, volumetric slices, 2D photos and so on. To jointly process these data modalities, we exploit a parametric level set method that utilizes ellipsoidal radial basis functions. This method not only allows us to analytically and compactly represent the object, it also confers on us the ability to overcome calibration-related noise that originates from inaccurate acquisition parameters. This essentially implicit regularization leads to a highly robust and scalable reconstruction, surpassing other traditional methods. In our results we first demonstrate the ability of the method to compactly represent complex objects. We then show that our reconstruction method is robust both to a small number of measurements and to noise in the acquisition parameters. Finally, we demonstrate our reconstruction abilities from diverse modalities such as volume slices obtained from liquid displacement (similar to CT scans and XRays), and visual measurements obtained from shape silhouettes as well as point-clouds.

Shape from projections via differentiable forward projector for computed tomography

“…This includes (Elangovan and Whitaker, 2001;Whitaker and Elangovan, 2002;Alvino and Yezzi, 2004) that are based on the Mumford-Shah model (Mumford and Shah, 1989) where boundaries are represented using level-sets (Osher and Fedkiw, 2004). Recently, the parametric level-set method (Aghasi et al, 2011) has been used for tomographic segmentation in (Kadu et al, 2018;Eliasof et al, 2020) where level-sets are represented as an aggregation of radial basis functions. Although the parametric levelset method has fewer unknown variables, its forward projection still depends on a regular grid.…”

Multi-Reconstruction from Points Cloud by Using a Modified Vector-Valued Allen–Cahn Equation

“…Bretin et al proposed a variational approach for the reconstruction of a volume from slices by using a minimizer of a geometric regularity criterion, Willmore energy, with inclusion-exclusion constraints associated with the cross-sections. In [32], the authors considered the problem of 3D shape reconstruction from multimodal data, given uncertain calibration parameters. To analytically and compactly represent the object, they exploited a parametric level set method, which utilized ellipsoidal radial basis functions.…”