Gaussian Fusion: Accurate 3D Reconstruction via Geometry-Guided Displacement Interpolation

Chen, Duo; Tang, Zixin; Xu, Zhenyu; Zheng, Yunan; Liu, Yiguang

doi:10.1109/iccv48922.2021.00586

Cited by 5 publications

(10 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Secondly, (𝜇𝜇 𝑡𝑡 ) 0≤𝑡𝑡≤1 is a geodesic curve in the (𝒫𝒫 2 (𝒳𝒳), 𝑊𝑊 2 ) space. Corollary [3] follows from Theorem Displacement interpolation and its corollaries [6], which holds for 𝑝𝑝 > 1. It should be noted that the geodesic curves in 𝒫𝒫 𝓅𝓅 (𝒳𝒳)differ for various 𝑝𝑝 > 1.…”

Section: Corollary [3]mentioning

confidence: 90%

“…The geodesic from 𝑁𝑁(0, 𝛴𝛴 1 ) to 𝑁𝑁(0, 𝛴𝛴 2 ) is 𝑁𝑁�0, 𝐴𝐴(𝑡𝑡)𝛴𝛴 1 𝐴𝐴(𝑡𝑡)� for 𝑡𝑡 ∈ [0,1]. Corollary [3] of Theorem Displacement interpolation presents the concept of displacement interpolation as geodesics on L 2 -Wasserstein space.…”

Section: Corollary [3]mentioning

confidence: 97%

“…Within the context of fusion, displacement interpolation involves the movement of points along this geodesic curve. Gaussian Fusion [3] assumes a sufficiently small local surface follows a Gaussian distribution. While an explicit solution exists for the geodesic curve between Gaussian measures on the L 2 -Wasserstein space [2,5], this approach cannot be directly applied in scenarios with noisy and corrupt point clouds.…”

Section: Optimal Transportmentioning

confidence: 99%

“…The depth map, as illustrated in Figure 1(a), obtained from RGB-D sensors can be transformed into point clouds. Given two input point clouds 𝐴𝐴, 𝐵𝐵 ∈ ℝ 3 , where each point 𝑝𝑝 𝑖𝑖 𝐴𝐴 ∈ 𝐴𝐴, 𝑝𝑝 𝑗𝑗 𝐵𝐵 ∈ 𝐵𝐵 in it are coordinates (𝑥𝑥, 𝑦𝑦, 𝑧𝑧) of the world, the optimal transport methods want to find dense correspondence between A and B. Gaussian Fusion [3] assume these points is an optimal transport problem-namely, moving points from A to B with minimum cost. Gaussian Fusion [3] introduces the displacement Interpolation approach to determine the new fusion point between A and B.…”

Section: D Reconstruction With Rgb-d Sensorsmentioning

confidence: 99%

“…(2)(𝜇𝜇 𝑡𝑡 ) 0≤𝑡𝑡≤1 is a geodesic curve in the space (𝒫𝒫 2 (𝒳𝒳), 𝑊𝑊 2 ). Corollary [3] states that the displacement interpolation can be represented as geodesics on the L 2 -Wasserstein space (see Figure 2), denoted as (𝒫𝒫 2 (𝒳𝒳), 𝑊𝑊 2 ). Here, 𝜇𝜇 0 and 𝜇𝜇 1 are Borel probability measures belonging to 𝒫𝒫 2 (𝒳𝒳), which is constructed by probability measures with order 2, defined on a complete, separable, metric, locally compact space(𝒳𝒳, 𝑑𝑑).…”

Section: Corollary [3]mentioning

confidence: 99%

See 4 more Smart Citations

Optimal Transport Approaches for Cloud Fusion in 3D Reconstruction

Chen

2024

J. Phys.: Conf. Ser.

View full text Add to dashboard Cite

Cloud fusion plays a crucial role in achieving accurate 3D models. This paper introduces two optimal transport-based approaches, Gaussian Fusion and Cyclical Fusion, to showcase the effectiveness of optimal transport in establishing dense correspondence among incoming 3D point clouds. These approaches encompass several important components. Firstly, the fusion correspondences are derived from the optimal transport problem. The core principle of Gaussian Fusion and Cyclical Fusion involves manipulating points along geodesic curves to effectively leverage significant local geometric information. Secondly, this paper explores the fundamental concepts underlying Gaussian Fusion (displacement interpolation) and Cyclical Fusion (cyclical monotonicity), which greatly enhance the accuracy and completeness of the reconstruction process. And provides evidence for the uniqueness of displacement interpolation as geodesics on L2-Wasserstein space and the rationality of cyclical monotonicity in this context. Finally, the proposed approaches excel in capturing intricate surface details, particularly on small objects, when compared to the original fusion scheme which often introduces severe artifacts.

show abstract

Section: Corollary [3]mentioning

confidence: 90%

Section: Corollary [3]mentioning

confidence: 97%

Section: Optimal Transportmentioning

confidence: 99%

Section: D Reconstruction With Rgb-d Sensorsmentioning

confidence: 99%

Section: Corollary [3]mentioning

confidence: 99%

See 3 more Smart Citations

Optimal Transport Approaches for Cloud Fusion in 3D Reconstruction

Chen

2024

J. Phys.: Conf. Ser.

View full text Add to dashboard Cite

show abstract

Single Image Depth Prediction Made Better: A Multivariate Gaussian Take

Liu,

Kumar,

et al. 2023

2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

Neural-network-based single image depth prediction (SIDP) is a challenging task where the goal is to predict the scene's per-pixel depth at test time. Since the problem, by definition, is ill-posed, the fundamental goal is to come up with an approach that can reliably model the scene depth from a set of training examples. In the pursuit of perfect depth estimation, most existing state-of-the-art learning techniques predict a single scalar depth value per-pixel. Yet, it is well-known that the trained model has accuracy limits and can predict imprecise depth. Therefore, an SIDP approach must be mindful of the expected depth variations in the model's prediction at test time. Accordingly, we introduce an approach that performs continuous modeling of per-pixel depth, where we can predict and reason about the per-pixel depth and its distribution. To this end, we model per-pixel scene depth using a multivariate Gaussian distribution. Moreover, contrary to the existing uncertainty modeling methods-in the same spirit, where per-pixel depth is assumed to be independent, we introduce per-pixel covariance modeling that encodes its depth dependency w.r.t. all the scene points. Unfortunately, per-pixel depth covariance modeling leads to a computationally expensive continuous loss function, which we solve efficiently using the learned low-rank approximation of the overall covariance matrix. Notably, when tested on benchmark datasets such as KITTI, NYU, and SUN-RGB-D, the SIDP model obtained by optimizing our loss function shows state-of-the-art results. Our method's accuracy (named MG) is among the top on the KITTI depth-prediction benchmark leaderboard 1 .

show abstract