Consistent Video Depth Estimation

Luo, Xuan; Huang, Jia-Bin; Szeliski, Richard; Matzen, Kevin; Kopf, Johannes

doi:10.48550/arxiv.2004.15021

Cited by 6 publications

(12 citation statements)

References 77 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Luo et al [13] fine-tuned a pretrained network for depth estimation to minimize the spatial and disparity losses extracted from 3D geometric constraints. They leveraged the optical flow to establish the coupling between point clouds, and used these coupling for extracting 3D geometric constraints.…”

Section: Related Workmentioning

confidence: 99%

“…From previous studies, it is known that the penalization of a geometric inconsistency can enhance the accuracy of depth estimation [6,9,11,13]. In comparison to other approaches, we propose a novel additional objective, WCL, for depth estimation from monocular camera images.…”

Section: Overviewmentioning

confidence: 99%

“…The origin of coordinate X is equal to camera pose X. Similar to [6,9,13], our WCL penalizes the inconsistency between two point clouds having the same geometry. The WCL is defined as L wass :…”

Section: Overviewmentioning

confidence: 99%

“…3[b], there is no point that does not take a coupling as in Fig. 3[a] [6,9,11,13]. It means that our soft coupling in Fig.…”

Section: Overviewmentioning

confidence: 99%

“…Several previous studies have proposed objectives that attempt to penalize an inconsistency on 3D geometric constraints [6,9,11,13]. However, their performance is limited owing to a strong approximation for the evaluation of consistency [6,11] and the dependence on other incomplete algorithms to obtain the correspondences between point clouds [6,13]. In contrast, our method attempts to directly measure a geometric consistency from 3D point clouds without any indirect process and bold approximation.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

PLG-IN: Pluggable Geometric Consistency Loss with Wasserstein Distance in Monocular Depth Estimation

Hirose¹,

Koide²,

Kawano³

et al. 2020

Preprint

View full text Add to dashboard Cite

We propose a novel objective to penalize geometric inconsistencies, to improve the performance of depth estimation from monocular camera images. Our objective is designed with the Wasserstein distance between two point clouds estimated from images with different camera poses. The Wasserstein distance can impose a soft and symmetric coupling between two point clouds, which suitably keeps geometric constraints and leads differentiable objective. By adding our objective to the original ones of other state-of-the-art methods, we can effectively penalize a geometric inconsistency and obtain a highly accurate depth estimation. Our proposed method is evaluated on the Eigen split of the KITTI raw dataset.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Overviewmentioning

confidence: 99%

Section: Overviewmentioning

confidence: 99%

“…3[b], there is no point that does not take a coupling as in Fig. 3[a] [6,9,11,13]. It means that our soft coupling in Fig.…”

Section: Overviewmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

PLG-IN: Pluggable Geometric Consistency Loss with Wasserstein Distance in Monocular Depth Estimation

Hirose¹,

Koide²,

Kawano³

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

Cystoscopic depth estimation using gated adversarial domain adaptation

et al. 2023

View full text Add to dashboard Cite

Monocular depth estimation from camera images is very important for surrounding scene evaluation in many technical fields from automotive to medicine. However, traditional triangulation methods using stereo cameras or multiple views with the assumption of a rigid environment are not applicable for endoscopic domains. Particularly in cystoscopies it is not possible to produce ground truth depth information to directly train machine learning algorithms for using a monocular image directly for depth prediction. This work considers first creating a synthetic cystoscopic environment for initial encoding of depth information from synthetically rendered images. Next, the task of predicting pixel-wise depth values for real images is constrained to a domain adaption between the synthetic and real image domains. This adaptation is done through added gated residual blocks in order to simplify the network task and maintain training stability during adversarial training. Training is done on an internally collected cystoscopy dataset from human patients. The results after training demonstrate the ability to predict reasonable depth estimations from actual cystoscopic videos and added stability from using gated residual blocks is shown to prevent mode collapse during adversarial training.

show abstract

Tangent Space Backpropagation for 3D Transformation Groups

Teed

Deng

2021

2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

We address the problem of performing backpropagation for computation graphs involving 3D transformation groups SO(3), SE(3), and Sim(3). 3D transformation groups are widely used in 3D vision and robotics, but they do not form vector spaces and instead lie on smooth manifolds. The standard backpropagation approach, which embeds 3D transformations in Euclidean spaces, suffers from numerical difficulties. We introduce a new library, which exploits the group structure of 3D transformations and performs backpropagation in the tangent spaces of manifolds. We show that our approach is numerically more stable, easier to implement, and beneficial to a diverse set of tasks. Our plug-and-play PyTorch library is available at https: //github.com/princeton-vl/lietorch

show abstract

Consistent Video Depth Estimation

Cited by 6 publications

References 77 publications

PLG-IN: Pluggable Geometric Consistency Loss with Wasserstein Distance in Monocular Depth Estimation

PLG-IN: Pluggable Geometric Consistency Loss with Wasserstein Distance in Monocular Depth Estimation

Cystoscopic depth estimation using gated adversarial domain adaptation

Tangent Space Backpropagation for 3D Transformation Groups

Contact Info

Product

Resources

About