A Real-Time Interactive Augmented Reality Depth Estimation Technique for Surgical Robotics

Kalia, Megha; Navab, Nassir; Salcudean, Tim

doi:10.1109/icra.2019.8793610

Cited by 29 publications

(13 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In machine perception, recognition of some functional factors such as the shape of a scene from an image and image independence from its appearance seems to be fundamental [ 7 , 8 , 9 ]. DE has great potential for use in disparate applications, including grasping in robotics, robot-assisted surgery, computer graphics, and computational photography [ 10 , 11 , 12 , 13 , 14 , 15 ]. Figure 1 schematically illustrates the evaluation trend of DE.…”

Section: Introductionmentioning

confidence: 99%

Monocular Depth Estimation Using Deep Learning: A Review

Masoumian

Rashwan

Cristiano

et al. 2022

Sensors

View full text Add to dashboard Cite

In current decades, significant advancements in robotics engineering and autonomous vehicles have improved the requirement for precise depth measurements. Depth estimation (DE) is a traditional task in computer vision that can be appropriately predicted by applying numerous procedures. This task is vital in disparate applications such as augmented reality and target tracking. Conventional monocular DE (MDE) procedures are based on depth cues for depth prediction. Various deep learning techniques have demonstrated their potential applications in managing and supporting the traditional ill-posed problem. The principal purpose of this paper is to represent a state-of-the-art review of the current developments in MDE based on deep learning techniques. For this goal, this paper tries to highlight the critical points of the state-of-the-art works on MDE from disparate aspects. These aspects include input data shapes and training manners such as supervised, semi-supervised, and unsupervised learning approaches in combination with applying different datasets and evaluation indicators. At last, limitations regarding the accuracy of the DL-based MDE models, computational time requirements, real-time inference, transferability, input images shape and domain adaptation, and generalization are discussed to open new directions for future research.

show abstract

Section: Introductionmentioning

confidence: 99%

Monocular Depth Estimation Using Deep Learning: A Review

Masoumian

Rashwan

Cristiano

et al. 2022

Sensors

View full text Add to dashboard Cite

show abstract

“…Estimation of dense depth measurements is crucial in various 3D vision and robotics applications such as augmented and mixed reality [1], [2], scene reconstruction [3], [4], autonomous driving [5], [6], and obstacle avoidance [7], [8]. For obtaining a reliable prediction of depth in outdoor scenes, measurements from various sensors are used.…”

Section: Introductionmentioning

confidence: 99%

SemAttNet: Toward Attention-Based Semantic Aware Guided Depth Completion

et al. 2022

View full text Add to dashboard Cite

Depth completion involves recovering a dense depth map from a sparse map and an RGB image. Recent approaches focus on utilizing color images as guidance images to recover depth at invalid pixels. However, color images alone are not enough to provide the necessary semantic understanding of the scene. Consequently, the depth completion task suffers from sudden illumination changes in RGB images (e.g., shadows). In this paper, we propose a novel three-branch backbone comprising color-guided, semanticguided, and depth-guided branches. Specifically, the color-guided branch takes a sparse depth map and RGB image as an input and generates color depth which includes color cues (e.g., object boundaries) of the scene. The predicted dense depth map of color-guided branch along-with semantic image and sparse depth map is passed as input to semantic-guided branch for estimating semantic depth. The depth-guided branch takes sparse, color, and semantic depths to generate the dense depth map. The color depth, semantic depth, and guided depth are adaptively fused to produce the output of our proposed three-branch backbone. In addition, we also propose to apply semantic-aware multi-modal attention-based fusion block (SAMMAFB) to fuse features between all three branches. We further use CSPN++ with Atrous convolutions to refine the dense depth map produced by our three-branch backbone. Extensive experiments show that our model achieves state-of-the-art performance in the KITTI depth completion benchmark at the time of submission.INDEX TERMS State-of-the-art Depth Completion approach on KITTI depth completion benchmark, Attention-based fusion for depth completion, Semantic-guided depth completion

show abstract

“…Depth values are considered to be the most essential information in the fields of autonomous driving [1], [2] augmented reality [3], [4] virtual reality [5], [6] etc. To obtain reliable values of depth for indoor and outdoor scenes, various sensors are used to measure it.…”

Section: Introductionmentioning

confidence: 99%