Practical Depth Estimation with Image Segmentation and Serial U-Nets

Cantrell, Kyle; Miller, Craig; Morato, Carlos W.

doi:10.5220/0009781804060414

Cited by 6 publications

(3 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The three maps in parallel are concatenated along the channel axis to form a tensor of the shape (480, 640, 5). Further, it is fed into a 2D-UNet model with a ResNet-50 backbone [18] and trained end-to-end. The 3DBGES-UNet model is trained for 150 epochs on NYU-Depth v2 data considering 20K samples for training and 694 samples for testing.…”

Section: Proposed 3dbges-unet For Monocular Depth Estimationmentioning

confidence: 99%

See 1 more Smart Citation

A Novel 3D-Unet Deep Learning Framework Based on High-Dimensional Bilateral Grid for Edge Consistent Single Image Depth Estimation

Sharma

Tushar

et al. 2020

2020 International Conference on 3D Immersion (IC3D)

View full text Add to dashboard Cite

The task of predicting smooth and edge-consistent depth maps is notoriously difficult for single image depth estimation. This paper proposes a novel Bilateral Grid based 3D convolutional neural network, dubbed as 3DBG-UNet, that parameterizes high dimensional feature space by encoding compact 3D bilateral grids with UNets and infers sharp geometric layout of the scene. Further, another novel 3DBGES-UNet model is introduced that integrate 3DBG-UNet for inferring an accurate depth map given a single color view. The 3DBGES-UNet concatenates 3DBG-UNet geometry map with the inception network edge accentuation map and a spatial object's boundary map obtained by leveraging semantic segmentation and train the UNet model with ResNet backbone. Both models are designed with a particular attention to explicitly account for edges or minute details. Preserving sharp discontinuities at depth edges is critical for many applications such as realistic integration of virtual objects in AR video or occlusion-aware view synthesis for 3D display applications.The proposed depth prediction network achieves state-of-theart performance in both qualitative and quantitative evaluations on the challenging NYUv2-Depth data. The code and corresponding pre-trained weights will be made publicly available.

show abstract

Section: Proposed 3dbges-unet For Monocular Depth Estimationmentioning

confidence: 99%

“…We quantitatively compare our 3DBGES-UNet model with CNN algorithms Cantrell et al [18] (Serial U-Net), Ramamonjisoa et al [8] (SharpNet), Eigen et al [26] (MSDN) in Table 1. We evaluate these methods using most common error metrics from prior works [18,8,26]. The error metrics are defined as:…”

Section: Evaluation and Comparative Analysismentioning

confidence: 99%

A Novel 3D-Unet Deep Learning Framework Based on High-Dimensional Bilateral Grid for Edge Consistent Single Image Depth Estimation

Sharma

Tushar

et al. 2020

2020 International Conference on 3D Immersion (IC3D)

View full text Add to dashboard Cite

show abstract

“…This not only allows the model to detect the presence of a particular object, but also provides an accurate mapping of the location and boundaries of the object in an image. Depth estimations of the environment can be achieved through the use of stereo vision cameras or through LIDAR, however, deep learning has been able to achieve similar depth predictions using only 2D RGB images as demonstrated in [20] and [21].…”

Section: Introductionmentioning

confidence: 99%

How Many Features is an Image Worth? Multi-Channel CNN for Steering Angle Prediction in Autonomous Vehicles

Munger¹,

Morato²

2021

AI, Machine Learning and Applications

View full text Add to dashboard Cite

This project explores how raw image data obtained from AV cameras can provide a model with more spatial information than can be learned from simple RGB images alone. This paper leverages the advances of deep neural networks to demonstrate steering angle predictions of autonomous vehicles through an end-to-end multi-channel CNN model using only the image data provided from an onboard camera. Image data is processed through existing neural networks to provide pixel segmentation and depth estimates and input to a new neural network along with the raw input image to provide enhanced feature signals from the environment. Various input combinations of Multi-Channel CNNs are evaluated, and their effectiveness is compared to single CNN networks using the individual data inputs. The model with the most accurate steering predictions is identified and performance compared to previous neural networks.

show abstract

Semantic Segmentation and Depth Estimation with RGB and DVS Sensor Fusion for Multi-view Driving Perception

Natan

Miura

2022

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Practical Depth Estimation with Image Segmentation and Serial U-Nets

Cited by 6 publications

References 8 publications

A Novel 3D-Unet Deep Learning Framework Based on High-Dimensional Bilateral Grid for Edge Consistent Single Image Depth Estimation

A Novel 3D-Unet Deep Learning Framework Based on High-Dimensional Bilateral Grid for Edge Consistent Single Image Depth Estimation

How Many Features is an Image Worth? Multi-Channel CNN for Steering Angle Prediction in Autonomous Vehicles

Semantic Segmentation and Depth Estimation with RGB and DVS Sensor Fusion for Multi-view Driving Perception

Contact Info

Product

Resources

About