Fully convolutional multi‐scale dense networks for monocular depth estimation

Liu, Jiwei; Zhang, Yunzhou; Cui, Jiahua; Feng, Yonghui; Pang, Linzhuo

doi:10.1049/iet-cvi.2018.5645

Cited by 8 publications

(7 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Xu et al [19] Fu et al [10] Liu et al [21] Our estimations can be directly applied. 2) The depth ground truth gotten by LIDAR can be straightly exploited without processing into a dense map.…”

Section: B State-of-the-art Comparisionsmentioning

confidence: 99%

“…It is worth noting that Eigen et al [2] and Liu et al [21] used the effective part of ground truth data, which was the lower 2/3 part of the image. Meanwhile, Liu et al [21] and Fu et al [10] applied the dense depth maps to train the network. While the input of our network is the full image, and the original sparse data is utilized as the ground truth.…”

Section: B State-of-the-art Comparisionsmentioning

confidence: 99%

“…Meanwhile, these methods fused features based on the information extracted by the encoder which decides the richness of the information. Liu et al [21] and Su et al [22] chose the channel attention mechanism for enriching the semantic information of the features, but for spatial details, it was not improved well. In contrast, We reckon that utilizing the spatial attention mechanism can obtain outstanding performance by mining the spatial details.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

MSFNet:Multi-scale features network for monocular depth estimation

Pei

2021

Preprint

View full text Add to dashboard Cite

In recent years, monocular depth estimation is applied to understand the surrounding 3D environment and has made great progress. However, there is an ill-posed problem on how to gain depth information directly from a single image. With the rapid development of deep learning, this problem is possible to be solved. Although more and more approaches are proposed one after another, most of existing methods inevitably lost details due to continuous downsampling when mapping from RGB space to depth space. To the end, we design a Multi-scale Features Network (MSFNet), which consists of Enhanced Diverse Attention (EDA) module and Upsample-Stage Fusion (USF) module. The EDA module employs the spatial attention method to learn significant spatial information, while USF module complements low-level detail information with high-level semantic information from the perspective of multi-scale feature fusion to improve the predicted effect. In addition, since the simple samples are always trained to a better effect first, the hard samples are difficult to converge. Therefore, we design a batch-loss to assign large loss factors to the harder samples in a batch. Experiments on NYU-Depth V2 dataset and KITTI dataset demonstrate that our proposed approach is more competitive with the state-of-the-art methods in both qualitative and quantitative evaluation.

show abstract

Section: B State-of-the-art Comparisionsmentioning

confidence: 99%

Section: B State-of-the-art Comparisionsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

MSFNet:Multi-scale features network for monocular depth estimation

Pei

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Ranftl et al [16] suggested the application of vision transformers as the backbone for dense predictions, where tokens from the different stages of the vision transformer were assembled in an image-like representation using a convolutional decoder. Liu et al [21] proposed a similar approach to ours in which they adopted fully convolutional multiscale dense network based on DenseNet169 [22] for monocular depth estimation. They also proposed a dense upsampling block that includes a sequence of convolutional filters followed by a pixel shuffle operation to obtain higher resolution output.…”

Section: Related Workmentioning

confidence: 99%

DTS-Depth: Real-Time Single-Image Depth Estimation Using Depth-to-Space Image Construction

Ibrahem

Salem

Kang

2022

Sensors

View full text Add to dashboard Cite

As most of the recent high-resolution depth-estimation algorithms are computationally so expensive that they cannot work in real time, the common solution is using a low-resolution input image to reduce the computational complexity. We propose a different approach, an efficient and real-time convolutional neural network-based depth-estimation algorithm using a single high-resolution image as the input. The proposed method efficiently constructs a high-resolution depth map using a small encoding architecture and eliminates the need for a decoder, which is typically used in the encoder–decoder architectures employed for depth estimation. The proposed algorithm adopts a modified MobileNetV2 architecture, which is a lightweight architecture, to estimate the depth information through the depth-to-space image construction, which is generally employed in image super-resolution. As a result, it realizes fast frame processing and can predict a high-accuracy depth in real time. We train and test our method on the challenging KITTI, Cityscapes, and NYUV2 depth datasets. The proposed method achieves low relative absolute error (0.028 for KITTI, 0.167 for CITYSCAPES, and 0.069 for NYUV2) while working at speed reaching 48 frames per second on a GPU and 20 frames per second on a CPU for high-resolution test images. We compare our method with the state-of-the-art methods on depth estimation, showing that our method outperforms those methods. However, the architecture is less complex and works in real time.

show abstract

“…Eigen et al [33] first applied a multi-scale CNN architecture to predict depth maps from monocular images, which helps capture image details. Following this, some other CNN-based methods [34] were proposed to estimate monocular depth. Xu et al [35] combined CNN and conditional random field to improve the smoothness of estimated depth maps.…”

Section: Depth Prediction From a Single Imagementioning

confidence: 99%

A Unified Framework for Depth Prediction from a Single Image and Binocular Stereo Matching

et al. 2020

View full text Add to dashboard Cite

Depth information has long been an important issue in computer vision. The methods for this can be categorized into (1) depth prediction from a single image and (2) binocular stereo matching. However, these two methods are generally regarded as separate tasks, which are accomplished in different network architectures when using deep learning-based methods. This study argues that these two tasks can be achieved using only one network with the same weights. We modify existing networks for stereo matching to perform the two tasks. We first enable the network capable of accepting both a single image and an image pair by duplicating the left image when the right image is absent. Then, we introduce a training procedure that alternatively selects training samples of depth prediction from a single image and binocular stereo matching. In this manner, the trained network can perform both tasks and single-image depth prediction even benefits from stereo matching to achieve better performance. Experimental results on KITTI raw dataset show that our model achieves state-of-the-art performances for accomplishing depth prediction from a single image and binocular stereo matching in the same architecture.

show abstract

Fully convolutional multi‐scale dense networks for monocular depth estimation

Cited by 8 publications

References 39 publications

MSFNet:Multi-scale features network for monocular depth estimation

MSFNet:Multi-scale features network for monocular depth estimation

DTS-Depth: Real-Time Single-Image Depth Estimation Using Depth-to-Space Image Construction

A Unified Framework for Depth Prediction from a Single Image and Binocular Stereo Matching

Contact Info

Product

Resources

About