EdgeStereo: An Effective Multi-task Learning Network for Stereo Matching and Edge Detection

Abstract

Recently, leveraging on the development of endto-end convolutional neural networks (CNNs), deep stereo matching networks have achieved remarkable performance far exceeding traditional approaches. However, state-of-theart stereo frameworks still have difficulties at finding correct correspondences in texture-less regions, detailed structures, small objects and near boundaries, which could be alleviated by geometric clues such as edge contours and corresponding constraints. To improve the quality of disparity estimates in these challenging areas, we propose an effective multi-task learning network, EdgeStereo, composed of a disparity estimation branch and an edge detection branch, which enables end-to-end predictions of both disparity map and edge map. To effectively incorporate edge cues, we propose the edge-aware smoothness loss and edge feature embedding for inter-task interactions. It is demonstrated that based on our unified model, edge detection task and stereo matching task can promote each other. In addition, we design a compact module called residual pyramid to replace the commonly-used multi-stage cascaded structures or 3-D convolution based regularization modules in current stereo matching networks. By the time of the paper submission, EdgeStereo achieves state-of-art performance on the Fly-ingThings3D dataset, KITTI 2012 and KITTI 2015 stereo benchmarks, outperforming other published stereo matching methods by a noteworthy margin. EdgeStereo also achieves comparable generalization performance for disparity estimation because of the incorporation of edge cues.

Self-Supervised Monocular Depth Hints

“…It is tempting to use a state-of-the-art stereo algorithm instead, e.g. [3,2,29], however most modern stereo algorithms are supervised using the LiDAR ground truth from the KITTI dataset. Using one of these would cause us to be implicitly learning from laser-scanned ground-truth data.…”

Review of Stereo Matching Algorithms Based on Deep Learning

“…e FlowNet provides the basic 2D encoder-decoder structure. Later, a lot of networks [23,24,26,27,32] have been proposed based on this. Optical flow estimation requires precise per-pixel localization, and it also depends on finding correspondences between two input images.…”

“…Tons of algorithms based on this have been proposed. ese methods could roughly be categorized into two groups: 2D encode-decoder structures [23][24][25][26][27] and regularization modules composed of 3D convolutions [28][29][30][31]. DispNetC [24] computes a correlation volume from the left and right image features (encoding) and utilizes a CNN to directly regress (decoding) disparity maps.…”

Performance analysis of a fuzzy disparity selector for stereo matching of image segments under radiometric variations

“…The authors propose a feature-based matching methodology as opposed to a deep learning-based approach. The main reason for this decision is the fact that most deep learning methods demand a large amount of processing power [5][6][7]. This will be an extremely limiting factor if environments with restricted resources are considered where the resources are not abundant.…”