Combination of Time-of-Flight depth and stereo using semiglobal optimization

Fischer, Jan; Arbeiter, Georg; Verl, Alexander

doi:10.1109/icra.2011.5979999

Cited by 21 publications

(17 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They are often based on a line search that is guided by the ToF data. Global methods, [5,31,35,27,44,26,32,51,52,40,45] add the ToF information as an additional data term in a global energy functional is then jointly optimized. While the depth maps obtained are smoother due to the usage of prior information/regularizers, this is at the cost of additional computational resources.…”

Section: Overview Of Fusion Methodsmentioning

confidence: 99%

“…While [31,35,27,44,45] employ different graphical models for inference, [51,32] formulate the problem in a variational framework. The last sub-group of the global methods [5,26,52,40] contains those which use other non-local optimization strategies. After a discussion of commonalities in each group, we will proceed to describe each method in detail.…”

Section: Overview Of Fusion Methodsmentioning

confidence: 99%

“…A speed -quality trade off has always to be made depending on the application. For Human Computer Interaction and Robotic/Navigation applications a fast system that is able to detect and eliminate erroneous values [4,5] may suffice. More sophisticated multimedia application on the other hand require high quality depth maps partly with speed constraints imposed on the system (e.g.…”

Section: Requirementsmentioning

confidence: 99%

See 2 more Smart Citations

A Survey on Time-of-Flight Stereo Fusion

Nair

Ruhl

Lenzen

et al. 2013

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. Due to the demand for depth maps of higher quality than possible with a single depth imaging technique today, there has been an increasing interest in the combination of different depth sensors to produce a "super-camera" that is more than the sum of the individual parts. In this survey paper, we give an overview over methods for the fusion of Time-of-Flight (ToF) and passive stereo data as well as applications of the resulting high quality depth maps. Additionally, we provide a tutorial-based introduction to the principles behind ToF stereo fusion and the evaluation criteria used to benchmark these methods.

show abstract

Section: Overview Of Fusion Methodsmentioning

confidence: 99%

Section: Overview Of Fusion Methodsmentioning

confidence: 99%

Section: Requirementsmentioning

confidence: 99%

See 1 more Smart Citation

A Survey on Time-of-Flight Stereo Fusion

Nair

Ruhl

Lenzen

et al. 2013

Lecture Notes in Computer Science

View full text Add to dashboard Cite

show abstract

“…All of these methods are limited to depth-map upsampling using only a single color image and do not exploit the added advantage offered by stereo matching, which can highly enhance the depth map both qualitatively and quantitatively. Recently, [12] proposed a method which combines TOF estimates with stereo in a semiglobal matching framework. However, at pixels where TOF disparity estimates are available, the image similarity term is ignored.…”

Section: A Related Workmentioning

confidence: 99%

High-resolution depth maps based on TOF-stereo fusion

Gandhi

Čech

Horaud

2012

2012 IEEE International Conference on Robotics and Automation

View full text Add to dashboard Cite

Abstract-The combination of range sensors with color cameras can be very useful for robot navigation, semantic perception, manipulation, and telepresence. Several methods of combining range-and color-data have been investigated and successfully used in various robotic applications. Most of these systems suffer from the problems of noise in the range-data and resolution mismatch between the range sensor and the color cameras, since the resolution of current range sensors is much less than the resolution of color cameras. High-resolution depth maps can be obtained using stereo matching, but this often fails to construct accurate depth maps of weakly/repetitively textured scenes, or if the scene exhibits complex self-occlusions. Range sensors provide coarse depth information regardless of presence/absence of texture. The use of a calibrated system, composed of a time-of-flight (TOF) camera and of a stereoscopic camera pair, allows data fusion thus overcoming the weaknesses of both individual sensors. We propose a novel TOF-stereo fusion method based on an efficient seed-growing algorithm which uses the TOF data projected onto the stereo image pair as an initial set of correspondences. These initial "seeds" are then propagated based on a Bayesian model which combines an image similarity score with rough depth priors computed from the low-resolution range data. The overall result is a dense and accurate depth map at the resolution of the color cameras at hand. We show that the proposed algorithm outperforms 2D image-based stereo algorithms and that the results are of higher resolution than off-the-shelf color-range sensors, e.g., Kinect. Moreover, the algorithm potentially exhibits real-time performance on a single CPU.

show abstract

“…A priori methods using ToFCs have been used with most of the common state-of-the-art stereo matching algorithms including dynamic programming (DP) (Gudmundsson et al, 2008), graph cuts (Hahne and Alexa, 2008); (Song et al, 2011), belief propagation (Jiejie Zhu et al, 2011) and semi-global matching (SGM) (Fischer et al, 2011). ToF range data is used to overcome the limitations of stereo in homogenous image regions while stereo range data is retained near depth discontinuities.…”

Section: Data Fusion With Stereo Systemsmentioning

confidence: 99%

Data Fusion of Lidar Into a Region Growing Stereo Algorithm

Veitch-Michaelis

Müller

Storey

et al. 2015

Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci.

View full text Add to dashboard Cite

ABSTRACT:Stereo vision and LIDAR continue to dominate standoff 3D measurement techniques in photogrammetry although the two techniques are normally used in competition. Stereo matching algorithms generate dense 3D data, but perform poorly on low-texture image features. LIDAR measurements are accurate, but imaging requires scanning and produces sparse point clouds. Clearly the two techniques are complementary, but recent attempts to improve stereo matching performance on low-texture surfaces using data fusion have focused on the use of time-of-flight cameras, with comparatively little work involving LIDAR.A low-level data fusion method is shown, involving a scanning LIDAR system and a stereo camera pair. By directly imaging the LIDAR laser spot during a scan, unique stereo correspondences are obtained. These correspondences are used to seed a regiongrowing stereo matcher until the whole image is matched. The iterative nature of the acquisition process minimises the number of LIDAR points needed. This method also enables simple calibration of stereo cameras without the need for targets and trivial coregistration between the stereo and LIDAR point clouds. Examples of this data fusion technique are provided for a variety of scenes.

show abstract

Combination of Time-of-Flight depth and stereo using semiglobal optimization

Cited by 21 publications

References 9 publications

A Survey on Time-of-Flight Stereo Fusion

A Survey on Time-of-Flight Stereo Fusion

High-resolution depth maps based on TOF-stereo fusion

Data Fusion of Lidar Into a Region Growing Stereo Algorithm

Contact Info

Product

Resources

About