Efficient Dense Scene Flow from Sparse or Dense Stereo Data

Wedel, Andreas; Rabe, Clemens; Vaudrey, Tobi; Brox, Thomas; Franke, Uwe; Cremers, Daniel

doi:10.1007/978-3-540-88682-2_56

Cited by 148 publications

(152 citation statements)

References 16 publications

(26 reference statements)

Supporting

Mentioning

150

Contrasting

Unclassified

Order By: Relevance

“…One direct application is the computation of scene flow, incorporating three data terms [28]. We are currently investigating extensions of the proposed optical flow method to adopt it to this stereo scene flow case.…”

mentioning

confidence: 99%

An Improved Algorithm for TV-L 1 Optical Flow

Wedel

Pock

Zach

et al. 2009

Lecture Notes in Computer Science

Self Cite

308

216

View full text Add to dashboard Cite

Fig. 1. Optical flow for the backyard and mini cooper scene of the Middlebury optical flow benchmark. Optical flow captures the dynamics of a scene by estimating the motion of every pixel between two frames of an image sequence. The displacement of every pixel is shown as displacement vectors on top of the commonly used flow color scheme (see Figure 5). Abstract.A look at the Middlebury optical flow benchmark [5] reveals that nowadays variational methods yield the most accurate optical flow fields between two image frames. In this work we propose an improvement variant of the original duality based TV-L 1 optical flow algorithm in [31] and provide implementation details. This formulation can preserve discontinuities in the flow field by employing total variation (TV) regularization. Furthermore, it offers robustness against outliers by applying the robust L 1 norm in the data fidelity term.Our contributions are as follows. First, we propose to perform a structure-texture decomposition of the input images to get rid of violations in the optical flow constraint due to illumination changes. Second, we propose to integrate a median filter into the numerical scheme to further increase the robustness to sampling artefacts in the image data. We experimentally show that very precise and robust estimation of optical flow can be achieved with a variational approach in realtime. The numerical scheme and the implementation are described in a detailed way, which enables reimplementation of this high-end method.

show abstract

“…We estimate the disparity for each frame independently. A joint estimation of motion and disparity from video is also possible [38]. We assume that the stereo pair is approximately rectified, i.e., for a particular pixel in view 1 the corresponding pixel in view 2 lies close Figure 4.…”

Section: Methodsmentioning

confidence: 99%

Pose Estimation and Segmentation of People in 3D Movies

Alahari

Seguin

Sivic

et al. 2013

2013 IEEE International Conference on Computer Vision

View full text Add to dashboard Cite

We seek to obtain a pixel-wise segmentation and pose estimation of multiple people in a stereoscopic video. This involves challenges such as dealing with unconstrained stereoscopic video, non-stationary cameras, and complex indoor and outdoor dynamic scenes. The contributions of our work are two-fold: First, we develop a segmentation model incorporating person detection, pose estimation, as well as colour, motion, and disparity cues. Our new model explicitly represents depth ordering and occlusion. Second, we introduce a stereoscopic dataset with frames extracted from feature-length movies "StreetDance 3D" and "Pina". The dataset contains 2727 realistic stereo pairs and includes annotation of human poses, person bounding boxes, and pixel-wise segmentations for hundreds of people. The dataset is composed of indoor and outdoor scenes depicting multiple people with frequent occlusions. We demonstrate results on our new challenging dataset, as well as on the H2view dataset from (Sheasby et al. ACCV 2012).

show abstract

“…However, it is difficult to recover a scene flow compatible with several observed optical flows which may be contradictory. Some authors introduce constraints of a full calibrated stereo structure [7,8,9,10].…”

Section: Related Workmentioning

confidence: 99%

Local/global scene flow estimation

Quiroga

Devernay

Crowley

2013

2013 IEEE International Conference on Image Processing

View full text Add to dashboard Cite

The scene flow describes the 3D motion of every point in a scene between two time steps. We present a novel method to estimate a dense scene flow using intensity and depth data. It is well known that local methods are more robust under noise while global techniques yield dense motion estimation. We combine local and global constraints to solve for the scene flow in a variational framework. An adaptive TV (Total Variation) regularization is used to preserve motion discontinuities. Besides, we constrain the motion using a set of 3D correspondences to deal with large displacements. In the experimentation our approach outperforms previous scene flow from intensity and depth methods in terms of accuracy.

show abstract

“…The method computes scene flow by joint estimation of the disparity maps and the motion field from a calibrated stereoscopic image sequence within a unified variational framework. In [15], the depth and 3D motion are decoupled because the nature of motion estimation and disparity estimation are very different and the problems can be solved more efficiently.…”

Section: Related Workmentioning

confidence: 99%

“…This paper adopts the approach in [14,15] to estimate the 3D motion, but for a different purpose. A prior probabilistic model is built from the scene flow estimated and used in stereo estimation at time t+1.…”

Section: Related Workmentioning

confidence: 99%

Disparity Estimation in Stereo Sequences using Scene Flow

Liu

Philomin

2009

Procedings of the British Machine Vision Conference 2009

View full text Add to dashboard Cite

This paper presents a method for estimating disparity images from a stereo image sequence. While many existing stereo algorithms work well on a single pair of stereo images, it is not sufficient to simply apply them to temporal frames independently without considering the temporal consistency between adjacent frames. Our method integrates the state-of-the-art stereo algorithm with the scene flow concept in order to capture the temporal correspondences. It computes the dense disparity images and scene flow in a practical and unified process: the disparity is initialized by a hybrid stereo approach which employs the over-segmentation based stereo and pixelwise iterative stereo; then the scene flow, estimated via a variational approach, is used to predict the disparity image and to compute its confidence map for the next frame. The prediction is modeled as a prior probability distribution and is built into an energy function defined for stereo matching on the next frame. The disparity can be estimated by minimizing this energy function. Experimental results show that the algorithm is able to estimate the disparity images in an accurate and temporally consistent fashion.

show abstract

Efficient Dense Scene Flow from Sparse or Dense Stereo Data

Cited by 148 publications

References 16 publications

An Improved Algorithm for TV-L 1 Optical Flow

Pose Estimation and Segmentation of People in 3D Movies

Local/global scene flow estimation

Disparity Estimation in Stereo Sequences using Scene Flow

Contact Info

Product

Resources

About