Efficient Joint Segmentation, Occlusion Labeling, Stereo and Flow Estimation

Yamaguchi, Kazuo; McAllester, David; Urtasun, Raquel

doi:10.1007/978-3-319-10602-1_49

Cited by 256 publications

(254 citation statements)

References 29 publications

Supporting

Mentioning

252

Contrasting

Unclassified

Order By: Relevance

“…As in optical flow estimation, this approach eventually fails to recover large displacements of small objects. Following recent developments in optical flow (Yamaguchi et al, 2013, Nir et al, 2008, Wulff and Black, 2014, Sun et al, 2013 and stereo (Yamaguchi et al, 2014, Bleyer et al, 2011, Bleyer et al, 2012, Vogel et al (Vogel et al, 2013, Vogel et al, 2014 proposed a slanted-plane model which assigns each pixel to an image segment and each segment to one of several rigidly moving 3D plane proposals, thus casting the task as a discrete optimization problem. Fusion moves are leveraged for solving binary subproblems with quadratic pseudo-boolean optimization (QPBO) (Rother et al, 2007).…”

Section: Related Workmentioning

confidence: 99%

Joint 3d Estimation of Vehicles and Scene Flow

Menze

Heipke

Geiger

2015

ISPRS Ann. Photogramm. Remote Sens. Spatial Inf. Sci.

315

215

View full text Add to dashboard Cite

ABSTRACT:Three-dimensional reconstruction of dynamic scenes is an important prerequisite for applications like mobile robotics or autonomous driving. While much progress has been made in recent years, imaging conditions in natural outdoor environments are still very challenging for current reconstruction and recognition methods. In this paper, we propose a novel unified approach which reasons jointly about 3D scene flow as well as the pose, shape and motion of vehicles in the scene. Towards this goal, we incorporate a deformable CAD model into a slanted-plane conditional random field for scene flow estimation and enforce shape consistency between the rendered 3D models and the parameters of all superpixels in the image. The association of superpixels to objects is established by an index variable which implicitly enables model selection. We evaluate our approach on the challenging KITTI scene flow dataset in terms of object and scene flow estimation. Our results provide a prove of concept and demonstrate the usefulness of our method.

show abstract

Section: Related Workmentioning

confidence: 99%

Joint 3d Estimation of Vehicles and Scene Flow

Menze

Heipke

Geiger

2015

ISPRS Ann. Photogramm. Remote Sens. Spatial Inf. Sci.

315

215

View full text Add to dashboard Cite

show abstract

“…Other recent methods [36,37], targeting more specific applications, propose novel multi-frame methods for computing the 3D structure and motion of a scene, as observed from a stereo camera rig on a moving vehicle. The work in [36] is based on the assumption that the captured scene is static; only the stereo rig moves.…”

Section: Previous Relevant Workmentioning

confidence: 99%

“…The work in [36] is based on the assumption that the captured scene is static; only the stereo rig moves. A semi-global matching [18] approach is exploited to independently compute a disparity and a flow field from the stereo and the motion pairs, respectively.…”

Section: Previous Relevant Workmentioning

confidence: 99%

Frequency-Domain Joint Motion and Disparity Estimation Using Steerable Filters

2018

View full text Add to dashboard Cite

Abstract:In this paper, the problem of joint disparity and motion estimation from stereo image sequences is formulated in the spatiotemporal frequency domain, and a novel steerable filter-based approach is proposed. Our rationale behind coupling the two problems is that according to experimental evidence in the literature, the biological visual mechanisms for depth and motion are not independent of each other. Furthermore, our motivation to study the problem in the frequency domain and search for a filter-based solution is based on the fact that, according to early experimental studies, the biological visual mechanisms can be modelled based on frequency-domain or filter-based considerations, for both the perception of depth and the perception of motion. The proposed framework constitutes the first attempt to solve the joint estimation problem through a filter-based solution, based on frequency-domain considerations. Thus, the presented ideas provide a new direction of work and could be the basis for further developments. From an algorithmic point of view, we additionally extend state-of-the-art ideas from the disparity estimation literature to handle the joint disparity-motion estimation problem and formulate an algorithm that is evaluated through a number of experimental results. Comparisons with state-of-the-art-methods demonstrate the accuracy of the proposed approach.

show abstract

“…The proposed system uses the stereo reconstruction described in [10]. While the running time of this algorithm is typically over five seconds for high definition images, the input image size can be considerably reduced by considering only the area surrounding the gaze fixation point.…”

Section: Eye Tracking and Stereo Reconstruction Synergymentioning

confidence: 99%

Intention recognition for gaze controlled robotic minimally invasive laser ablation

Gras

Yang

2016

2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

View full text Add to dashboard Cite

Abstract-Eye tracking technology has shown promising results for allowing hands-free control of robotically-mounted cameras and tools. However existing systems present only limited capabilities in allowing the full range of camera motions in a safe, intuitive manner. This paper introduces a framework for the recognition of surgeon intention, allowing activation and control of the camera through natural gaze behaviour. The system is resistant to noise such as blinking, while allowing the surgeon to look away safely at any time. Furthermore, this paper presents a novel approach to control the translation of the camera along its optical axis using a combination of eye tracking and stereo reconstruction. Combining eye tracking and stereo reconstruction allows the system to determine which point in 3D space the user is fixating, enabling a translation of the camera to achieve the optimal viewing distance. In addition, the eye tracking information is used to perform automatic laser targeting for laser ablation. The desired target point of the laser, mounted on a separate robotic arm, is determined with the eye tracking thus removing the need to manually adjust the laser's target point before starting each new ablation. The calibration methodology used to obtain millimetre precision for the laser targeting without the aid of visual servoing is described. Finally, a user study validating the system is presented, showing clear improvement with median task times under half of those of a manually controlled robotic system.

show abstract

Efficient Joint Segmentation, Occlusion Labeling, Stereo and Flow Estimation

Cited by 256 publications

References 29 publications

Joint 3d Estimation of Vehicles and Scene Flow

Joint 3d Estimation of Vehicles and Scene Flow

Frequency-Domain Joint Motion and Disparity Estimation Using Steerable Filters

Intention recognition for gaze controlled robotic minimally invasive laser ablation

Contact Info

Product

Resources

About