Stereoscopic Scene Flow for Robotic Assisted Minimally Invasive Surgery

Abstract

Information about the 3D shape and motion of tissue surfaces at the surgical site during minimally invasive surgery is important for providing metric measurements that enable the deployment of image-guidance and enhanced robotic control. This article presents a scene flow algorithm that recovers the deformation and 3D structure of the surgical field-of-view from stereoscopic images by propagating information starting from a sparse set of candidate seed matches. By imposing spatial and temporal constraints the proposed algorithm is able to reconstruct dense 3D scene flow accurately and efficiently. Validation is performed using simulation data to evaluate the method against varying levels of image noise and results are also presented for benchmark phantom model data. The practical value of proposed method is shown by qualitative results for in vivo videos from robotic assisted procedures.

Real-Time Dense Stereo Reconstruction Using Convex Optimisation with a Cost-Volume for Image-Guided Robotic Surgery

“…The proposed algorithm is quantitatively evaluated by two cardiac datasets collected from [14] which have an associated registered CT model as ground truth as shown in Fig. 3.…”

Optical techniques for 3D surface reconstruction in computer-assisted laparoscopic surgery

“…Another technique also operating at near real-time frame rates that uses local optimization to propagate disparity information around correspondence seeds obtained using feature matching as developed in (Stoyanov et al, 2005b) has also been shown to perform well for endoscopic images . This method has the desirable property of ignoring regions with highlights, occlusions or high uncertainty and has also recently been extended to recover temporal motion as well as 3D shape (Stoyanov, 2012a). An extension of the algorithm showing reduced noise artefacts has also been presented (Bernhardt et al, 2012).…”

“…(e) Disparity image obtained using the images in (b) and (c) with the algorithm reported in where lighter colours are closer to the camera. (f) 3D motion of the surface in images (b) and (c) obtained between two time frames using stereoscopic scene flow where warmer colours represent larger motion (Stoyanov, 2012a). (g) An overlay of the stereoscopic image pair illustrating parallax between the stereo views.…”

Robust surface tracking combining features, intensity and illumination compensation

“…4 to mimic the periodic deformation of the tissue surface induced by the cardiac cycle and respiration [23]. The environment can generate synthetic image sequences by performing small but arbitrary rotations and translations to the pixels of one template image.…”