Real-Time Stereo Reconstruction in Robotically Assisted Minimally Invasive Surgery

Abstract

Abstract. The recovery of 3D tissue structure and morphology during robotic assisted surgery is an important step towards accurate deployment of surgical guidance and control techniques in minimally invasive therapies. In this article, we present a novel stereo reconstruction algorithm that propagates disparity information around a set of candidate feature matches. This has the advantage of avoiding problems with specular highlights, occlusions from instruments and view dependent illumination bias. Furthermore, the algorithm can be used with any feature matching strategy allowing the propagation of depth in very disparate views. Validation is provided for a phantom model with known geometry and this data is available online in order to establish a structured validation scheme in the field. The practical value of the proposed method is further demonstrated by reconstructions on various in vivo images of robotic assisted procedures, which are also available to the community.

Robust Dense Endoscopic Stereo Reconstruction for Minimally Invasive Surgery

“…We compared our algorithm to the two latest methods in dense reconstruction from stereo in MIS - [5] and [6] -over various pair of frames from our data. Although their techniques often yielded accurate results, they would still present significant outliers in difficult areas as in figure 1.…”

“…Later [12], Vagvolgyi et al proposed a method to overlay a kidney model onto the stereo display by registering the model to the kidney surface reconstructed from stereo data. More recently [5], Stoyanov et al presented a method to perform near real-time stereo reconstruction in MIS based on belief propagation. A similar work has been recently proposed in [6] where hybrid recursive matching was used.…”

Optical techniques for 3D surface reconstruction in computer-assisted laparoscopic surgery

“…Careful validation, both in controlled environments as well as in clinical scenarios, is crucial for establishing a new system in the clinic (Jannin et al, 2002(Jannin et al, , 2006. So far, most of the validation studies relating to 3D surface reconstruction have been performed on numerically simulated data (Hu et al, 2007;, on phantom models with known ground truth geometry and motion characteristics (Hu et al, 2007;Wu and Qu, 2007;Richa et al, 2008b;Noonan et al, 2009) or with ground truth data obtained by scanning techniques Hu et al, 2007;Wu and Qu, 2007;Richa et al, 2008b;Noonan et al, 2009;Stoyanov et al, 2010). In this context, ensuring that there is no deformation between data acquisition using scanning and observing the tissue with a laparoscope can be a practical challenge.…”

“…To register between the experimental and ground truth coordinate systems, high contrast fiducial markers are typically attached to the object under observation. The most challenging scenarios for obtaining reference data are cadaver, in vivo and wet lab experiments, which are mostly presented to qualitatively demonstrate practical feasibility (Richa et al, 2008a,b;Noonan et al, 2009;Stoyanov et al, 2010). Quantitative in vivo validation of techniques could be performed by calibrating the endoscope with an intra-operative CT as also shown in (Feuerstein et al, 2008) and acquiring the images under breath-hold.…”

“…To advance future research in 3D reconstruction, standardized data repositories need to be established and shared for comparison of the different techniques. Some efforts relating to stereoscopic reconstruction have been made in this direction, but these are just preliminary and need further attention Stoyanov et al, 2010).…”