Image guidance for robotic minimally invasive coronary artery bypass

Kang

Advances in Visual Computing

2014

Abstract. The alignment of pre-operative 3D scans with intra-operative 2D images is important for providing better image guidance. Specifically, overlaying the 3D centerlines of coronary arteries on top of X-ray angiography images reduces the uncertainty inherent in 2D images used during cardiovascular interventions. Because of the dynamic cardiovascular motion from the heartbeat and respiration, a non-rigid registration approach should be applied in contrast registration of the static vascular structure. In this paper, a modified TPS-RPM method is adopted as a non-rigid registration based on a feature-based approach. The proposed method is evaluated on 12 clinical datasets to highlight the necessity of a non-rigid registration approach. IntroductionImage registration between pre-operative images and intra-operative images is a vital technology for image-guided surgery. X-ray angiography (XA) is the modality most commonly used during percutaneous coronary intervention (PCI) for visualization of vessel anatomy and catheter guidance because it is able to capture temporal changes in vessel structures during intervention. However, vessel structures are visible only for 20 seconds after a contrast medium is injected, and the projective nature of angiography makes it difficult for the physician to guide the catheter through the patient's vessel system. Similar challenges occur for other minimally invasive procedures [1,2]. To overcome these challenges, a pre-operative image modality is acquired to yield a 3D total anatomic view that can be used to reduce visual uncertainty and to assess the region of occlusion when planning the intervention. Currently, the most commonly used pre-operative imaging modality for percutaneous coronary interventions is 3D computed tomography angiography (CTA) as a complement. Therefore, image registration between CTA and XA provides surgeons with 3D depth perception and better guidance of the catheter, which leads to increased accuracy.

Section: Introductionmentioning

confidence: 99%

Non-rigid Registration of Vascular Structures for Aligning 2D X-ray Angiography with 3D CT Angiography

Kang

Advances in Visual Computing

2014

Computer Animation & Virtual

“…E-mail: nbeato@eecs.ucf.edu head-mounted display (HMD), a wearable device that captures the user's view of the environment through mounted cameras and displays the processed images to the user via screens in front of the eyes. 2,3 The VST-HMD is especially useful in augmented virtuality where real objects must be separated from the regions where virtual objects will appear in the synthesized image, e.g., within portals, 4 registered to overlay real objects, 5 or providing virtual surrounds as in Figure 1. Typically, blue or green colored physical objects can be used to identify the virtual content, thereby making chroma keying an effective technique for user immersion.…”

Section: Introductionmentioning

confidence: 99%

Interactive chroma keying for mixed reality

Beato

Zhang²,

Colbert³

et al. 2009

In Mixed Reality (MR) applications, immersion of virtual objects in captured video contributes to the perceived unification of two worlds, one real, one synthetic. Since virtual actors and surround may appear both closer and farther than real objects, compositing must consider spatial relationships in the resulting world. Chroma keying, often called blue screening or green screening, is one common solution to this problem. This method is under-constrained and most commonly addressed through a combination of environment preparation and commercial products. In interactive MR domains that impose restrictions on the video camera hardware, such as in experiences using video see-through (VST) head-mounted displays (HMD), chroma keying becomes even more difficult due to the relatively low camera quality, the use of multiple camera sources (one per eye), and the required processing speed. Dealing with these constraints requires a fast and affordable solution. In our approach, we precondition the chroma key by using principal component analysis (PCA) to obtain usable alpha mattes from video streams in real-time on commodity graphics processing units (GPUs). In addition, we demonstrate how our method compares to off-line commercial keying tools and how it performs with respect to signal noise within the video stream.

2013 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)

“…Past attempts to perform AR on deformable organs made use of markers or navigation aids placed close to the area of navigation targets [37] or require interaction to refine elastic registration between pre and intro-operative data [24]. Others [9] build pre-operatively a dynamic 4D model of the heart which is registered to intra-operative data thanks to the use of ECG. In [36], Su et al proposed and 3D-3D iterative closest point (ICP) registration with an image-based tracking to superimpose the 3D model onto laparoscopic images for kidney partial resection.…”

Section: Augmented Reality In Mismentioning

confidence: 99%

Image-guided simulation of heterogeneous tissue deformation for augmented reality during hepatic surgery

Haouchine

Dequidt

Peterlík

et al. 2013

119

107

Figure 1: A sequence of images showing the superimposition of the 3D real-time biomechanical model onto the human liver, undergoing deformation due to instrument interaction during minimally invasive hepatic surgery. The liver is represented in wireframe, the tumor in purple, the hepatic vein is shown in blue and the portal vein in green. ABSTRACTThis paper presents a method for real-time augmentation of vascular network and tumors during minimally invasive liver surgery. Internal structures computed from pre-operative CT scans can be overlaid onto the laparoscopic view for surgery guidance. Compared to state-of-the-art methods, our method uses a real-time biomechanical model to compute a volumetric displacement field from partial three-dimensional liver surface motion. This permits to properly handle the motion of internal structures even in the case of anisotropic or heterogeneous tissues, as it is the case for the liver and many anatomical structures. Real-time augmentation results are presented on in vivo and phantom data and illustrate the benefits of such an approach for minimally invasive surgery.