Feature-based real-time endoscopic mosaicking

Bergen, Tobias; Ruthotto, Steffen; Münzenmayer, Christian; Rupp, Stephan; Paulus, Dietrich; Winter, Christian

doi:10.1109/ispa.2009.5297633

Cited by 15 publications

(10 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They achieved a rate of 5 fps on standard PC hardware in 2010. Bergen et al described live stitching of liver images at a frame rate of 7 fps in 2009 [23]. Since 2006, Hager's group [119], [127] have presented several real-time mosaicking approaches for the retina (see section II-B), reporting frame rates of 30 fps.…”

Section: G Online Vs Offline Methodsmentioning

confidence: 99%

Stitching and Surface Reconstruction From Endoscopic Image Sequences: A Review of Applications and Methods

Bergen

Wittenberg

2016

IEEE J. Biomed. Health Inform.

Self Cite

View full text Add to dashboard Cite

Endoscopic procedures form part of routine clinical practice for minimally invasive examinations and interventions. While they are beneficial for the patient, reducing surgical trauma and making convalescence times shorter, they make orientation and manipulation more challenging for the physician, due to the limited field of view through the endoscope. However, this drawback can be reduced by means of medical image processing and computer vision, using image stitching and surface reconstruction methods to expand the field of view. This paper provides a comprehensive overview of the current state of the art in endoscopic image stitching and surface reconstruction. The literature in the relevant fields of application and algorithmic approaches is surveyed. The technological maturity of the methods and current challenges and trends are analyzed.

show abstract

Section: G Online Vs Offline Methodsmentioning

confidence: 99%

Stitching and Surface Reconstruction From Endoscopic Image Sequences: A Review of Applications and Methods

Bergen

Wittenberg

2016

IEEE J. Biomed. Health Inform.

Self Cite

View full text Add to dashboard Cite

show abstract

“…In white light endoscopy, visual artifacts in the transition regions of the output image are less critical for surgeons. Thus, many endoscopic image mosaicking algorithms [7,18,19] just omit the blending process and overwrite concurrent pixel values directly by the current image.…”

Section: Blending Methodsmentioning

confidence: 99%

“…At this point, the overview images must exhibit a sufficient high quality to preserve fine vessel structures and illumination information during the interpolation of overlapping image regions. In white light endoscopy, image mosaics showing seamless overlaps are often provided without any interpolation step [7,[18][19][20], or using only a basic linear blending technique [26]. In the case of PDD illumination, an adequate interpolation method becomes more complex, because the fluorescence intensity can strongly vary across the whole image sequence.…”

Section: Introductionmentioning

confidence: 99%

A non-linear multi-scale blending algorithm for fluorescence bladder images

et al. 2010

View full text Add to dashboard Cite

The composition of panoramic images of the internal urinary bladder wall from single endoscope images can strongly support the off-line documentation and surgery planning for cancer treatment, as well as assist the reidentification of multi-focal tumors during a cystoscopy. Unlike white light endoscopy, fluorescence techniques such as photodynamic diagnostics (PDD) lower the risk of missing flat and small tumors due to an enhanced tissue contrast. As a result of the low illumination power and the free hand movement of the endoscope, PDD video sequences usually show strong variations in illumination and resolution. During the subsequent mosaicking and blending process, these effects impede the preservation of original and unbiased input image information in the composed panoramic overview image, and make it difficult to avoid visual interpolation artifacts. Thus, a non-linear intensity based multi-scale blending method for fluorescence images is developed. Based on a highest intensity decision, a region mask modeling the endoscopic illumination characteristic is used to weight the input images on several sub-bands of a Laplacian pyramid. In comparison to basic linear interpolation and standard multiscale methods the new method preserves high fluorescence intensities as well as fine vessel structures in the final image composition. Furthermore the adaptive characteristics of the blending algorithm allow the physician to move the endoscope more freely along the bladder wall during the image mosaicking process.A. Behrens ( ) · M. Guski · T. Stehle · S. Gross · T. Aach

show abstract

“…Since complex blending algorithms often result in a computational overload, other endoscopic image mosaicking algorithms, like [8,9,27,28] omit the blending process and overwrite concurrent pixel values directly by the current image. Using white light illumination and assuming a homogeneous illumination of the endoscopic image, artificial edge artifacts at the image borders are often negligible.…”

Section: Blendingmentioning

confidence: 99%

“…In a similar approach Seshamni et al [34] apply motion vectors for retinal image registration and catadioptric imaging using cylindrical projection. An endoscopic mosaicking approach based on the Kanade-LucasTomasi (KLT) tracking algorithm [41] and corner-like features is developed by Bergen et al [9]. Using a simultaneous localization and mapping (SLAM) approach, Mountney et al [30] provide a dynamic view expansion and a 3D texture model of the MIS environment.…”

Section: Introductionmentioning

confidence: 99%

Real-time image composition of bladder mosaics in fluorescence endoscopy

et al. 2010

View full text Add to dashboard Cite

Today, photodynamic diagnostics is commonly used in endoscopic intervention of the urinary bladder. Excited by a narrow band illumination, fluorescence markers enhance the visual contrast between benign and malignant tissue. Since in this modality the endoscope must be moved close to the bladder wall to provide sufficiently exposed images, the field of view (FOV) of the endoscope is very limited. This impedes the navigation and the re-identifying of multi-focal tumors for the physician. Thus, an image providing a larger FOV, composed from single images is highly desired during the intervention for surgery assistance. Since endoscopic mosaicking in real-time is still an open issue, we introduce a new feature-based image mosaicking algorithm for fluorescence endoscopy. Using a multi-threaded software design, the extraction of SURF features, the matching and the image stitching are separated in single processing threads. In an optimization step we discuss the tradeoff between feature repeatability and processing time. After adjusting an optimal thread synchronization, the optimal workload of each thread results in a fast and real-time capable computation of image mosaics. On a standard hardware platform our algorithm performs within the RealTimeFrame framework with an update rate of 8.17 frames per second at full input image resolution (780 × 576). Providing a fast A. Behrens ( ) · M. Bommes · T. Stehle · S. Gross · T. Aach growing image with an extended FOV during the intervention, the mosaic displayed on a second monitor promises high potential for surgery assistance.

show abstract

Feature-based real-time endoscopic mosaicking

Cited by 15 publications

References 24 publications

Stitching and Surface Reconstruction From Endoscopic Image Sequences: A Review of Applications and Methods

Stitching and Surface Reconstruction From Endoscopic Image Sequences: A Review of Applications and Methods

A non-linear multi-scale blending algorithm for fluorescence bladder images

Real-time image composition of bladder mosaics in fluorescence endoscopy

Contact Info

Product

Resources

About