Stitching and Surface Reconstruction From Endoscopic Image Sequences: A Review of Applications and Methods

Bergen, Tobias; Wittenberg, Thomas

doi:10.1109/jbhi.2014.2384134

Cited by 84 publications

(40 citation statements)

References 150 publications

(212 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Moreover, for robotic SILS procedures the surgical site visualization issues can be overcome by introducing a virtual environment including a dynamic model of the robot within a surgical scene recreated from pre-operative information. This virtual environment can be updated in real-time by exploiting 3D reconstruction of the soft tissues [4,22].…”

Section: Introductionmentioning

confidence: 99%

Dense soft tissue 3D reconstruction refined with super-pixel segmentation for robotic abdominal surgery

et al. 2015

View full text Add to dashboard Cite

Purpose: Single Incision Laparoscopic Surgery (SILS) decreases post-operative infections, but introduces limitations in the surgeon's manoeuverability and in the surgical field of view. This work aims at enhancing intraoperative surgical visualization by exploiting the 3D information about the surgical site. An interactive guidance system is proposed wherein the pose of pre-operative tissue models is updated online. A critical process involves the intraoperative acquisition of tissue surfaces. It can be achieved using stereoscopic imaging and 3D reconstruction techniques. This work contributes to this process by proposing new methods for improved dense 3D reconstruction of soft tissues, which allows a more accurate deformation identification and facilitates the registration process.Methods: Two methods for soft tissue 3D reconstruction are proposed: Method 1 follows the traditional approach of the block matching algorithm. Method 2 performs a non-parametric Modified Census Transform to be more robust to illumination variation. The Simple Linear Iterative Clustering (SLIC) super pixel al- gorithm is exploited for disparity refinement by filling holes in the disparity images.Results: The methods were validated using two video datasets from the Hamlyn Centre, achieving an accuracy of 2.95 mm and 1.66 mm respectively. A comparison with ground truth data demonstrated the disparity refinement procedure: (i) increases the number of reconstructed points by up to 43%; (ii) does not a↵ect the accuracy of the 3D reconstructions significantly.Conclusion: Both methods give results that compare favourably with the state-of-the-art methods. The computational time constraints their applicability in realtime, but can be greatly improved by using a GPU implementation.

show abstract

Section: Introductionmentioning

confidence: 99%

Dense soft tissue 3D reconstruction refined with super-pixel segmentation for robotic abdominal surgery

et al. 2015

View full text Add to dashboard Cite

show abstract

“…Viewport enhancement A further possible application of AR is to improve the viewing conditions of the surgeon. This can be done by expanding the restricted viewport and visualizing the surrounding area using image stitching methods [19,153,241], potentially even in 3D [265]. Similar techniques have also been investigated for the purpose of video summarization, e.g., to obtain a condensed representation of an examination for a medical record (see Section 4.2.4).…”

Section: Augmented Realitymentioning

confidence: 99%

“…Liao et al [120] extend this idea by a method to map the panorama to a three-dimensional ultrasound image. A review about recent advances in the field of endoscopic image mosaicing and panorama generation can be found in [19].…”

Section: Summarizationmentioning

confidence: 99%

Content-based processing and analysis of endoscopic images and videos: A survey

Münzer

Schoeffmann

Böszörményi

2017

Multimed Tools Appl

108

View full text Add to dashboard Cite

In recent years, digital endoscopy has established as key technology for medical screenings and minimally invasive surgery. Since then, various research communities with manifold backgrounds have picked up on the idea of processing and automatically analyzing the inherently available video signal that is produced by the endoscopic camera. Proposed works mainly include image processing techniques, pattern recognition, machine learning methods and Computer Vision algorithms. While most contributions deal with realtime assistance at procedure time, the post-procedural processing of recorded videos is still in its infancy. Many post-processing problems are based on typical Multimedia methods like indexing, retrieval, summarization and video interaction, but have only been sparsely addressed so far for this domain. The goals of this survey are (1) to introduce this research field to a broader audience in the Multimedia community to stimulate further research, (2) to describe domain-specific characteristics of endoscopic videos that need to be addressed in a pre-processing step, and (3) to systematically bring together the very diverse research results for the first time to provide a broader overview of related research that is currently not perceived as belonging together.

show abstract

“…Very recently, there have been some alternative methods, namely the use of deep Reinforcement Learning (RL), where a policy is created in order to maximize the rewards of an agent [9], [10]. On the other hand, omnidirectional cameras have been used in an increasing number of applications ranging from surveillance to robot navigation [11], [12], [13], [14], which require a The input is an omnidirectional image with an initial state of the bounding box, represented in the world coordinate system. Using this information, a set of possible actions are applied in order to detect the pedestrian in the 3D environment.…”

Section: Introductionmentioning

confidence: 99%

OmniDRL: Robust Pedestrian Detection using Deep Reinforcement Learning on Omnidirectional Cameras

Pais

Dias

Nascimento

et al. 2019

2019 International Conference on Robotics and Automation (ICRA)

View full text Add to dashboard Cite

Pedestrian detection is one of the most explored topics in computer vision and robotics. The use of deep learning methods allowed the development of new and highly competitive algorithms. Deep Reinforcement Learning has proved to be within the state-of-the-art in terms of both detection in perspective cameras and robotics applications. However, for detection in omnidirectional cameras, the literature is still scarce, mostly because of their high levels of distortion. This paper presents a novel and efficient technique for robust pedestrian detection in omnidirectional images. The proposed method uses deep Reinforcement Learning that takes advantage of the distortion in the image. By considering the 3D bounding boxes and their distorted projections into the image, our method is able to provide the pedestrian's position in the world, in contrast to the image positions provided by most state-of-the-art methods for perspective cameras. Our method avoids the need of preprocessing steps to remove the distortion, which is computationally expensive. Beyond the novel solution, our method compares favorably with the state-of-the-art methodologies that do not consider the underlying distortion for the detection task.1 Although undistorted images keep the perspective projection constraints (i.e., straight lines in the world will be projected into straight lines in the image), the objects will be stretched. This means that the objects should not be approximated by regular bounding boxes, which are used in most of the DL techniques for object detection (we note that there are alternatives that do not consider regular bounding boxes in [19]).

show abstract

Stitching and Surface Reconstruction From Endoscopic Image Sequences: A Review of Applications and Methods

Cited by 84 publications

References 150 publications

Dense soft tissue 3D reconstruction refined with super-pixel segmentation for robotic abdominal surgery

Dense soft tissue 3D reconstruction refined with super-pixel segmentation for robotic abdominal surgery

Content-based processing and analysis of endoscopic images and videos: A survey

OmniDRL: Robust Pedestrian Detection using Deep Reinforcement Learning on Omnidirectional Cameras

Contact Info

Product

Resources

About