Novel experimental and software methods for image reconstruction and localization in capsule endoscopy

Abstract

Background and study aims Capsule endoscopy (CE) is invaluable for minimally invasive endoscopy of the gastrointestinal tract; however, several technological limitations remain including lack of reliable lesion localization. We present an approach to 3D reconstruction and localization using visual information from 2D CE images. Patients and methods Colored thumbtacks were secured in rows to the internal wall of a LifeLike bowel model. A PillCam SB3 was calibrated and navigated linearly through the lumen by a high-precision robotic arm. The motion estimation algorithm used data (light falling on the object, fraction of reflected light and surface geometry) from 2D CE images in the video sequence to achieve 3D reconstruction of the bowel model at various frames. The ORB-SLAM technique was used for 3D reconstruction and CE localization within the reconstructed model. This algorithm compared pairs of points between images for reconstruction and localization. Results As the capsule moved through the model bowel 42 to 66 video frames were obtained per pass. Mean absolute error in the estimated distance travelled by the CE was 4.1 ± 3.9 cm. Our algorithm was able to reconstruct the cylindrical shape of the model bowel with details of the attached thumbtacks. ORB-SLAM successfully reconstructed the bowel wall from simultaneous frames of the CE video. The “track” in the reconstruction corresponded well with the linear forwards-backwards movement of the capsule through the model lumen. Conclusion The reconstruction methods, detailed above, were able to achieve good quality reconstruction of the bowel model and localization of the capsule trajectory using information from the CE video and images alone.

A New Active Locomotion Capsule Endoscopy under Magnetic Control and Automated Reading Program

“…53 The most widely used CEs in the world, Pillcam SB3 (Medtronic Co., Ltd., Minneapolis, MN, USA) and Mirocam (IntroMedic), have a resolution of 320 ×320 pixels, while first-generation and second-generation MCE (Ankon Technology) have a resolution of 480 × 480 pixels and 720 × 720 pixels, respectively. 31,35,54,55 However, these CEs still have lower image resolutions than standard or high-definition wired endoscopies (400,000-1,000,000 pixels). 56 CE cannot achieve a higher resolution due to the limitations of its wireless design and small size.…”

Endoscopic ultrasonography and small‐bowel endoscopy: Present and future

“…Koulaouzidis et al . have recently proposed a 3‐D reconstruction model to precisely localize the distance traveled by CE through a 30‐cm bowel model with a mean absolute error of 4.1 ± 3.9 cm . The consensus process on nomenclature and descriptions of the most frequent SB lesions is an evolving research field .…”

“…167 Koulaouzidis et al have recently proposed a 3-D reconstruction model to precisely localize the distance traveled by CE through a 30-cm bowel model with a mean absolute error of 4.1 AE 3.9 cm. 168 The consensus process on nomenclature and descriptions of the most frequent SB lesions is an evolving research field. 169 During the next decade, one can expect that capsule reading will be automatically carried out by integrating multiple software programs into a cloud-based machine-learning scientific platform allowing remote computer services to learn from past experiences and/or endoscopic databases.…”

Gastrointestinal diagnosis using non-white light imaging capsule endoscopy

“…Internal localization of CE is performed mainly through optical imaging techniques, such as lumen reconstruction-8 based methodologies using sparse or dense depth-reconstruction techniques 185 or through structured light 3D 9 scanners 186 . It is used primarily to aid navigation of CE, perform direct intervention on diseases and retarget 10 pathological sites for subsequent treatment or follow up.…”

“…It is used primarily to aid navigation of CE, perform direct intervention on diseases and retarget 10 pathological sites for subsequent treatment or follow up. Internal localization is often combined with external 11 localization for computer-aided active capsule locomotion within laboratory prototypes 185,187 A detailed discussion of 12 CE locomotion is available elsewhere 185 . 13…”