Magnetic- Visual Sensor Fusion-based Dense 3D Reconstruction and Localization for Endoscopic Capsule Robots

Turan, Mehmet; Almalioglu, Yasin; Örnek, Evin Pınar; Araújo, Hélder; Yanik, Mehmet Fatih; Sitti, Metin

doi:10.1109/iros.2018.8594485

Cited by 12 publications

(6 citation statements)

References 45 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…• Hybrid Tracking Some methods combine external tracking sensors and visual information. Turan et al [59] proposed a non-rigid and deformable RGB depth fusion method, which combines magnetic positioning and RGB images to provide excellent, precise tracking results for endoscopic capsule robots. Charreyron et al [60] integrated magnetic kinematic information into a bundle adjustment procedure as a regularization term to penalize deviations between camera poses.…”

Section: Figure 5 Example Of Hand-eye Calibrationmentioning

confidence: 99%

The Future of Endoscopic Navigation: A Review of Advanced Endoscopic Vision Technology

Jin

Zhang

et al. 2021

IEEE Access

View full text Add to dashboard Cite

Minimally invasive medicine has become mainstream because of its crucial clinical significance in providing a low risk of postoperative complications, limited blood loss, short postoperative recovery time, and small sizes of associated physiological tissue wounds. Endoscopic navigation systems comprise a research hot spot in medical science and technology and are an essential means to achieve precision medicine and improve surgical operation safety. As a core component in endoscopic navigation during minimally invasive surgery, endoscopes play a critical role in disease diagnosis and treatment. The development of endoscopic vision technologies has resulted in a renewed drive to further develop endoscopic navigation systems. Multiple endoscopic optical imaging modalities provide data sources for endoscopic vision technology, including white-light endoscopy, contrast-enhanced imaging and technologies involving magnified observation. Endoscopic vision is a specific application of computer vision involving the use of endoscopes that include instrument tracking, endoscopic view expansion, and suspicious lesion tracking in the application of endoscopic navigation. These techniques help surgeons or surgical robots locate instruments and lesions and expand the field of view of the endoscope. Although these technologies have been applied to various clinical and pre-clinical diagnoses and treatments, the use and combination of these advanced technologies in endoscopic navigation system for specific clinical requirements remains challenging. This review performs a broad survey of advanced endoscopic vision technologies and their application in endoscopic navigation systems. Finally, we discuss the challenges and future directions in implementing and developing endoscopic navigation systems.

show abstract

Section: Figure 5 Example Of Hand-eye Calibrationmentioning

confidence: 99%

The Future of Endoscopic Navigation: A Review of Advanced Endoscopic Vision Technology

Jin

Zhang

et al. 2021

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Growth of minimally invasive surgical techniques require scaled-down instrument design. However, the reduction in size has a noticeable impact on the surgeons' dexterity [87]. A possible solution for overcoming scaling issues are microelectromechanical systems (MEMS) pop-up devices [88].…”

Section: Tissue Manipulatorsmentioning

confidence: 99%

The Role of Soft Robotic Micromachines in the Future of Medical Devices and Personalized Medicine

et al. 2021

View full text Add to dashboard Cite

Developments in medical device design result in advances in wearable technologies, minimally invasive surgical techniques, and patient-specific approaches to medicine. In this review, we analyze the trajectory of biomedical and engineering approaches to soft robotics for healthcare applications. We review current literature across spatial scales and biocompatibility, focusing on engineering done at the biotic-abiotic interface. From traditional techniques for robot design to advances in tunable material chemistry, we look broadly at the field for opportunities to advance healthcare solutions in the future. We present an extracellular matrix-based robotic actuator and propose how biomaterials and proteins may influence the future of medical device design.

show abstract

“…Deep VO techniques involve 6-DoF camera pose estimation, depth map reconstruction, object segmentation, optical flow extraction, and sensor fusion approaches [5], [9], [23]- [29]. Deep 6-DoF pose regression from raw RGB images was first proposed using CNNs, which was also extended to raw RGB-D images for challenging environments [30].…”

Section: Related Workmentioning

confidence: 99%

GANVO: Unsupervised Deep Monocular Visual Odometry and Depth Estimation with Generative Adversarial Networks

Almalioglu

Saputra

Gusmão

et al. 2019

2019 International Conference on Robotics and Automation (ICRA)

Self Cite

141

View full text Add to dashboard Cite

In the last decade, supervised deep learning approaches have been extensively employed in visual odometry (VO) applications, which is not feasible in environments where labelled data is not abundant. On the other hand, unsupervised deep learning approaches for localization and mapping in unknown environments from unlabelled data have received comparatively less attention in VO research. In this study, we propose a generative unsupervised learning framework that predicts 6-DoF pose camera motion and monocular depth map of the scene from unlabelled RGB image sequences, using deep convolutional Generative Adversarial Networks (GANs). We create a supervisory signal by warping view sequences and assigning the re-projection minimization to the objective loss function that is adopted in multi-view pose estimation and single-view depth generation network. Detailed quantitative and qualitative evaluations of the proposed framework on the KITTI [1] and Cityscapes [2] datasets show that the proposed method outperforms both existing traditional and unsupervised deep VO methods providing better results for both pose estimation and depth recovery.

show abstract

Magnetic- Visual Sensor Fusion-based Dense 3D Reconstruction and Localization for Endoscopic Capsule Robots

Cited by 12 publications

References 45 publications

The Future of Endoscopic Navigation: A Review of Advanced Endoscopic Vision Technology

The Future of Endoscopic Navigation: A Review of Advanced Endoscopic Vision Technology

The Role of Soft Robotic Micromachines in the Future of Medical Devices and Personalized Medicine

GANVO: Unsupervised Deep Monocular Visual Odometry and Depth Estimation with Generative Adversarial Networks

Contact Info

Product

Resources

About