Minimally invasive medicine has become mainstream because of its crucial clinical significance in providing a low risk of postoperative complications, limited blood loss, short postoperative recovery time, and small sizes of associated physiological tissue wounds. Endoscopic navigation systems comprise a research hot spot in medical science and technology and are an essential means to achieve precision medicine and improve surgical operation safety. As a core component in endoscopic navigation during minimally invasive surgery, endoscopes play a critical role in disease diagnosis and treatment. The development of endoscopic vision technologies has resulted in a renewed drive to further develop endoscopic navigation systems. Multiple endoscopic optical imaging modalities provide data sources for endoscopic vision technology, including white-light endoscopy, contrast-enhanced imaging and technologies involving magnified observation. Endoscopic vision is a specific application of computer vision involving the use of endoscopes that include instrument tracking, endoscopic view expansion, and suspicious lesion tracking in the application of endoscopic navigation. These techniques help surgeons or surgical robots locate instruments and lesions and expand the field of view of the endoscope. Although these technologies have been applied to various clinical and pre-clinical diagnoses and treatments, the use and combination of these advanced technologies in endoscopic navigation system for specific clinical requirements remains challenging. This review performs a broad survey of advanced endoscopic vision technologies and their application in endoscopic navigation systems. Finally, we discuss the challenges and future directions in implementing and developing endoscopic navigation systems.

The camera is the main sensor of vison-based human activity recognition, and its high-precision calibration of distortion is an important prerequisite of the task. Current studies have shown that multi-parameter model methods achieve higher accuracy than traditional methods in the process of camera calibration. However, these methods need hundreds or even thousands of images to optimize the camera model, which limits their practical use. Here, we propose a novel point-to-point camera distortion calibration method that requires only dozens of images to get a dense distortion rectification map. We have designed an objective function based on deformation between the original images and the projection of reference images, which can eliminate the effect of distortion when optimizing camera parameters. Dense features between the original images and the projection of the reference images are calculated by digital image correlation (DIC). Experiments indicate that our method obtains a comparable result with the multi-parameter model method using a large number of pictures, and contributes a 28.5% improvement to the reprojection error over the polynomial distortion model.

Gastric disease is a major health problem worldwide. Gastroscopy is the main method and the gold standard used to screen and diagnose many gastric diseases. However, several factors, such as the experience and fatigue of endoscopists, limit its performance. With recent advancements in deep learning, an increasing number of studies have used this technology to provide on-site assistance during real-time gastroscopy. This review summarizes the latest publications on deep learning applications in overcoming disease-related and nondisease-related gastroscopy challenges. The former aims to help endoscopists find lesions and characterize them when they appear in the view shed of the gastroscope. The purpose of the latter is to avoid missing lesions due to poor-quality frames, incomplete inspection coverage of gastroscopy, etc., thus improving the quality of gastroscopy. This study aims to provide technical guidance and a comprehensive perspective for physicians to understand deep learning technology in gastroscopy. Some key issues to be handled before the clinical application of deep learning technology and the future direction of disease-related and nondisease-related applications of deep learning to gastroscopy are discussed herein.

Background During flexible ureteroscopy (FURS), surgeons may lose orientation due to intrarenal structural similarities and complex shape of the pyelocaliceal cavity. Decision‐making required after initially misjudging stone size will also increase the operative time and risk of severe complications. Methods A intraoperative navigation system based on electromagnetic tracking (EMT) and simultaneous localization and mapping (SLAM) was proposed to track the tip of the ureteroscope and reconstruct a dense intrarenal three‐dimensional (3D) map. Furthermore, the contour lines of stones were segmented to measure the size. Results Our system was evaluated on a kidney phantom, achieving an absolute trajectory accuracy root mean square error (RMSE) of 0.6 mm. The median error of the longitudinal and transversal measurements was 0.061 and 0.074 mm, respectively. The in vivo experiment also demonstrated the effectiveness. Conclusion The proposed system worked effectively in tracking and measurement. Further, this system can be extended to other surgical applications involving cavities, branches and intelligent robotic surgery.

No abstract