MIS-SLAM: Real-Time Large-Scale Dense Deformable SLAM System in Minimal Invasive Surgery Based on Heterogeneous Computing

Song, Jingwei; Wang, Jun; Zhao, Liang; Huang, Shoudong; Dissanayake, Gamini

doi:10.1109/lra.2018.2856519

Cited by 107 publications

(66 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Beyond that, algorithms dealing with dynamic surfaces (e.g. Song et al, 2018;Xiao et al, 2019) need to be evaluated in more real environments and compared with implemented VO solution.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Orientation of Point Clouds for Complex Surfaces in Medical Surgery Using Trinocular Visual Odometry and Stereo Orb-Slam2

Kahmen

Haase

Lühmann

2020

Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci.

View full text Add to dashboard Cite

Abstract. In photogrammetry, computer vision and robotics, visual odometry (VO) and SLAM algorithms are well-known methods to estimate camera poses from image sequences. When dealing with unknown scenes there is often no reference data available and also the scene needs to be reconstructed for further analysis. In this contribution a trinocular visual odometry approach is implemented and compared to stereo VO and ORB-SLAM2 in an experimental setup imitating the scene of a knee replacement surgery. Two datasets are analysed. While a test-field provides excellent conditions for feature detection algorithms with its artificial texture assembled, extracted images show the knee joint itself solely in order to use only the homogenous, but in real application stable, region of the knee joint. The camera trajectories of VO and ORB-SLAM2 are transformed to corresponding coordinate systems and are subsequently evaluated. The tracking algorithms show poor quality when only the inappropriate surface of the knee is used but perform well when the artificial texture of the test-field is used. The third camera does not lead to a significant advantage in this setup using our implementation. Possible reasons, e.g. less overlap, are discussed in this contribution. Nevertheless, the quality of the oriented point clouds, obtained by trinocular dense matching, is less than 1mm for most of the analysed data. The experiment will be used to focus on further developments, e.g. dealing with specular reflections, and for evaluation purposes using different SLAM/ VO algorithms.

show abstract

“…Beyond that, algorithms dealing with dynamic surfaces (e.g. Song et al, 2018;Xiao et al, 2019) need to be evaluated in more real environments and compared with implemented VO solution.…”

Section: Discussionmentioning

confidence: 99%

“…For SLAM the scene needs to be stable between several key frames in order to perform loop closure, whereas for VO the scene needs to be stable only between single frames. As stated in Song et al (2018), loop closing is difficult in surgical vision due to deforming scenarios.…”

Section: Visual Odometry and Orb-slam2mentioning

confidence: 99%

Orientation of Point Clouds for Complex Surfaces in Medical Surgery Using Trinocular Visual Odometry and Stereo Orb-Slam2

Kahmen

Haase

Lühmann

2020

Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci.

View full text Add to dashboard Cite

show abstract

“…Yu et al (2015) solved a similar problem using as input a single RGB video (without depth). Finally, Song et al (2018) presented a real-time dense stereoscopic SLAM system, to be applied in minimal invasive surgery.…”

Section: Shape Estimation/tracking Of Deformable Objectsmentioning

confidence: 99%

Model-free vision-based shaping of deformable plastic materials

Cherubini

Ortenzi

Cosgun

et al. 2020

The International Journal of Robotics Research

View full text Add to dashboard Cite

We address the problem of shaping deformable plastic materials using non-prehensile actions. Shaping plastic objects is challenging, since they are difficult to model and to track visually. We study this problem, by using kinetic sand, a plastic toy material which mimics the physical properties of wet sand. Inspired by a pilot study where humans shape kinetic sand, we define two types of actions: pushing the material from the sides and tapping from above. The chosen actions are executed with a robotic arm using image-based visual servoing. From the current and desired view of the material, we define states based on visual features such as the outer contour shape and the pixel luminosity values. These are mapped to actions, which are repeated iteratively to reduce the image error until convergence is reached. For pushing, we propose three methods for mapping the visual state to an action. These include heuristic methods and a neural network, trained from human actions. We show that it is possible to obtain simple shapes with the kinetic sand, without explicitly modeling the material. Our approach is limited in the types of shapes it can achieve. A richer set of action types and multi-step reasoning is needed to achieve more sophisticated shapes. Figure 2. Human plastic material shaping study. Left: setup for the user study: an RGB-D camera (RealSense) is fixed above the sandbox, where participants mold the kinetic sand into a desired shape; first with one, then with two hands and third, using one of two provided tools. Center: four different shapes formed by the participants of our user study. Right: eight images (top: RGB, bottom: depth) acquired while a user is shaping the kinetic sand using both hands.

show abstract

“…Some improvements have been introduced to improve computational performance. In [7][8][9][10][11], there are some SLAM methods that use specialized computer hardware for parallel programming, for example, a graphic processing unit (GPU). The GPU is much faster than the central processing unit (CPU) for processing a large dimension of matrix arrays or parallel executions.…”

Section: Related Workmentioning

confidence: 99%

Cartographer SLAM Method for Optimization with an Adaptive Multi-Distance Scan Scheduler

et al. 2020

View full text Add to dashboard Cite

This paper presents the use of Google's simultaneous localization and mapping (SLAM) technique, namely Cartographer, and adaptive multistage distance scheduler (AMDS) to improve the processing speed. This approach optimizes the processing speed of SLAM which is known to have performance degradation as the map grows due to a larger scan matcher. In this proposed work, the adaptive method was successfully tested in an actual vehicle to map roads in real time. The AMDS performs a local pose correction by controlling the LiDAR sensor scan range and scan matcher search window with the help of scheduling algorithms. The scheduling algorithms manage the SLAM that swaps between short and long distances during map data collection. As a result, the algorithms efficiently improved performance speed similar to short distance LiDAR scans while maintaining the accuracy of the full distance of LiDAR. By swapping the scan distance of the sensor, and adaptively limiting the search size of the scan matcher to handle difference scan sizes, the pose's generation performance time is improved by approximately 16% as compared with a fixed scan distance, while maintaining similar accuracy.

show abstract

MIS-SLAM: Real-Time Large-Scale Dense Deformable SLAM System in Minimal Invasive Surgery Based on Heterogeneous Computing

Cited by 107 publications

References 23 publications

Orientation of Point Clouds for Complex Surfaces in Medical Surgery Using Trinocular Visual Odometry and Stereo Orb-Slam2

Orientation of Point Clouds for Complex Surfaces in Medical Surgery Using Trinocular Visual Odometry and Stereo Orb-Slam2

Model-free vision-based shaping of deformable plastic materials

Cartographer SLAM Method for Optimization with an Adaptive Multi-Distance Scan Scheduler

Contact Info

Product

Resources

About