Instance segmentation of point cloud captured by RGB-D sensor based on deep learning

Wang, Zhengtuo; Xu, Yanliang; Yu, Jiongyan; Xu, Guangyue; Fu, Jianzhong; Gu, Tianyi

doi:10.1080/0951192x.2021.1946853

Cited by 9 publications

(3 citation statements)

References 50 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Instance segmentation of point cloud is extremely important since the quality of segmentation will affect the performance of subsequent algorithms. The point cloud captured by RGB-D sensor can go over the instance segmentation process by applying the deep learning method YOLACT++ to instance segment the color image first and then matching the instance information with the point cloud [48].…”

Section: Discussionmentioning

confidence: 99%

Toward Intraoperative Visual Intelligence: Real-Time Surgical Instrument Segmentation for Enhanced Surgical Monitoring

Daneshgar Rahbar,

Pappas,

Jaber

2024

Preprint

View full text Add to dashboard Cite

A single paragraph of about 200 words maximum. For research articles, abstracts should give a pertinent overview of the work. We strongly encourage authors to use the following style of structured abstracts, but without headings: (1) Background: Place the question addressed in a broad context and highlight the purpose of the study; (2) Methods: briefly describe the main methods or treatments applied; (3) Results: summarize the article’s main findings; (4) Conclusions: indicate the main conclusions or interpretations. The abstract should be an objective representation of the article and it must not contain results that are not presented and substantiated in the main text and should not exaggerate the main conclusions.

show abstract

Section: Discussionmentioning

confidence: 99%

Toward Intraoperative Visual Intelligence: Real-Time Surgical Instrument Segmentation for Enhanced Surgical Monitoring

Daneshgar Rahbar,

Pappas,

Jaber

2024

Preprint

View full text Add to dashboard Cite

show abstract

“…The instance segmentation of point clouds is extremely important since the quality of segmentation affects the performance of subsequent algorithms. The point cloud captured by an RGB-D sensor can go through the instance segmentation process by applying the deep learning method YOLACT++ to instance segment the color image first and then matching the instance information with the point cloud [ 48 ].…”

Section: Discussionmentioning

confidence: 99%

Toward Intraoperative Visual Intelligence: Real-Time Surgical Instrument Segmentation for Enhanced Surgical Monitoring

Daneshgar Rahbar,

Pappas,

Jaber

2024

Healthcare

View full text Add to dashboard Cite

Background: Open surgery relies heavily on the surgeon’s visual acuity and spatial awareness to track instruments within a dynamic and often cluttered surgical field. Methods: This system utilizes a head-mounted depth camera to monitor surgical scenes, providing both image data and depth information. The video captured from this camera is scaled down, compressed using MPEG, and transmitted to a high-performance workstation via the RTSP (Real-Time Streaming Protocol), a reliable protocol designed for real-time media transmission. To segment surgical instruments, we utilize the enhanced U-Net with GridMask (EUGNet) for its proven effectiveness in surgical tool segmentation. Results: For rigorous validation, the system’s performance reliability and accuracy are evaluated using prerecorded RGB-D surgical videos. This work demonstrates the potential of this system to improve situational awareness, surgical efficiency, and generate data-driven insights within the operating room. In a simulated surgical environment, the system achieves a high accuracy of 85.5% in identifying and segmenting surgical instruments. Furthermore, the wireless video transmission proves reliable with a latency of 200 ms, suitable for real-time processing. Conclusions: These findings represent a promising step towards the development of assistive technologies with the potential to significantly enhance surgical practice.

show abstract

“…The find-object library in ROS is used for object recognition tasks, which includes two important nodes: find_object_2d and find_object_3d. The former is utilized for 2D object detection in camera images, while the latter requires a depth camera for object recognition and localization in 3D point clouds [9]or RGB-D images. The command to install the find_object package is "sudo apt-get install ros-melodic-find-object-2d".…”

Section: Object Detection and Recognitionmentioning

confidence: 99%

Research on simulated visual control of robotic arm in ROS environment

Qin,

Wang,

et al. 2024

Fourth International Conference on Mechanical, Electronics, and Electrical and Automation Control (METMS 2024)

View full text Add to dashboard Cite

Instance segmentation of point cloud captured by RGB-D sensor based on deep learning

Cited by 9 publications

References 50 publications

Toward Intraoperative Visual Intelligence: Real-Time Surgical Instrument Segmentation for Enhanced Surgical Monitoring

Toward Intraoperative Visual Intelligence: Real-Time Surgical Instrument Segmentation for Enhanced Surgical Monitoring

Toward Intraoperative Visual Intelligence: Real-Time Surgical Instrument Segmentation for Enhanced Surgical Monitoring

Research on simulated visual control of robotic arm in ROS environment

Contact Info

Product

Resources

About