Towards Fully Autonomous Ultrasound Scanning Robot With Imitation Learning Based on Clinical Protocols

Huang, Yulin; Wei, Xiao; Wang, Chuyang; Liu, Hengli; Huang, Rui; Sun, Zhenglong

doi:10.1109/lra.2021.3064283

Cited by 38 publications

(15 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For Vessel segmentation, we use a U-Net ( Ronneberger et al, 2015 ), in line with previous works such as Jiang et al (2021) and Xiao and Wang (2021) . The network is trained with manually collected and labeled data from two anatomically different US vessel phantoms.…”

Section: Methodsmentioning

confidence: 99%

“…An extensive review on robotic systems for manipulation of extracorporeal US may be found in Elek et al (2017) . Research work includes both systems for teleoperation, often integrating force feedback ( Pierrot et al, 1999 ), visual servoing methods ( Abolmaesumi et al, 2002 ; Royer et al, 2015 ), as well as more recently the creation of autonomous systems in Xiao and Wang (2021) and Jiang et al (2021) . In the context of improving robotically assisted surgery, prior works mostly use extracorporeal ( Mathur et al, 2019 ) or endolumninal probes (inserted through natural orifices), particularly Transrectal Ultrasound (TRUS), controlled via an external robot ( Hung et al, 2012 ; Mohareri et al, 2012 ) to visually track robotic instruments and target regions.…”

Section: Introductionmentioning

confidence: 99%

“…In this work, we follow a similar approach, based, however, on a simplified, easily scalable, data-driven algorithm to estimate the local tissue coupling quality. Building upon the works of Jiang et al (2021) and Xiao and Wang (2021) , we extend the idea of autonomous US scanning along anatomical features to a minimally invasive surgical robot and intracorporeal US As a core part of this endeavour, we present and evaluate a method for fusingg Inertial Measurement Unit (IMU) data with robot kinematics. In doing so, we improve the robot’s localization in 3D space and enable a more precise and reliable anatomic reconstruction from the US scans, despite the robot’s kinematic inaccuracies and without the need for restricting tracking systems.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Towards Autonomous Robotic Minimally Invasive Ultrasound Scanning and Vessel Reconstruction on Non-Planar Surfaces

et al. 2022

View full text Add to dashboard Cite

Autonomous robotic Ultrasound (US) scanning has been the subject of research for more than 2 decades. However, little work has been done to apply this concept into a minimally invasive setting, in which accurate force sensing is generally not available and robot kinematics are unreliable due to the tendon-driven, compliant robot structure. As a result, the adequate orientation of the probe towards the tissue surface remains unknown and the anatomy reconstructed from scan may become highly inaccurate. In this work we present solutions to both of these challenges: an attitude sensor fusion scheme for improved kinematic sensing and a visual, deep learning based algorithm to establish and maintain contact between the organ surface and the US probe. We further introduce a novel scheme to estimate and orient the probe perpendicular to the center line of a vascular structure. Our approach enables, for the first time, to autonomously scan across a non-planar surface and navigate along an anatomical structure with a robotically guided minimally invasive US probe. Our experiments on a vessel phantom with a convex surface confirm a significant improvement of the reconstructed curved vessel geometry, with our approach strongly reducing the mean positional error and variance. In the future, our approach could help identify vascular structures more effectively and help pave the way towards semi-autonomous assistance during partial hepatectomy and the potential to reduce procedure length and complication rates.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Towards Autonomous Robotic Minimally Invasive Ultrasound Scanning and Vessel Reconstruction on Non-Planar Surfaces

et al. 2022

View full text Add to dashboard Cite

show abstract

“…The B-mode images are segmented by a U-Net and further feed to a ring buffer to optimize the local vessel centerline. Besides, Huang et al imitated clinical protocols to automatically position a probe in the longitudinal direction of the carotid [24]. Both of the aforementioned methods require an initial positioning of the probe to the position that can visualize the target vessel.…”

Section: B Automatic Robotic Us Acquisitionmentioning

confidence: 99%

Towards Autonomous Atlas-Based Ultrasound Acquisitions in Presence of Articulated Motion

Jiang

Gao

Xie

et al. 2022

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

Robotic ultrasound (US) imaging aims at overcoming some of the limitations of free-hand US examinations, e.g. difficulty in guaranteeing intra-and inter-operator repeatability. However, due to anatomical and physiological variations between patients and relative movement of anatomical substructures, it is challenging to robustly generate optimal trajectories to examine the anatomies of interest, in particular, when they comprise articulated joints. To address this challenge, this paper proposes a vision-based approach allowing autonomous robotic US limb scanning. To this end, an atlas MRI template of a human arm with annotated vascular structures is used to generate trajectories and register and project them onto patients' skin surfaces for robotic US acquisition. To effectively segment and accurately reconstruct the targeted 3D vessel, we make use of spatial continuity in consecutive US frames by incorporating channel attention modules into a U-Net-type neural network. The automatic trajectory generation method is evaluated on six volunteers with various articulated joint angles. In all cases, the system can successfully acquire the planned vascular structure on volunteers' limbs. For one volunteer the MRI scan was also available, which allows the evaluation of the average radius of the scanned artery from US images, resulting in a radius estimation (1.2±0.05 mm) comparable to the MRI ground truth (1.2 ± 0.04 mm).

show abstract

“…Particularly, there have been numerous studies on the development of tele‐operated robotic ultra‐sonography systems 13–21 . For fully autonomous robotic ultrasonography, several studies proposed an optimization of the scan trajectory based on the visual servo, real‐time 3D reconstruction with the scanned images, and reproduction of the expert's operation in those days 22–28 . However, to the best of our knowledge, only a few studies have focussed on robot‐assisted auscultation.…”

Section: Introductionmentioning

confidence: 99%

Towards fully automated robotic platform for remote auscultation

Tsumura

Koseki

Nitta

et al. 2022

Robotics Computer Surgery

View full text Add to dashboard Cite

Background: Since most developed countries are facing an increase in the number of patients per healthcare worker due to a declining birth rate and an ageing population, relatively simple and safe diagnosis tasks may need to be performed using robotics and automation technologies, without specialists and hospitals. This study presents an automated robotic platform for remote auscultation, which is a highly cost-effective screening tool for detecting abnormal clinical signs. Method:The developed robotic platform is composed of a 6-degree-of-freedom cooperative robotic arm, LiDAR camera, and a spring-based mechanism holding an electric stethoscope. The platform enables autonomous stethoscope positioning based on external body information acquired using the LiDAR camera-based multiway registration; the platform also ensures safe and flexible contact, maintaining the contact force within a certain range through the passive-actuated mechanism.Results: Our preliminary results confirm that the robotic platform enables estimation of the landing positions required for cardiac examinations based on the depth and landmark information of the body surface. It also handles the stethoscope while maintaining the contact force without relying on the push-in displacement by the robotic arm. Conclusion:The developed robotic platform enables the estimation of the landing positions and handling the stethoscope while maintaining the contact force, which promises the potential of automatic remote auscultation.

show abstract

Towards Fully Autonomous Ultrasound Scanning Robot With Imitation Learning Based on Clinical Protocols

Cited by 38 publications

References 19 publications

Towards Autonomous Robotic Minimally Invasive Ultrasound Scanning and Vessel Reconstruction on Non-Planar Surfaces

Towards Autonomous Robotic Minimally Invasive Ultrasound Scanning and Vessel Reconstruction on Non-Planar Surfaces

Towards Autonomous Atlas-Based Ultrasound Acquisitions in Presence of Articulated Motion

Towards fully automated robotic platform for remote auscultation

Contact Info

Product

Resources

About