Automating Surgical Peg Transfer: Calibration With Deep Learning Can Exceed Speed, Accuracy, and Consistency of Humans

Hwang, Mina; Ichnowski, Jeffrey; Thananjeyan, Brijen; Seita, Daniel; Paradis, Samuel; Fer, Danyal; Low, Thomas; Goldberg, Ken

doi:10.1109/tase.2022.3171795

Cited by 13 publications

(6 citation statements)

References 46 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This paper focuses on a typical peg transfer task which is widely discussed [20]. It is a structured scenario where the operator's intention refers to which target peg he/she wants to transport the sponge ring to when he picks it up.…”

Section: Target Prediction Based On Trajectorymentioning

confidence: 99%

An Adaptive Auxiliary Framework for Teleoperated Laparoscopic Surgery

Yiwei,

Haoyuan,

Yubo

et al. 2024

Preprint

View full text Add to dashboard Cite

Novice surgeons perform poorly during robotic surgical teleoperation because of poor consistency, a laparoscopic view that lacks depth information, and low proficiency. To make the novice surgeon perform close to expert operation during teleoperation, this article proposes an auxiliary framework for robotic surgery. A Bayesian statistics-based target prediction algorithm is used to realize the prediction of the operator’s intention, and the information entropy is used to measure the credibility of the predicted target. An adaptive velocity-position hybrid control method based on dynamic bubbles is proposed to realize the efficient performance of surgical tasks by seamlessly switching between slow, precise movements and fast, less precise tasks with no need for frequent grasping and repositioning. Assistance for teleoperation is achieved using virtual fixture force assistance based on task guidance constraints and posture adaptive adjustment based on the predicted target. The proposed framework has been validated on a platform for simulated laparoscopic surgery based on a sigma7 teleoperated primary device and a redundant degree-of-freedom surgical robot secondary device to evaluate its usability and effectiveness. It has also shown its superiority in various metrics in a comprehensive performance comparison with the baseline solution.

show abstract

Section: Target Prediction Based On Trajectorymentioning

confidence: 99%

An Adaptive Auxiliary Framework for Teleoperated Laparoscopic Surgery

Yiwei,

Haoyuan,

Yubo

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

“…FLS tasks are commonly used as benchmarks for robot dexterity and for the evaluation of AI algorithms. In robotics research, peg transfer is still a benchmark for robot dexterity and AI research [17] even though it is one of the easiest tasks for humans, given that the blocks are rigid and easy to visually discern. More complicated tasks, such as gauze cutting [18], are still an open problem in robotics research.…”

Section: Flsmentioning

confidence: 99%

A Multiarm Robotic Platform for Scientific Exploration: Its Design, Digital Twins, and Validation

Marinho,

Quiroz-Omaña,

Harada

2024

IEEE Robot. Automat. Mag.

View full text Add to dashboard Cite

There is a large number of robotic platforms with two or more arms targeting surgical applications. Despite that, very few groups have employed such platforms for scientific exploration. Possible applications of a multiarm platform in scientific exploration involve the study of the mechanisms of intractable diseases by using organoids, i.e., miniature human organs. The study of organoids requires the preparation of a cranial window, which is done by carefully removing an 8-mm patch of the skull of a mouse. In this article, we present the first prototype of our artificial intelligence (AI) robot science platform for scientific experimentation, its digital twins, and validation experiments under teleoperation. The experiments showcase the dexterity of the platform by performing peg transfer, gauze cutting, mock experiments using eggs, and the world's first two operator teleoperated drilling for a cranial window. The digital twins and related control software are available free for noncommercial use at https://AISciencePlatform.github.io.

show abstract

“…Even if the agent intends to grasp across the center of the cherry, for example, the execution alone demands a sub-millimeter precision. Several previous works addressed fine grasping [2,10] but are limited to static scenarios with rigid surface support. In contrast, we consider a dynamic scene where failed grasps might move the object.…”

Section: Related Workmentioning

confidence: 99%

“…For a predetermined, specific task, it is possible to invest in dedicated hardware [5,6], specialized tools [7,8,9], and elaborately designed systems [10,11] to solve the aforementioned problems. However, this research investigates a more universal solution: assuming that fine manipulation is required, inaccuracy is unavoidable and real-time reaction is necessary, can we enable dynamic fine grasping without stable support?…”

Section: Introductionmentioning

confidence: 99%

Cherry-Picking with Reinforcement Learning

Zhang¹,

Ke²,

Deshpande³

et al. 2023

Preprint

View full text Add to dashboard Cite

Grasping small objects surrounded by unstable or non-rigid material plays a crucial role in applications such as surgery, harvesting, construction, disaster recovery, and assisted feeding. This task is especially difficult when fine manipulation is required in the presence of sensor noise and perception errors; this inevitably triggers dynamic motion, which is challenging to model precisely. Circumventing the difficulty to build accurate models for contacts and dynamics, data-driven methods like reinforcement learning (RL) can optimize task performance via trial and error. Applying these methods to real robots, however, has been hindered by factors such as prohibitively high sample complexity or the high training infrastructure cost for providing resets on hardware. This work presents CherryBot, an RL system that uses chopsticks for fine manipulation that surpasses human reactiveness for some dynamic grasping tasks. By carefully designing the training paradigm and algorithm, we study how to make a real-world robot learning system sample efficient and general while reducing the human effort required for supervision. Our system shows continual improvement through 30 minutes of real-world interaction: through reactive retry, it achieves an almost 100% success rate on the demanding task of using chopsticks to grasp small objects swinging in the air. We demonstrate the reactiveness, robustness and generalizability of CherryBot to varying object shapes and dynamics (e.g., external disturbances like wind and human perturbations). Videos are available at https://goodcherrybot.github.io/

show abstract

Automating Surgical Peg Transfer: Calibration With Deep Learning Can Exceed Speed, Accuracy, and Consistency of Humans

Cited by 13 publications

References 46 publications

An Adaptive Auxiliary Framework for Teleoperated Laparoscopic Surgery

An Adaptive Auxiliary Framework for Teleoperated Laparoscopic Surgery

A Multiarm Robotic Platform for Scientific Exploration: Its Design, Digital Twins, and Validation

Cherry-Picking with Reinforcement Learning

Contact Info

Product

Resources

About