Multimodal and Mixed Control of Robotic Endoscopes

Avellino, Ignacio; Bailly, Gilles; Arico, Mario; Morel, Guillaume; Canlorbe, Geoffroy

doi:10.1145/3313831.3376795

Cited by 9 publications

(10 citation statements)

References 66 publications

(89 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In our experiments, this human camera assistant is replaced by the Virtuose6D 5 ( Haption SA , Laval, France) robotic arm. As shown in ( Avellino et al, 2020 ), the dimensions, workspace, and supported payload of this robotic arm are well suited for robotic endoscope control 6 . The operational workspace is defined as a cube of side 450 mm and is located in the center of the workspace envelope.…”

Section: Methodsmentioning

confidence: 99%

“…In their experimental setup, the system could track either a single instrument or the midpoint between two tools. Similarly, Avellino et al (2020) have also employed kinematics for autonomous endoscope guidance in a co-manipulation scenario 1 . In ( Casals et al, 1996 ; Mudunuri, 2010 ), rule-based strategies switch the control mode between single-instrument tracking or tracking points that aggregate locations of all visible instruments.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Robotic Endoscope Control Via Autonomous Instrument Tracking

Gruijthuijsen

García-Peraza-Herrera

Borghesan

et al. 2022

Front. Robot. AI

View full text Add to dashboard Cite

Many keyhole interventions rely on bi-manual handling of surgical instruments, forcing the main surgeon to rely on a second surgeon to act as a camera assistant. In addition to the burden of excessively involving surgical staff, this may lead to reduced image stability, increased task completion time and sometimes errors due to the monotony of the task. Robotic endoscope holders, controlled by a set of basic instructions, have been proposed as an alternative, but their unnatural handling may increase the cognitive load of the (solo) surgeon, which hinders their clinical acceptance. More seamless integration in the surgical workflow would be achieved if robotic endoscope holders collaborated with the operating surgeon via semantically rich instructions that closely resemble instructions that would otherwise be issued to a human camera assistant, such as “focus on my right-hand instrument.” As a proof of concept, this paper presents a novel system that paves the way towards a synergistic interaction between surgeons and robotic endoscope holders. The proposed platform allows the surgeon to perform a bimanual coordination and navigation task, while a robotic arm autonomously performs the endoscope positioning tasks. Within our system, we propose a novel tooltip localization method based on surgical tool segmentation and a novel visual servoing approach that ensures smooth and appropriate motion of the endoscope camera. We validate our vision pipeline and run a user study of this system. The clinical relevance of the study is ensured through the use of a laparoscopic exercise validated by the European Academy of Gynaecological Surgery which involves bi-manual coordination and navigation. Successful application of our proposed system provides a promising starting point towards broader clinical adoption of robotic endoscope holders.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Robotic Endoscope Control Via Autonomous Instrument Tracking

Gruijthuijsen

García-Peraza-Herrera

Borghesan

et al. 2022

Front. Robot. AI

View full text Add to dashboard Cite

show abstract

“…Video capture (Figure 6b) is "more suitable for some steps of the projects" -p43, "all of our actions would be recorded" -p6. Interestingly, 11 of the 12 respondents who considered Capush for the activity "electronic" 6 would record videos with Capush, which 6 The survey software selected the two activities for which participants indicated capturing most frequently pictures or videos. Questions on current practices were asked for both activities, whereas exploration of Capush was centered on the Proc.…”

Section: Rq2: Exploration Of Framework Dimensions Through Capushmentioning

confidence: 99%

“…Both manual and automatic control require some form of user interface though to indicate where the camera should point or tell an automated control what to keep in the field of view. Depending on the level of indirection [8] and the desired automation, different control interfaces are possible, much of which have already been explored in previous work for both manual [25] and mixed control [6]. With our proof-of-concept prototype, we tested a basic implementation of the automatic tracking mode mentioned in the study and the video shown to respondents.…”

Section: Proof Of Concept Implementationsmentioning

confidence: 99%

Exploring Capturing Approaches in Shared Fabrication Workshops: Current Practice and Opportunities

Rigaud¹,

Bailly²,

Avellino

et al. 2022

Proc. ACM Hum.-Comput. Interact.

Self Cite

View full text Add to dashboard Cite

Capturing content of fabrication activities is the first step for producing knowledge resources and an integral part of maker culture. As a secondary task, it conflicts though with the fabrication activity, and thus it is often forgotten and knowledge resources end up incomplete. In this article, we investigate different dimensions of content capture for knowledge resources in fabrication workshops. Based on past work in this area, we first propose a framework through which we identify two research directions to investigate. From these, we derive three dimensions to explore in more depth: The number of capturing devices, their feature variety and the degree of automation of each feature. We then explore the design space resulting from these three dimensions with the help of a design concept and an online survey study (N=66). Results show (1) a variety of needs and preferences justifying feature variety and multiplicity, (2) challenges in defining the right degree of manual and automatic control, and (3) the socio-technical impact of cameras in a shared space regarding privacy and ethics. We conclude with discussions on the benefits and vulnerabilities of equipping fabrication workshop with distributed camera-based capturing devices and offer opportunities for design.

show abstract

“…Regarding the interaction technique to capture information during surgery, HCI research has explored voice and gesture-based systems [45,52] that are compatible with the sterility requirements. Touch can also be used for instance by using a sterile cover around buttons on the endoscope, although oftentimes surgeons are busy holding the surgical tools and cannot simply switch to the endoscope to press buttons [4].…”

Section: Implication: Making Video a Communicative Artifactmentioning

confidence: 99%

Surgical Video Summarization

Avellino

Nozari

Canlorbe

et al. 2021

Proc. ACM Hum.-Comput. Interact.

Self Cite

View full text Add to dashboard Cite

While surgical videos are valuable support material for activities around surgery, their summarization demands great amounts of time from surgeons, resulting in the production of very few videos. We study the practices involving surgical video to motivate and inform the future design of tools for their summarization. Through interviews and observations in a feld study, we fnd that (1) video summaries provide an important support for surgery, being used for self-improvement, education, discussing cases, scientifc research, patient communication and as legal resources; (2) video summarization follows a process hindered by the loss of knowledge that originates during recording; and, (3) surgeons develop ad-hoc coordination strategies which involve using the video itself for articulation work, making it both the feld of work and coordination artifact. We discuss ways in which tools can facilitate capturing knowledge during live action using these strategies.CCS Concepts: • Human-centered computing → Empirical studies in HCI.

show abstract

Multimodal and Mixed Control of Robotic Endoscopes

Cited by 9 publications

References 66 publications

Robotic Endoscope Control Via Autonomous Instrument Tracking

Robotic Endoscope Control Via Autonomous Instrument Tracking

Exploring Capturing Approaches in Shared Fabrication Workshops: Current Practice and Opportunities

Surgical Video Summarization

Contact Info

Product

Resources

About