The human machine interface implementation for the robot assisted endoscopic surgery system

Zhang, S.H.; Wang, D.X.; Zhang, Y.R.; Wang, Y.H.; Wang, Y.G.; Ma, Xiaojun

doi:10.1109/roman.2002.1045662

Cited by 13 publications

(9 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The system acts under the surgeon's decisions that are communicated to the robot through gestures of the surgical tools (5) acquired with the 3D tracking system (3), and the voice commands (4) interpreted by a voice recognizer (4). The images inside the abdomen appear in the screen (6). The 3D tracker system measures the markers attached to the surgeon's tools.…”

Section: Robot Architecture and Hmi Descriptionmentioning

confidence: 99%

Subspace-oriented energy distribution for the Time Domain Passivity Approach

Ott¹,

Artigas²,

Preusche³

2011

2011 IEEE/RSJ International Conference on Intelligent Robots and Systems

View full text Add to dashboard Cite

The main objective of this paper is to minimize the occluded areas in order to recognize the navigation of the surgeon's tools for a two-arm autonomous robotic system for laparoscopic procedures. This robotic assistant needs the tracking of the surgeon's surgical gestures in order to recognize the current maneuver and to execute the automated tasks of the robot. The surgical tools pose estimation is carried out by a Multiple Extended Kalman Filter (MEKF), where the movement models of the surgical tools depend on the maneuver which is being developed. This information is obtained by a maneuvers recognition system which is a part of the multimodal human machine interface (HMI) of the robot. The method proposed for reducing shadows has been applied to three invitro maneuvers which appear in the majority of the surgical protocols. The experiments show the behavior of this method for different time intervals of the occlusions.

show abstract

Section: Robot Architecture and Hmi Descriptionmentioning

confidence: 99%

Subspace-oriented energy distribution for the Time Domain Passivity Approach

Ott¹,

Artigas²,

Preusche³

2011

2011 IEEE/RSJ International Conference on Intelligent Robots and Systems

View full text Add to dashboard Cite

show abstract

“…Nowadays, most of the existing HMI methods rely on the operator's hands. Zhang et al adopted a commercially available, standard joystick as the candidate HMI device of the robot 5 ; Speers et al achieved the goal of controlling Robot Operating System-enabled robots by sending commands from the lightweight tablet 6 ; Ochiai et al proposed a remote control system to achieve the autonomy of multiple mobile robots using touch panel interface. 7 However, with the technology development demands, the operation tasks have become more and more complicated.…”

Section: Introductionmentioning

confidence: 99%

Eliminating drift of the head gesture reference to enhance Google Glass-based control of an NAO humanoid robot

Mao

Xiao

Song

et al. 2017

International Journal of Advanced Robotic Systems

View full text Add to dashboard Cite

This article presents a strategy for hand-free control of an NAO humanoid robot via head gesture detected by Google Glassbased multi-sensor fusion. First, we introduce a Google Glass-based robot system by integrating the Google Glass and the NAO humanoid robot, which is able to send robot commands through Wi-Fi communications between the Google Glass and the robot. Second, we detect the operator's head gestures by processing data from multiple sensors including accelerometers, geomagnetic sensors and gyroscopes. Next, we use a complementary filter to eliminate drift of the head gesture reference, which greatly improves the control performance. This is accomplished by the high-pass filter component on the control signal. Finally, we conduct obstacle avoidance experiments while navigating the robot to validate the effectiveness and reliability of this system. The experimental results show that the robot is smoothly navigated from its initial position to its destination with obstacle avoidance via the Google Glass. This hands-free control system can benefit those with paralysed limbs.

show abstract

“…Por lo tanto se plantean nuevos retos ligados al interfaz de usuario que gestiona la comunicación entre el cirujano y el sistema. Algunas de las soluciones que se han adoptado para resolver dichos retos se basan en utilizar palancas de mando (joystick), en el seguimiento por visión artificial de la pinza (Casals et al,1995, Zhang et al, 2002 o cabeza del cirujano (Nishikawa et al, 2003), el uso de un giróscopo acoplado a la cabeza del cirujano (Finlay and Ornstein, 1995), el guiado mediante la voz (Sackier et al,1997, Muñoz et al, 2000, Fernández et al 2002, Muñoz et al, 2001, Gomez et al, 2009, mediante los ojos del cirujano (Noonan et al, 2008) o los gestos de su rostro (Nishikawa et al, 2003).…”

Section: Introductionunclassified