Dynamic Gesture Recognition Using a Smart Glove in Hand-Assisted Laparoscopic Surgery

Abstract

This paper presents a methodology for movement recognition in hand-assisted laparoscopic surgery using a textile-based sensing glove. The aim is to recognize the commands given by the surgeon's hand inside the patient's abdominal cavity in order to guide a collaborative robot. The glove, which incorporates piezoresistive sensors, continuously captures the degree of flexion of the surgeon's fingers. These data are analyzed throughout the surgical operation using an algorithm that detects and recognizes some defined movements as commands for the collaborative robot. However, hand movement recognition is not an easy task, because of the high variability in the motion patterns of different people and situations. The data detected by the sensing glove are analyzed using the following methodology. First, the patterns of the different selected movements are defined. Then, the parameters of the movements for each person are extracted. The parameters concerning bending speed and execution time of the movements are modeled in a prephase, in which all of the necessary information is extracted for subsequent detection during the execution of the motion. The results obtained with 10 different volunteers show a high degree of precision and recall.

Advantage of Steerable Catheter and Haptic Feedback for a 5-DOF Vascular Intervention Robot System

“…Many future studies related to this research can be performed. In recent studies, various sensors and artificial intelligence were applied to surgical robots [24][25][26]. In the present study, force measurements using the FT sensor were performed only on the catheters.…”

Natural Hand Gesture Recognition with an Electronic Textile Goniometer

“…Gesture recognition allows to distinguish specific user motions that intend to convey messages and can be used in a myriad of applications such as human-computer interface (HCI), (1)(2)(3)(4)(5)(6)(7)(8)(9)(10)(11) medicine applications including rehabilitation, (12)(13)(14) and personal identification. (15) Gesture recognition mainly consists of segmenting the acquired signals corresponding to gestures and extracting features that allow gesture classification.…”

“…(7) Electronic textiles (e-textiles) that can be embedded in clothing are being increasingly applied for gesture recognition. (9)(10)(11)(12)(13)(14)(15) The electrical properties (e.g., resistance) of flexible e-textiles enable gesture recognition given their variation with bending, stretching, and shape. Moreover, e-textiles provide greater wearing comfort and flexibility than sensors implemented on solid electronics.…”

“…This sensor conforms a new concept for goniometers, and Santos et al used a doublelayered goniometer to recognize hand movements for supporting laparoscopic surgery using robots. (14) The authors used rule-based classification to heuristically analyze the sensor patterns for recognition.…”

“…The sensor used has a double-layered structure to exhibit the above-mentioned complementary resistance patterns. Although various recognition methods, such as dynamic time warping (DTW), (2,4) a genetic algorithm, (11) a hidden Markov model, (14) and rule-based classification, (7,13) can be used with this sensor, we selected DTW given its high performance obtained from simple computations on dynamic signals. We validated the functional feasibility of the proposed approach from the data of 10 subjects performing six HCI gestures.…”