Scenario-Based Programming of Voice-Controlled Medical Robotic Systems

Abstract

An important issue in medical robotics is communication between physicians and robots. Speech-based communication is of particular advantage in robot-assisted surgery. It frees the surgeon’s hands; hence, he can focus on the principal tasks. Man-machine voice communication is the subject of research in various domains (industry, social robotics), but medical robots are very specific. They must precisely synchronize their activities with operators. Voice commands must be possibly short. They must be executed without significant delays. An important factor is the use of a vision system that provides visual information in direct synchronization with surgeon actions. Its functions could be also controlled using speech. The aim of the research presented in this paper was to develop a method facilitating creation of voice-controlled medical robotic systems, fulfilling the mentioned requirements and taking into account possible scenarios of man-machine collaboration in such systems. A robot skill description (RSD) format was proposed in order to facilitate programming of voice control applications. A sample application was developed, and experiments were conducted in order to draw conclusions regarding the usefulness of speech-based interfaces in medical robotics. The results show that a reasonable selection of system functions controlled by voice may lead to significant improvement of man-machine collaboration.

Recognition of Human Actions through Speech or Voice Using Machine Learning Techniques

“…Although the model's overall accuracy is high, there is room for improvement. Research such as [20] specified that this type of method will facilitate the creation of voicecontrolled systems, fulfilling the essential requirements of human beings and taking into account the possible scenarios of human-machine collaboration, achieving a significant improvement. Also, Kumar et al [35] described that the proposed algorithms offer better performance during their operation than existing technologies, suggesting the support vector machine algorithm.…”

“…The results show that the combination of human-computer interaction and deep learning is deeply applied in gesture recognition, speech recognition, emotion recognition, and intelligent robot steering. On the other hand, approaches based on machine learning techniques and their integration into robotics have shown it is possible to generate scenario-based programming of voice-controlled medical robotic systems, such as [20] that reveals that human-machine voice communication is the subject of research in various fields (industry, social robotics). Having reasonable results on voice-controlled system functions can lead to a significant improvement in human-machine collaboration.…”

Design and Validation of a Low-Level Controller for Hierarchically Controlled Exoskeletons

“…While exoskeletons provide direct contact on a human for rehabilitation and assistance, other medical devices with similar hardware can assist in a variety of other tasks. These medical devices include teleoperation for robotic surgery with haptic feedback [ 26 ], assistive rehabilitation [ 27 ], emotional and mental care [ 28 ], robot-assisted surgery with speech-based communication [ 29 ], and even training simulators for experienced physicians in assessing and evaluating ankle clonus, a set of involuntary and rhythmic muscle spasms [ 30 ]. Additionally, humanoid robots are commonly driven using similar sensor feedback and actuation approaches to exoskeletons and other medical devices.…”