Continuous Semi-autonomous Prosthesis Control Using a Depth Sensor on the Hand

Abstract

Modern myoelectric prostheses can perform multiple functions (e.g., several grasp types and wrist rotation) but their intuitive control by the user is still an open challenge. It has been recently demonstrated that semi-autonomous control can allow the subjects to operate complex prostheses effectively; however, this approach often requires placing sensors on the user. The present study proposes a system for semi-autonomous control of a myoelectric prosthesis that requires a single depth sensor placed on the dorsal side of the hand. The system automatically pre-shapes the hand (grasp type, size, and wrist rotation) and allows the user to grasp objects of different shapes, sizes and orientations, placed individually or within cluttered scenes. The system “reacts” to the side from which the object is approached, and enables the user to target not only the whole object but also an object part. Another unique aspect of the system is that it relies on online interaction between the user and the prosthesis; the system reacts continuously on the targets that are in its focus, while the user interprets the movement of the prosthesis to adjust aiming. Experimental assessment was conducted in ten able-bodied participants to evaluate the feasibility and the impact of training on prosthesis-user interaction. The subjects used the system to grasp a set of objects individually (Phase I) and in cluttered scenarios (Phase II), while the time to accomplish the task (TAT) was used as the performance metric. In both phases, the TAT improved significantly across blocks. Some targets (objects and/or their parts) were more challenging, requiring thus significantly more time to handle, but all objects and scenes were successfully accomplished by all subjects. The assessment therefore demonstrated that the system is indeed robust and effective, and that the subjects could successfully learn how to aim with the system after a brief training. This is an important step toward the development of a self-contained semi-autonomous system convenient for clinical applications.

Excitation of natural spinal reflex loops in the sensory-motor control of hand prostheses

“…Some of these types of disturbance corrections can be achieved by endowing the prosthesis with greater intelligence and autonomy ( 53 , 54 ), including the implementation of mechanisms such as anti-slip features ( 55 , 56 ), adjusting thresholds, and gains at the software level ( 57 ). However, enhancing prosthesis autonomy to handle disturbances comes with the potential drawback of diminishing overall embodiment and reducing the user’s perception of prosthetic control ( 58 ). In our present study, we aimed to establish an interaction that targets task-related neural activity and that aligns with voluntary control, thus alleviating potential concerns inherent in automatic software implementation regarding lack of embodiment and ownership and preserving the user’s sense of control throughout the integration process.…”

A Review of Myoelectric Control for Prosthetic Hand Manipulation

“…Mouchoux et al [ 102 ] proposed a novel AR feedback semi-autonomous control scheme that not only improved the flexibility of EMG control, but also effectively improved user experience by shortening operation time and reducing muscle activity. Castro et al [ 103 ] designed a shared control scheme by placing a depth sensor on the back of the prosthetic hand, which enabled online interaction between users and the prosthetic hand. Users were responsible for aiming at the whole or part of the object, and the control system continuously responded to the aimed target.…”

i‐MYO: A multi‐grasp prosthetic hand control system based on gaze movements, augmented reality, and myoelectric signals

“…1 However, amputees still struggle to use the six grasp types to control a multi-grasp prosthetic hand. [2][3][4][5] Eye movement signals are one of the possible approaches. 6 To address this challenge, this paper proposes a hybrid eye signal and myoelectric signals (i-MYO) control scheme that merges gaze movements and augmented reality in bionics.…”

“…The natural hand can grasp objects using different grasp types, for example, six common grasp types in 1 . However, amputees still struggle to use the six grasp types to control a multi‐grasp prosthetic hand 2–5 . Eye movement signals are one of the possible approaches 6 .…”