Shoulder kinematics plus contextual target information enable control of multiple distal joints of a simulated prosthetic arm and hand

Abstract

Background Prosthetic restoration of reach and grasp function after a trans-humeral amputation requires control of multiple distal degrees of freedom in elbow, wrist and fingers. However, such a high level of amputation reduces the amount of available myoelectric and kinematic information from the residual limb. Methods To overcome these limits, we added contextual information about the target’s location and orientation such as can now be extracted from gaze tracking by computer vision tools. For the task of picking and placing a bottle in various positions and orientations in a 3D virtual scene, we trained artificial neural networks to predict postures of an intact subject’s elbow, forearm and wrist (4 degrees of freedom) either solely from shoulder kinematics or with additional knowledge of the movement goal. Subjects then performed the same tasks in the virtual scene with distal joints predicted from the context-aware network. Results Average movement times of 1.22s were only slightly longer than the naturally controlled movements (0.82 s). When using a kinematic-only network, movement times were much longer (2.31s) and compensatory movements from trunk and shoulder were much larger. Integrating contextual information also gave rise to motor synergies closer to natural joint coordination. Conclusions Although notable challenges remain before applying the proposed control scheme to a real-world prosthesis, our study shows that adding contextual information to command signals greatly improves prediction of distal joint angles for prosthetic control.

Multichannel haptic feedback unlocks prosthetic hand dexterity

“…This might lead to suboptimal grip forces applied to the objects, which could be ameliorated via a soft-synergy approach with mass-dependent variable gains 54 , or through a grasped object slip prevention algorithm 55 . Additionally, techniques to automate the shoulder, elbow, and wrist joints could be highly beneficial to reduce the cognitive burden on the amputee to multitask by sensing upper body compensatory motions 56 or shoulder kinematics with respect to the grasped object locations 57 to autonomously plan smooth prosthetic arm trajectories. These types of motion planning algorithms could reduce unnecessary oscillations in grip forces and corresponding haptic sensations during object transportation 58 while reflexive compensation for inertial loads during transport could be used to proactively prevent slip of the grasped objects 59 .…”

Visual Data based Throwing Estimation for Feasibility Assessment

“…Since the shoulder is the closest part of the upper limb to the trunk of the body, it may indeed be functional even in people with upper-limb disabilities. In addition, the shoulder is a part of the body that is expected to be applied as an interface for prosthetic hands, and there is a high possibility that signals useful for estimation can be read (5) .…”

Hybrid FPGA–CPU-Based Architecture for Object Recognition in Visual Servoing of Arm Prosthesis

“…Despite great progress in upper limb bionic prostheses, allowing for object-of-interest reaching and grasping, the key remaining issues relate to their control by the operator. To overcome the limitations of traditional control solely based on the electromyographic (EMG) activity of the remaining muscles, promising alternatives consider hybrid systems combining noninvasive motion capture and vision control [1,2]. They include camera vision modules that allow for recognition of the subject's intention to grasp an object and assist visual control of prosthetic arms for object reaching and grasping [3].…”

“…Real-time performance is also a mandatory requirement for our target application [2,7]. As the fastest visuomotor response to a perturbation takes about 90 ms [8], and feedback delays of 100 ms or more are known to deteriorate the performance of online feedback control [9], computation time should remain as low as possible, and below 100 ms.…”