Intuitive movement-based prosthesis control enables arm amputees to reach naturally in virtual reality

Abstract

Impressive progresses are being made in bionic limbs design and control. Yet, controlling the numerous joints of a prosthetic arm necessary to place the hand at a correct position and orientation to grasp objects remains challenging. Here, we designed an intuitive, movement-based prosthesis control that leverages natural arm coordination to predict distal joints missing in arm amputees based on proximal stump motion and knowledge of the movement goal. This control was validated on 29 participants, including 7 above-elbow amputees, who picked and placed bottles in a wide range of locations in virtual reality, with median success rates over 99% and movement times identical to those of natural movements. Remarkably, this was achieved without any prior training, indicating that this control is intuitive and instantaneously usable. It could be used for phantom limb pain management in virtual reality, or to augment reaching capabilities of invasive neural interfaces usually more focused on hand and grasp control.

3D-ARM-Gaze: a public dataset of 3D Arm Reaching Movements with Gaze information in virtual reality

“…With respect to prosthesis control, recent developments include movement-based control strategies whereby the movement of the prosthetic limb is controlled based on the motion of the residual limb and its natural coordination with the missing joints [1][2][3][4][5][6][7]. Although several public datasets exist that include arm movements which could be used to model and exploit this natural coordination [8][9][10], the overall body postures from which arm movements were produced in those datasets were generally left free or poorly controlled, and none of those datasets were designed to maximize the workspace spanned by participants with their arm (i.e.…”

“…Besides the specific goal of providing information relevant to movement-based prosthesis control [1][2][3][4][5][6][7], and in particular those missing in [6] to get closer to a real-life setting, we believe that the present dataset could be useful to a range of broader contexts.…”

“…This was achieved using an unsupervised self-organizing network (more specifically, a Growing Natural Gas (GNG) algorithm [45]), trained on all arm postures produced by each participant during the IA phase, in order to generate a new set of 200 targets that best represent those postures. This step was necessary to circumvent problems inherent to the fact that anatomical joints' limit are interdependent making some arm configurations impossible, in particular when maximal excursion at multiple joints are involved simultaneously (see [6] for more details). The resulting new set of 200 targets that best represent the reachable workspace of each participant was then used for all next phases.…”

“…Indeed, soft tissues around the biceps and the triceps are such that the upper arm sensor is not able to follow accurately the humeral rotation. To counter this, we applied the same method as designed in [6] to correct errors in humeral rotation based on additional information from the forearm sensor. Then, the virtual arm was simplified, switching from 9-DOFs to 7-DOFs to obtain the custom arm which matches an ideal anatomical arm description.…”

“…Second, the workspace was maximized by the means of a procedure which initially covers the widest possible workspace (i.e. set by the maximal range of motion of participants), and then subsequently uses a self-organizing map to best represent the space covered by participants (i.e producing natural arm movements within this space) [6]. Third, we recorded head and gaze motion in addition to movements of the trunk, shoulders, and arm joints, thereby collecting the entire kinematic chain between the object the participant is gating (and gazing) at and the hand moved to reach it.…”