User training for machine learning controlled upper limb prostheses: a serious game approach

Abstract

Background Upper limb prosthetics with multiple degrees of freedom (DoFs) are still mostly operated through the clinical standard Direct Control scheme. Machine learning control, on the other hand, allows controlling multiple DoFs although it requires separable and consistent electromyogram (EMG) patterns. Whereas user training can improve EMG pattern quality, conventional training methods might limit user potential. Training with serious games might lead to higher quality EMG patterns and better functional outcomes. In this explorative study we compare outcomes of serious game training with conventional training, and machine learning control with the users’ own one DoF prosthesis. Methods Participants with upper limb absence participated in 7 training sessions where they learned to control a 3 DoF prosthesis with two grips which was fitted. Participants received either game training or conventional training. Conventional training was based on coaching, as described in the literature. Game-based training was conducted using two games that trained EMG pattern separability and functional use. Both groups also trained functional use with the prosthesis donned. The prosthesis system was controlled using a neural network regressor. Outcome measures were EMG metrics, number of DoFs used, the spherical subset of the Southampton Hand Assessment Procedure and the Clothespin Relocation Test. Results Eight participants were recruited and four completed the study. Training did not lead to consistent improvements in EMG pattern quality or functional use, but some participants improved in some metrics. No differences were observed between the groups. Participants achieved consistently better results using their own prosthesis than the machine-learning controlled prosthesis used in this study. Conclusion Our explorative study showed in a small group of participants that serious game training seems to achieve similar results as conventional training. No consistent improvements were found in either group in terms of EMG metrics or functional use, which might be due to insufficient training. This study highlights the need for more research in user training for machine learning controlled prosthetics. In addition, this study contributes with more data comparing machine learning controlled prosthetics with Direct Controlled prosthetics.

The Need to Work Arm in Arm: Calling for Collaboration in Delivering Neuroprosthetic Limb Replacements

“…These include computer gaming, virtual reality environments, and augmented reality (Resnik et al, 2011;Winslow et al, 2018;Boschmann et al, 2021). Several studies have demonstrated successful prosthesis control with these training systems, in addition to individuals reporting increased usability and motivation (Tabor et al, 2018;Kristoffersen et al, 2021). Not only do these methods improve participant engagement, but they also provide an at-home training system that is more convenient and affordable than daily sessions with a physiotherapist.…”

Serious Games Are Not Serious Enough for Myoelectric Prosthetics

“… Heatmap of the authors’ software classification against the performed tasks: serious games using abstract tasks [ 2 , 10 - 12 , 15 - 30 ], tasks related to ADL [ 10 , 31 ], and posture reproduction tasks [ 4 , 13 , 32 , 33 ]; as well as simulators using abstract tasks [ 34 - 36 ], ADL-related tasks [ 19 , 37 - 54 ], and posture reproduction tasks [ 56 - 67 ]. ADL: activities of daily living.…”

“…However, it was rarely tested whether these myogames increase prosthetic ability or speed up the learning process of acquiring that skill. The one research group that tested for an increase in prosthetic ability did not find evidence of a significant increase following the playing of a myogame for different control schemes [ 19 , 30 ].…”

“…Serious games for prosthetic training can only be expected to yield fruitful results beyond engagement when they incorporate tasks that are found to facilitate prosthetic skill. We recommend that the research community investigates which types of myogame tasks might facilitate transfer, as the only existing results at the time of writing this paper indicate a lack of effectiveness [ 19 , 30 , 73 ]. This lack does not necessarily hold true for all tasks that are not related to activities of daily living, however, and ignoring abstract tasks entirely would exclude a wide range of possible avenues for prosthetic game development.…”

“…More long-term and ideally home-based experiments are needed to conclusively test for any prosthetic skill transfer that might occur with the consistent use of prosthetic gaming devices. Even though previous studies indicate that no change in prosthetic ability occurs after training with a myogame [ 19 , 30 , 73 ], these only tested the effect of comparatively short training sessions with able-bodied people or very small groups of prosthesis users. It should also be tested whether prosthetic gaming has the potential to support traditional prosthetic training by allowing for supplementary practice sessions between visits to the prosthetist.…”