Immersive augmented reality system for the training of pattern classification control with a myoelectric prosthesis

Abstract

Background Hand amputation can have a truly debilitating impact on the life of the affected person. A multifunctional myoelectric prosthesis controlled using pattern classification can be used to restore some of the lost motor abilities. However, learning to control an advanced prosthesis can be a challenging task, but virtual and augmented reality (AR) provide means to create an engaging and motivating training. Methods In this study, we present a novel training framework that integrates virtual elements within a real scene (AR) while allowing the view from the first-person perspective. The framework was evaluated in 13 able-bodied subjects and a limb-deficient person divided into intervention (IG) and control (CG) groups. The IG received training by performing simulated clothespin task and both groups conducted a pre- and posttest with a real prosthesis. When training with the AR, the subjects received visual feedback on the generated grasping force. The main outcome measure was the number of pins that were successfully transferred within 20 min (task duration), while the number of dropped and broken pins were also registered. The participants were asked to score the difficulty of the real task (posttest), fun-factor and motivation, as well as the utility of the feedback. Results The performance (median/interquartile range) consistently increased during the training sessions (4/3 to 22/4). While the results were similar for the two groups in the pretest, the performance improved in the posttest only in IG. In addition, the subjects in IG transferred significantly more pins (28/10.5 versus 14.5/11), and dropped (1/2.5 versus 3.5/2) and broke (5/3.8 versus 14.5/9) significantly fewer pins in the posttest compared to CG. The participants in IG assigned (mean ± std) significantly lower scores to the difficulty compared to CG (5.2 ± 1.9 versus 7.1 ± 0.9), and they highly rated the fun factor (8.7 ± 1.3) and usefulness of feedback (8.5 ± 1.7). Conclusion The results demonstrated that the proposed AR system allows for the transfer of skills from the simulated to the real task while providing a positive user experience. The present study demonstrates the effectiveness and flexibility of the proposed AR framework. Importantly, the developed system is open source and available for download and further development.

Exploring the Embodiment of a Virtual Hand in a Spatially Augmented Respiratory Biofeedback Setting

“…Such solutions, extremely valuable in clinical applications too (Matamala-Gomez et al, 2021), demonstrate further potential through their compatibility with other technologically advanced approaches like neuromodulation (Kannape et al, 2019). Furthermore, AR solutions are currently explored to train the control of prosthetic systems (Boschmann et al, 2021). Interestingly, the study in Monti et al (2020) adopted a VR-based RR biofeedback approach to generate and investigate an "embreathment" illusion by ecologically mapping the subjects' breaths onto a virtual body observed from a first-person perspective, improving the embodiment of the individual on the avatar.…”

Virtual reality as a main basis for forming modern educational technologies

“…But also nothing prevents to show the process itself much deeper, namely, what happens inside the atom [2]. Thus, this will provide an opportunity not only to obtain information about this phenomenon, but also to show in detail everything that happens to the student [3].…”

The Need to Work Arm in Arm: Calling for Collaboration in Delivering Neuroprosthetic Limb Replacements

“…Creating a reliable, wireless communication system with preserved signal strength is a challenging endeavor. It requires the use of appropriately packaged electronics capable of efficient and safe power management in a way that does not require larger or more implantable hardware (Borton et al, 2013;Seo et al, 2016). The most successful wireless design thus far is the IMES system, which uses implantable sensors within the residual limb for prosthetic control (Weir et al, 2009).…”

“…To increase motivation and participation, alternative training systems with a game-based model have been developed. These include computer gaming, virtual reality environments, and augmented reality (Resnik et al, 2011;Winslow et al, 2018;Boschmann et al, 2021). Several studies have demonstrated successful prosthesis control with these training systems, in addition to individuals reporting increased usability and motivation (Tabor et al, 2018;Kristoffersen et al, 2021).…”