Wearable Augmented Reality Application for Shoulder Rehabilitation

Abstract

Augmented reality (AR) technology is gaining popularity and scholarly interest in the rehabilitation sector because of the possibility to generate controlled, user-specific environmental and perceptual stimuli which motivate the patient, while still preserving the possibility to interact with the real environment and other subjects, including the rehabilitation specialist. The paper presents the first wearable AR application for shoulder rehabilitation, based on Microsoft HoloLens, with real-time markerless tracking of the user’s hand. Potentialities and current limits of commercial head-mounted displays (HMDs) are described for the target medical field, and details of the proposed application are reported. A serious game was designed starting from the analysis of a traditional rehabilitation exercise, taking into account HoloLens specifications to maximize user comfort during the AR rehabilitation session. The AR application implemented consistently meets the recommended target frame rate for immersive applications with HoloLens device: 60 fps. Moreover, the ergonomics and the motivational value of the proposed application were positively evaluated by a group of five rehabilitation specialists and 20 healthy subjects. Even if a larger study, including real patients, is necessary for a clinical validation of the proposed application, the results obtained encourage further investigations and the integration of additional technical features for the proposed AR application.

Upper Limb Physical Rehabilitation Using Serious Videogames and Motion Capture Systems: A Systematic Review

“…On the other hand, in [ 8 ], the potential offered by Microsoft HoloLens was explored, i.e., an optical device placed on the head which does not require markers or sensors for the following of the arm or the hand. An application was developed with augmented reality, using the engine from the game Unity and the Microsoft HoloToolkit, for the improvement of the range of movement of the shoulder, allowing a perfect remote interaction with the personal doctor.…”

“…In this group of works, the development of videogames that used the platform Unity was noticeable [ 53 ]. For example, in [ 8 ], with the purpose of treating any deficit of the upper limb which deteriorates the range of motion, a videogame was designed and developed from traditional rehabilitation exercises with the Rolyan range-of-motion shoulder arc. The game presents a curved tube, with mobile colored rings around it.…”

“…Commercial video games were used in [ 7 , 12 , 19 , 20 , 21 ], which, despite encouraging the execution of physical activity and supporting the player’s motivation, were not adapted to the particular characteristics of physical rehabilitation. For this reason, most of the studies proposed active video games specifically for rehabilitation, to increase motivation and adherence to therapies [ 8 , 11 , 13 , 15 , 16 , 17 , 18 , 22 , 23 , 25 ], in some cases associated with a configuration module allowing the health professional to adjust the characteristics of the game according to the diagnosis and progress of the patient in treatment [ 9 , 10 , 14 , 24 ]. In the particular case of commercial rehabilitation products, such as ArmeoSenso, Bimeo, or JRS Wave, they respond adequately to such requirements in the area of physical rehabilitation, although the additional costs involved must be taken into account.…”

“…In this sense, out of the works included in this review only five presented home rehabilitation systems that allow the therapist to remotely adjust and monitor the configuration of the game according to the patient’s rehabilitation objectives, incorporating the recording of information in an associated computer system. Out of these, in [ 8 ], IMUs are used, in [ 9 , 13 , 20 ] the Kinect sensor was used, and two of the three works analyzed in [ 23 ] used the 5DT Data Glove Ultra and the Nintendo Wiimote. Moreover, in four of these five works, videogames developed specifically for rehabilitation were proposed.…”

Upper Limb Rehabilitation Tools in Virtual Reality Based on Haptic and 3D Spatial Recognition Analysis: A Pilot Study

“…Range of motion (ROM) is a method used by clinicians to check the health condition of the joint and improve the range of motion, affecting the qualitative characteristics of active motion [ 36 ]. The double-curved shoulder arc (Fit to Function Therapy, NY, USA) ( (accessed on 1 April 2021)) form ( Figure 1 ) used in upper limb rehabilitation is a rehabilitation tool that can check active movement through abnormal conversion behavior and increase the performance difficulty through multiple transformations [ 37 , 38 ]. However, tools that can be analyzed objectively analyzed during the rehabilitation in a virtual environment are currently being developed.…”