Project Butterfly: Synergizing Immersive Virtual Reality with Actuated Soft Exosuit for Upper-Extremity Rehabilitation

Elor, Aviv; Lessard, Steven; Teodorescu, Mircea; Kurniawan, Sri

doi:10.1109/vr.2019.8798014

Cited by 22 publications

(13 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…stroke survivors or cognitive disabilities). Our lab has worked with disability groups within our lab exploring various virtual reality mediums for users with cognitive disabilities, testing soft exo-suits meant for post-stroke rehabilitation, and physical rehabilitation games for users with cognitive disabilities [9], [56]- [58]. In the future we will collect motion data of these varying groups to develop more inclusive patient models.…”

Section: A Limitations Of the Study And Future Workmentioning

confidence: 99%

Predictive Shoulder Kinematics of Rehabilitation Exercises Through Immersive Virtual Reality

et al. 2022

Self Cite

View full text Add to dashboard Cite

Objective: The adoption of telehealth rapidly accelerated due to the global COVID19 pandemic disrupting communities and in-person healthcare practices. While telehealth had initial benefits in enhancing accessibility for remote treatment, physical rehabilitation has been heavily limited due to the loss of hands-on evaluation tools. This paper presents an immersive virtual reality (iVR) pipeline for replicating physical therapy success metrics through applied machine learning of patient observation. Methods: We demonstrate a method of training gradient boosted decision-trees for kinematic estimation to replicate mobility and strength metrics with an off-the-shelf iVR system. During a two-month study, training data was collected while a group of users completed physical rehabilitation exercises in an iVR game. Utilizing this data, we trained on iVR based motion capture data and OpenSim biomechanical simulations. Results: Our final model indicates that upper-extremity kinematics from OpenSim can be accurately predicted using the HTC Vive head-mounted display system with a Mean Absolute Error less than 0.78 degrees for joint angles and less than 2.34 Nm for joint torques. Additionally, these predictions are viable for run-time estimation, with approximately a 0.74 ms rate of prediction during exercise sessions. Conclusion: These findings suggest that iVR paired with machine learning can serve as an effective medium for collecting evidence-based patient success metrics in telehealth. Significance: Our approach can help increase the accessibility of physical rehabilitation with off-the-shelf iVR head-mounted display systems by providing therapists with metrics needed for remote evaluation.

show abstract

Section: A Limitations Of the Study And Future Workmentioning

confidence: 99%

Predictive Shoulder Kinematics of Rehabilitation Exercises Through Immersive Virtual Reality

et al. 2022

Self Cite

View full text Add to dashboard Cite

show abstract

“…Figure 1 shows the experimental setup. Through Vive and the Unity Game Engine, motion capture and behavior game data were recorded and stored during runtime at 90 Hz using a data exportation method developed in previous studies by Elor et al [11], [49]. Motions were captured through utilizing the HTC Vive Pro 2018's outside-in tracking algorithm by a constellation of "lighthouse" laser systems [1].…”

Section: A Data Collectionmentioning

confidence: 99%

“…II. SYSTEM DESIGN Our physical therapy system is based on a game called "Project Butterfly" (PBF) that was proposed by Elor et al [49], [50]. Previously, PBF explored the feasibility of a virtual reality enhanced exo-skeleton for post-stroke and elderly assistance through two exercises, but was not designed or tested for upper extremity physical therapy over a extended period of time with varying custom exercise movements as reported in this study.…”

Section: Introductionmentioning

confidence: 99%

Gaming Beyond the Novelty Effect of Immersive Virtual Reality for Physical Rehabilitation

et al. 2022

Self Cite

View full text Add to dashboard Cite

Immersive Virtual Reality (iVR) Head-Mounted Display (HMD) systems paired with serious exercise games can positively augment physical rehabilitation process from both engagement and analytics perspectives. This paper presents a serious game for iVR HMD based long term upper-extremity exercise. We demonstrate the capabilities of our game through a case study with five users recovering from upper-extremity injuries. We examine how our program maintains engagement and motivation over eight weeks, where users completed bi-weekly prescribed movements framed as protecting a virtual butterfly. We assess user experiences through a mixture of biomarkers from brainwave, heart rate, and galvanic skin response recorded at runtime as well as motion capture and behavioral game data. Our results suggest that the iVR game was an effective medium in inducing high compliance, physical performance, and biometric changes even with increasing difficulty beyond the novelty effect period. We conclude with considerations of future work for iVR physical therapy games that adapt to biometric response.

show abstract

“…PSC uses score and sensory feedback as a motivator to keep the user engaged over their exercise sessions [27]. Another one of our previous studies has explored iVR games to strictly follow exercises by protecting a "cute" virtual butterfly in "Project Butterfly" (PBF) [26]. With therapeutic goals in mind, designing iVR experiences like PSC and PBF where the user is in complete control of the environment and is guided by emotive-based incentives can be an excellent approach for iVR physical exercise experiences.…”

Section: Users With Cognitive Disabilitymentioning

confidence: 99%

“…Past studies with the PSC iVR framework have shown great potential to increase compliance for adults with physical disabilities [26,27]. Nevertheless, these users have not been explored with CAVE in this study due to resource constraints.…”

Section: Considerations and Limitationsmentioning

confidence: 99%

On Shooting Stars

Elor

Powell

Mahmoodi

et al. 2020

ACM Trans. Comput. Healthcare

Self Cite

View full text Add to dashboard Cite

Inactivity and a lack of engagement with exercise is a pressing health problem in the United States and beyond. Immersive Virtual Reality (iVR) is a promising medium to motivate users through engaging virtual environments. Currently, modern iVR lacks a comparative analysis between research and consumer-grade systems for exercise and health. This article examines two such iVR mediums: the Cave Automated Virtual Environment (CAVE) and the head-mounted display (HMD). Specifically, we compare the room-scale Mechdyne CAVE and HTC Vive Pro HMD with a custom in-house exercise game that was designed such that user experiences were as consistent as possible between both systems. To ensure that our findings are generalizable for users of varying abilities, we recruited 40 participants with and without cognitive disabilities with regard to the fact that iVR environments and games can differ in their cognitive challenge between users. Our results show that across all abilities, the HMD excelled in in-game performance, biofeedback response, and player engagement. We conclude with considerations in utilizing iVR systems for exergaming with users across cognitive abilities.

show abstract

Project Butterfly: Synergizing Immersive Virtual Reality with Actuated Soft Exosuit for Upper-Extremity Rehabilitation

Cited by 22 publications

References 36 publications

Predictive Shoulder Kinematics of Rehabilitation Exercises Through Immersive Virtual Reality

Predictive Shoulder Kinematics of Rehabilitation Exercises Through Immersive Virtual Reality

Gaming Beyond the Novelty Effect of Immersive Virtual Reality for Physical Rehabilitation

On Shooting Stars

Contact Info

Product

Resources

About