Fusion of Haptic and Gesture Sensors for Rehabilitation of Bimanual Coordination and Dexterous Manipulation

Abstract

Disabilities after neural injury, such as stroke, bring tremendous burden to patients, families and society. Besides the conventional constrained-induced training with a paretic arm, bilateral rehabilitation training involves both the ipsilateral and contralateral sides of the neural injury, fitting well with the fact that both arms are needed in common activities of daily living (ADLs), and can promote good functional recovery. In this work, the fusion of a gesture sensor and a haptic sensor with force feedback capabilities has enabled a bilateral rehabilitation training therapy. The Leap Motion gesture sensor detects the motion of the healthy hand, and the omega.7 device can detect and assist the paretic hand, according to the designed cooperative task paradigm, as much as needed, with active force feedback to accomplish the manipulation task. A virtual scenario has been built up, and the motion and force data facilitate instantaneous visual and audio feedback, as well as further analysis of the functional capabilities of the patient. This task-oriented bimanual training paradigm recruits the sensory, motor and cognitive aspects of the patient into one loop, encourages the active involvement of the patients into rehabilitation training, strengthens the cooperation of both the healthy and impaired hands, challenges the dexterous manipulation capability of the paretic hand, suits easy of use at home or centralized institutions and, thus, promises effective potentials for rehabilitation training.

Effectiveness of Serious Games for Leap Motion on the Functionality of the Upper Limb in Parkinson’s Disease: A Feasibility Study

“…Besides, others studies integrate support devices in addition to the LMC to assist the patient. In [ 27 ], the fusion of the LMC and the Omega.7 haptic sensor with force feedback capabilities has enabled a bilateral rehabilitation training therapy. The LMC tracks the healthy hand and the Omega.7 device haptically interacts with the impaired hand.…”

Mirror Therapy Using Gesture Recognition for Upper Limb Function, Neck Discomfort, and Quality of Life After Chronic Stroke: A Single-Blind Randomized Controlled Trial

“…However, the GR mirror therapy group is required because of the accuracy and speed to perform tasks in program configuration [19]. A previous study reported an improvement in coordination, dexterity, hand use ability, and grasp force using a Leap motion controller with stroke patients [12,24].…”

“…These GR algorithms mainly focus on the movement of arm, hands, eyes, legs, and other body parts. The main idea is to capture body movements using capture devices and send the acquired data to a computer [12]. A remarkable example is shown in physical rehabilitation, where the low-cost hardware and algorithms accomplish outstanding results in therapy of patients with mobility issues.…”

A review on manipulation skill acquisition through teleoperation‐based learning from demonstration

“…In this case, it needs to modulate the F I G U R E 8 Tele-rehabilitation by multimodal teleoperation adapted from ref. [127]. (a) The Leap Motion sensor tracks the motion of a healthy hand.…”