An FPGA-Based Upper-Limb Rehabilitation Device for Gesture Recognition and Motion Evaluation Using Multi-Task Recurrent Neural Networks

Abstract

Upper-Extremity motor impairment affects millions of Americans due to cerebrovascular incidents, spinal cord injuries, or brain trauma. Current therapy practices used to assist these individuals in regaining motor functionality often require extensive time at rehabilitation facilities with potentially prohibitive travel or financial costs. This work presents a mobile low-cost field programmable gate array (FPGA)-smart rehabilitation system that can be used in home environments. The prototype is a rehabilitation table instrumented with a capacitive sensor array (CSA) to track upper-extremity motions of the user through proximity or touch. In addition, inertial measurement units (IMUs) are placed on the affected upper limb and combined with the CSA data with our sensor fusion signal processing architecture. Motions are classified and evaluated using multi-task convolutional recurrent neural networks with three additional motion quality output classes to personalize recognition based on the particular motor skills of each patient. The prototype achieves above 99% accuracy with 32-bit fixed-point format implementation for recognizing dynamic motions and identifying unnatural characteristics (i.e., tremor or limited flexion and extension) in upper limb motions based on sensor values. The convolutional recurrent neural network (C-RNN) fusion classification network is implemented on a 200 MHz Zynq ZCU104 FPGA using an HLS-based design optimized with pipelining and parallelism techniques and achieves 5.4x speedup compared to ARM ® Cortex-A53 implementation running at an operating frequency of 1.3 GHz. The prototype is also demonstrated to perform the machine learning classification in real-time.

A Review of the Optimal Design of Neural Networks Based on FPGA

“…In September of the same year, Siyu Xiong et al [50] used an FPGA-based CNN to speed up the detection and segmentation of 3D brain tumors, providing a new direction for the improvement of automatic segmentation of brain tumors. In 2022, H. Liu et al [51] designed an FPGA-based multi-task recurrent neural network gesture recognition and motion assessment upper limb rehabilitation device, making it unnecessary for patients with upper limb movement disorders to spend a large amount of time in the hospital for upper limb rehabilitation training. Using this device can help patients to complete the same professional upper limb rehabilitation training at home in the same way as in the hospital, helping the patients in reducing the burden of medical expenses and reducing the investment of a large number of medical resources.…”

Framework design of reconfigurable convolution neural network acceleration system for edge computing

“…Convolutional neural networks (CNNs) are extensively used for complex tasks such as computer vision, speech recognition, and image classification, including object detection and handwriting recognition [2][3][4] [5]. Field-Programmable Gate Arrays (FPGAs) are popular edge computing devices and are considered the most suitable platform for implementing CNN-based algorithms due to their architecture based on Lookup Tables (LUTs) and parallel computing features [6].…”