Continuous and Unified Modeling of Joint Kinematics for Multiple Activities

Abstract

Intuitive control of powered prosthetic lower limbs is still an open-ended research goal. Current controllers employ discrete locomotion modes for well-defined and frequently encountered scenarios such as stair ascent, stair descent, or ramps. Non-standard movements such as side-shuffling into cars and avoiding obstacles are challenging to powered limb users. Human locomotion is a continuous motion comprising rhythmic and non-rhythmic movements, fluidly adapting to the environment. It exhibits strong inter-joint coordination and the movement of a single joint can be largely predicted based on the movement of the rest of the body. We explore a continuous and unified kinematics estimation strategy for a wide variety of movements without the need for labeled examples. Our data-driven approach uses natural body motion from the intact limbs and trunk to generate a kinematic reference trajectory for prosthetic joints. Wearable sensors were worn by 63 subjects without disabilities to record full-body kinematics during typical scenarios (flat ground and stairs), and non-rhythmic and atypical movements (side shuffles, weaving through cones, backward walking). A Recurrent Neural Network (RNN) was trained to predict right ankle and knee kinematics from the kinematics of other joints as inputs. Results were assessed on 3 different test subjects previously unseen by the network. All predictions had a RMSE of less than 7.5 degrees and a high correlation across activities. These offline predictions were robust to subject-specific variations such as walking speed and step length. Additionally, to test the feasibility of using a data-driven reference towards prosthetic control in real-time, a systems test was designed with a single participant. The controller acquired live kinematics, generated predictions using a pre-trained neural network, and demonstrated the capability to actuate the knee joint of a powered prosthesis for the treadmill walking task.

A Non-Laboratory Gait Dataset of Full Body Kinematics and Egocentric Vision

“…With the dataset we present here, we intend to contribute data captured outside of the lab to enable comparative analyses with in-laboratory activities. This kind of data can be used to train data-driven predictive models of human movement 2,3 , or activity 4,5 or terrain 6 classification.…”

“…The second example analysis is using either the kinematic data alone 2 or also combining the the visual data 3 to predict knee and ankle motion. See Fig.…”

“…visual data zeroed out, to show its relative contribution. An interesting feature of this approach is that the system can target either instantaneous joint angles 2 or can predict into the future 3 . We expect that predicting further into the future should be more difficult, but this has yet to be systematically studied to our knowledge.…”

Optimizing Representations of Multiple Simultaneous Attributes for Gait Generation Using Deep Learning

“…In this manuscript we used these representations for a specific task i.e., to generate gait trajectories at a few desired speeds. However, these representations can potentially be used to personalize other predictive models of gait [11], [12], [13], instead of using more standard quantifiable attributes like height, weight etc.…”