Toward Marker-Free 3D Pose Estimation in Lifting: A Deep Multi-View Solution

Abstract

Lifting is a common manual material handling task performed in the workplaces. It is considered as one of the main risk factors for Work-related Musculoskeletal Disorders. To improve work place safety, it is necessary to assess musculoskeletal and biomechanical risk exposures associated with these tasks, which requires very accurate 3D pose. Existing approaches mainly utilize marker-based sensors to collect 3D information. However, these methods are usually expensive to setup, timeconsuming in process, and sensitive to the surrounding environment. In this study, we propose a multi-view based deep perceptron approach to address aforementioned limitations. Our approach consists of two modules: a "view-specific perceptron" network extracts rich information independently from the image of view, which includes both 2D shape and hierarchical texture information; while a "multi-view integration" network synthesizes information from all available views to predict accurate 3D pose. To fully evaluate our approach, we carried out comprehensive experiments to compare different variants of our design. The results prove that our approach achieves comparable performance with former marker-based methods, i.e. an average error of 14.72 ± 2.96 mm on the lifting dataset. The results are also compared with state-of-the-art methods on HumanEva-I dataset [1], which demonstrates the superior performance of our approach.

Deep Learning for Musculoskeletal Force Prediction

“…More recently, deep models combining recurrent and convolutional modules have been applied to time series of biological signals, for example in the prediction of kinematic parameters from EMG data41. There has also been growing interest in the use of deep learning architectures for motion capture-independent pose estimation24.…”

Automatic Classification of Squat Posture Using Inertial Sensors: Deep Learning Approach

“…However, 3-D motion capture systems are unsuitable for personal fitness because they require large, complex, and expensive measurement environments comprising multiple motion tracking cameras and markers affixed to the bodies of subjects. The other method is an image-processing approach that employs deep convolutional neural networks to learn the image features for activity recognition [6,7] and human pose estimation [8][9][10]. Because the video image processing needs high computation power, it is not proper for self-coaching system in home.…”

Home-Based Physical Therapy with an Interactive Computer Vision System

“…In addition, MotionTalk [43] and VERA (Virtual Exercise Rehabilitation Assistant) [17] are examples of systems that employ a Microsoft (MS) Kinect sensor for tracking a patients movements. Among other notable RGB-D cameras 1 , the Intel RealSense TM camera has been used for 1 Other notable RGB-D cameras include BlasterX Senz3D (Creative rehabilitation [4,8,11] Pushing forward on the contact-free and low-cost devices, the computer vision community enjoys the challenge of using RGB cameras only to analyze three-dimensional (3D) body pose [24,25,26,28]. In fact, accurate human pose estimation is considered one of the most challenging tasks in the field of computer vision [21,42], because images of people have large in-class variations caused by the intrinsic deformation of the shape of the human body and high variability in human clothing and environmental factors [22].…”