Multi-Fusion Sensors for Action Recognition based on Discriminative Motion Cues and Random Forest

Hafeez, Sadaf; Jalal, Ahmad; Kamal, Shaharyar

doi:10.1109/comtech52583.2021.9616668

Cited by 15 publications

(3 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…human health and fitness conditions, has increased [1], [2], [3], [4], especially for ambient assisted-living. To increase the performance of their recognition systems, many researchers in this field have utilized inertial sensors, while others have employed depth or RGB sensors [5], [6], [7], [8]. The wide range of applications of HAR systems include smart home environments [9], security surveillance [10], and human health monitoring [11].…”

Section: Introductionmentioning

confidence: 99%

Multi-Sensor-Based Action Monitoring and Recognition via Hybrid Descriptors and Logistic Regression

et al. 2023

Self Cite

View full text Add to dashboard Cite

Section: Introductionmentioning

confidence: 99%

Multi-Sensor-Based Action Monitoring and Recognition via Hybrid Descriptors and Logistic Regression

et al. 2023

Self Cite

View full text Add to dashboard Cite

“…Given that camera-based sensors can provide contextual information, many researchers fuse the inertial data with camera-based data [ 25 ]. Nam et al [ 26 ] and Hafeez et al [ 27 ] fused accelerometer features with camera-based features to increase the classification accuracy for ambulatory activities. Doherty et al [ 28 ] used the context provided by cameras to identify the specific classed of activity once an accelerometer had identified the level of activity being undertaken.…”

Section: Introductionmentioning

confidence: 99%

Development and Testing of a Daily Activity Recognition System for Post-Stroke Rehabilitation

Proffitt,

Ma,

Skubic

2023

Sensors

View full text Add to dashboard Cite

Those who survive the initial incidence of a stroke experience impacts on daily function. As a part of the rehabilitation process, it is essential for clinicians to monitor patients’ health status and recovery progress accurately and consistently; however, little is known about how patients function in their own homes. Therefore, the goal of this study was to develop, train, and test an algorithm within an ambient, in-home depth sensor system that can classify and quantify home activities of individuals post-stroke. We developed the Daily Activity Recognition and Assessment System (DARAS). A daily action logger was implemented with a Foresite Healthcare depth sensor. Daily activity data were collected from seventeen post-stroke participants’ homes over three months. Given the extensive amount of data, only a portion of the participants’ data was used for this specific analysis. An ensemble network for activity recognition and temporal localization was developed to detect and segment the clinically relevant actions from the recorded data. The ensemble network, which learns rich spatial-temporal features from both depth and skeletal joint data, fuses the prediction outputs from a customized 3D convolutional–de-convolutional network, customized region convolutional 3D network, and a proposed region hierarchical co-occurrence network. The per-frame precision and per-action precision were 0.819 and 0.838, respectively, on the test set. The outcomes from the DARAS can help clinicians to provide more personalized rehabilitation plans that benefit patients.

show abstract

“…Given that camera-based sensors can provide contextual information, many researchers fuse the inertial data with camera-based data [65]. Nam et al [66] and Hafeez et al [67]fused accelerometer features with camera-based features to increase the accuracy of classification for ambulatory activities. Doherty et al [68] used the context provided by cameras to identify the specific class of activity once an accelerometer has identified the level of activity being undertaken.…”

Section: Inertialmentioning

confidence: 99%

Kinematic assessment for stroke patients in a stroke game and a daily activity recognition and assessment system

Ma¹

View full text Add to dashboard Cite

Stroke is the leading cause of serious, long-term disabilities among which deficits in motor abilities in arms or legs are most common. Those who suffer a stroke can recover through effective rehabilitation which is delicately personalized. To achieve the best personalization, it is essential for clinicians to monitor patients' health status and recovery progress accurately and consistently. Traditionally, rehabilitation involves patients performing exercises in clinics where clinicians oversee the procedure and evaluate patients' recovery progress. Following the in-clinic visits, additional home practices are tailored and assigned to patients. The in-clinic visits are important to evaluate recovery progress. The information collected can then help clinicians customize home practices for stroke patients. However, as the number of in-clinic sessions is limited by insurance policies, the recovery information collected in-clinic is often insufficient. Meanwhile, the home practice programs report low adherence rates based on historic data. Given that clinicians rely on patients to self-report adherence, the actual adherence rate could be even lower. Despite the limited feedback clinicians could receive, the measurement method is subjective as well. In practice, classic clinical scales are mostly used for assessing the qualities of movements and the recovery status of patients. However, these clinical scales are evaluated subjectively with only moderate inter-rater and intra-rater reliabilities. Taken together, clinicians lack a method to get sufficient and accurate feedback from patients, which limits the extent to which clinicians can personalize treatment plans. This work aims to solve this problem. To help clinicians obtain abundant health information regarding patients' recovery in an objective approach, I've developed a novel kinematic assessment toolchain that consists of two parts. The first part is a tool to evaluate stroke patients' motions collected in a rehabilitation game setting. This kinematic assessment tool utilizes body-tracking in a rehabilitation game. Specifically, a set of upper body assessment measures were proposed and calculated for assessing the movements using skeletal joint data. Statistical analysis was applied to evaluate the quality of upper body motions using the assessment outcomes. Second, to classify and quantify home activities for stroke patients objectively and accurately, I've developed DARAS, a daily activity recognition and assessment system that evaluates daily motions in a home setting. DARAS consists of three main components: daily action logger, action recognition part, and assessment part. The logger is implemented with a Foresite system to record daily activities using depth and skeletal joint data. Daily activity data in a realistic environment were collected from sixteen post-stroke participants. The collection period for each participant lasts three months. An ensemble network for activity recognition and temporal localization was developed to detect and segment the clinically relevant actions from the recorded data. The ensemble network fuses the prediction outputs from customized 3D Convolutional-De-Convolutional, customized Region Convolutional 3D network and a proposed Region Hierarchical Co-occurrence network which learns rich spatial-temporal features from either depth data or joint data. The per-frame precision and the per-action precision were 0.819 and 0.838, respectively, on the validation set. For the recognized actions, the kinematic assessments were performed using the skeletal joint data, as well as the longitudinal assessments. The results showed that, compared with non-stroke participants, stroke participants had slower hand movements, were less active, and tended to perform fewer hand manipulation actions. The assessment outcomes from the proposed toolchain help clinicians to provide more personalized rehabilitation plans that benefit patients.

show abstract

Multi-Fusion Sensors for Action Recognition based on Discriminative Motion Cues and Random Forest

Cited by 15 publications

References 31 publications

Multi-Sensor-Based Action Monitoring and Recognition via Hybrid Descriptors and Logistic Regression

Multi-Sensor-Based Action Monitoring and Recognition via Hybrid Descriptors and Logistic Regression

Development and Testing of a Daily Activity Recognition System for Post-Stroke Rehabilitation

Kinematic assessment for stroke patients in a stroke game and a daily activity recognition and assessment system

Contact Info

Product

Resources

About