Fully automated image-based estimation of postural point-features in children with cerebral palsy using deep learning

Abstract

Objective: To automate identification of postural point-features from colour videos of children with neuromotor disability, during clinical assessment. The automatic identification of 13 points of interest (2, 6, 2, 3 points on the head, trunk, pelvis, arm respectively) is required to estimate the location and orientation of head, trunk, and arm segments, from videos of the clinical test "Segmental Assessment of Trunk Control" (SATCo) which is a test of seated postural control. Methods: Three expert operators manually annotated 13 point-features in every fourth image of 177 short (5-10 second) videos (25 Hz) showing 12 children with cerebral palsy (ages: 4.52 ±2.4, male: 9), participating in SATCo testing. Linear interpolation for the remaining images resulted in 30,825 annotated images. Meanpooling and max-pooling convolutional neural networks were trained with cross-validation, giving held-out test results for all children. Results: The point-features were estimated with error 4.40 ±3.75 pixels (mean-pooling), and 4.49 ±4.45 pixels (maxpooling), at approximately 100 images per second. Trunk segment angles (head, neck, 6 thoraco-lumbar-pelvic segments) were estimated with error 6.4° ±2.8° allowing accurate classification (F1 > 80%) of deviation from a reference posture at thresholds up to 3°, 3°, 2° respectively. Contact between arm point features (elbow, wrist) and supporting surface was classified at F1 = 80.5%. Conclusion and Significance: This study demonstrates, for the first time, a technical solution to automate identification of i) a sitting segmental-posture including individual trunk segments, ii) changes away from that posture, and iii) support from the upper limb, required for the clinical SATCo.

Abnormal Infant Movements Classification With Deep Learning on Pose-Based Features

“…These approaches make use of large datasets to train deep learning models capable of achieving state-ofthe-art classification accuracy. Several studies have attempted to make use of the general improvements to accuracy provided by deep learning by applying deep learning frameworks to similar movement related diagnostic activities [8], [21], [35]. Whilst the results are promising, the holistic application of deep learning in the healthcare domain faces several challenges, most notably the large amount of data required for suitable results, and the problem of understandable AI.…”

Can machine learning automatically detect the aligned trunk in sitting directly from raw video using a depth camera?

“…These assessments, although reliable, are subjective and most consider the head/trunk as a single unit, ignoring both its multisegmental composition and any use of the hands for support to maintain a balanced posture (Heyrman et al, 2011;Pountney et al, 1999;Reid, 1997;Russell et al, 2002;Verheyden et al, 2004). The Segmental Assessment of Trunk Control (SATCo) is unique in addressing these issues, evaluating control based on 1) the position of individual trunk segments in space relative to a defined aligned posture and 2) the use of external support (Butler et al, 2010) (Cunningham et al, 2018).…”

“…While this work (Cunningham et al, 2018) demonstrates, for the first time, a feasible technical solution to automate tracking of individual trunk segments in a given sitting posture, and of changes away from that posture, it leaves unsolved the automated identification of the aligned trunk posture to act as a reference.…”