Interpreting Deep Learning based Cerebral Palsy Prediction with Channel Attention

Zhu, Manli; Men, Qianhui; Ho, Edmond S. L.; Leung, Howard; Shum, Hubert P. H.

doi:10.1109/bhi50953.2021.9508619

Cited by 13 publications

(10 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For our model, one video is classified as F+ if the video level score got by Top-K strategy is greater than 0.5 in the seen video. For STAM [18], MS-G3D [15], Zhu et.al [27], the classification threshold is set to 0.5. Since action detection tasks also include classification, we also report the video-level performance of WOAD [7] and W-TALC [19].…”

Section: Resultsmentioning

confidence: 99%

Weakly Supervised Online Action Detection for Infant General Movements

Tongyi¹,

Jia²,

Zhang³

et al. 2022

Preprint

View full text Add to dashboard Cite

To make the earlier medical intervention of infants' cerebral palsy (CP), early diagnosis of brain damage is critical. Although general movements assessment(GMA) has shown promising results in early CP detection, it is laborious. Most existing works take videos as input to make fidgety movements(FMs) classification for the GMA automation. Those methods require a complete observation of videos and can not localize video frames containing normal FMs. Therefore we propose a novel approach named WO-GMA to perform FMs localization in the weakly supervised online setting. Infant body keypoints are first extracted as the inputs to WO-GMA. Then WO-GMA performs local spatio-temporal extraction followed by two network branches to generate pseudo clip labels and model online actions. With the clip-level pseudo labels, the action modeling branch learns to detect FMs in an online fashion. Experimental results on a dataset with 757 videos of different infants show that WO-GMA can get state-of-the-art video-level classification and cliplevel detection results. Moreover, only the first 20% duration of the video is needed to get classification results as good as fully observed, implying a significantly shortened FMs diagnosis time. Code is available at: https://github.com/scofiedluo/WO-GMA.

show abstract

Section: Resultsmentioning

confidence: 99%

Weakly Supervised Online Action Detection for Infant General Movements

Tongyi¹,

Jia²,

Zhang³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…• Conv2D-1, Conv2D-2 [7]: They are two 2D convolutional neural networks, each of them consists of two 2D convolutional layers with differences in the output channel sizes. • CANet [9]: This method proposes a 2D convolutional neural network with the squeeze-and-excitation channel attention module. The whole system is proposed for the CP classification task on the MINI-RGBD dataset.…”

Section: B Comparing With State-of-the-art Methodsmentioning

confidence: 99%

“…In order to evaluate the robustness of our system and other state-of-the-art DNN-based methods [9], [20], [21], we simulate different datasets by adding different levels of Gaussian noise to the infant joint The noise level is divided into four levels from 15% standard deviation to 120% standard deviation of each infant's joint pose data. The tests results are displayed in Fig.…”

Section: F Robustness Testmentioning

confidence: 99%

“…McCay et al [7] proposed a fully connected deep learning network and four Convolutional Neural Network (CNN)-based deep learning architectures to classify the abnormal movements of CP infants by using the histogram of joint orientation 2D and joint displacement 2D features, achieved the highest prediction accuracy of 91.67% on the MINI-RGBD dataset [8]. Zhu [9] further applied the channel attention mechanism on the 2D-CNN model to interpret the CP prediction outcome on the same dataset. However, the robustness and generality of their proposed method have not been fully evaluated since the results are obtained from a single small dataset.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Cerebral Palsy Prediction with Frequency Attention Informed Graph Convolutional Networks

Zhang¹,

Shum²,

Ho³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Early diagnosis and intervention are clinically considered the paramount part of treating cerebral palsy (CP), so it is essential to design an efficient and interpretable automatic prediction system for CP. We highlight a significant difference between CP infants' frequency of human movement and that of the healthy group, which improves prediction performance. However, the existing deep learning-based methods did not use the frequency information of infants' movement for CP prediction. This paper proposes a frequency attention informed graph convolutional network and validates it on two consumer-grade RGB video datasets, namely MINI-RGBD and RVI-38 datasets. Our proposed frequency attention module aids in improving both classification performance and system interpretability. In addition, we design a frequency-binning method that retains the critical frequency of the human joint position data while filtering the noise. Our prediction performance achieves stateof-the-art research on both datasets. Our work demonstrates the effectiveness of frequency information in supporting the prediction of CP non-intrusively and provides a way for supporting the early diagnosis of CP in the resource-limited regions where the clinical resources are not abundant.

show abstract

“…This makes it difficult to discover inter-joint correlations that are important for capturing coordination among different joints of human gait from raw joint features. We also observed that the majority of existing work only applies DL on individual joint features such as the time series of joint positions [23,28]. As a consequence, these methods usually perform sub-optimally on smaller-scale medical datasets.…”

Section: Introductionmentioning

confidence: 93%

A Two-stream Convolutional Network for Musculoskeletal and Neurological Disorders Prediction

Zhu¹,

Men²,

Ho³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Musculoskeletal and neurological disorders are the most common causes of walking problems among older people, and they often lead to diminished quality of life. Analyzing walking motion data manually requires trained professionals and the evaluations may not always be objective. To facilitate early diagnosis, recent deep learning-based methods have shown promising results for automated analysis, which can discover patterns that have not been found in traditional machine learning methods. We observe that existing work mostly applies deep learning on individual joint features such as the time series of joint positions. Due to the

show abstract

Interpreting Deep Learning based Cerebral Palsy Prediction with Channel Attention

Cited by 13 publications

References 11 publications

Weakly Supervised Online Action Detection for Infant General Movements

Weakly Supervised Online Action Detection for Infant General Movements

Cerebral Palsy Prediction with Frequency Attention Informed Graph Convolutional Networks

A Two-stream Convolutional Network for Musculoskeletal and Neurological Disorders Prediction

Contact Info

Product

Resources

About