Infant Cognitive Scores Prediction with Multi-stream Attention-Based Temporal Path Signature Features

Zhang, Xin; Cheng, Jiale; Ni, Hao; Li, Chenyang; Xu, Xiangmin; Wu, Zhengwang; Wang, Li; Lin, Weili; Shen, Dinggang; Li, Gang

doi:10.1007/978-3-030-59728-3_14

Cited by 5 publications

(12 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Considering the average performance of all tasks, our method reaches a state-of-the-art RMSE 0.1631. It is noteworthy that BrainPSNet [24] introduces the path signature in equation ( 2) to describe the development pattern of brain regions, but it is poor-performed for neglecting the graph structure among brain regions and the benefits may brought by the learnable matrices in equation (3).…”

Section: Ablation Studymentioning

confidence: 99%

“…Learning a set of compact representations, which effectively capture the spatial and temporal cortical developmental patterns, is one of the most important techniques to deal with the small sample size problem. However, existing methods usually simply flatten brain morphological feature vectors at every time point into a vector [1,23,24], which obviously neglect the potential connectivity among brain regions. It has been revealed that the massive brain region connections form elegant topologies, such as small-worldness and modular organization, which can be probed using the graph theoretical modeling method [25].…”

Section: Introductionmentioning

confidence: 99%

“…Path signature is a graded sequence of statistics to characterize streamed data. It has been applied to acquire effective features of the path [13,17,14,24], and might be helpful to extract the developmental information in brain regions.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Learning Infant Brain Developmental Connectivity for Cognitive Score Prediction

Cheng

Zhang

et al. 2021

Machine Learning in Medical Imaging

Self Cite

View full text Add to dashboard Cite

During infancy, the human brain develops rapidly in terms of structure, function and cognition. The tight connection between cognitive skills and brain morphology motivates us to focus on individual level cognitive score prediction using longitudinal structural MRI data. In the early postnatal stage, the massive brain region connections contain some intrinsic topologies, such as small-worldness and modular organization. Accordingly, graph convolutional networks can be used to incorporate different region combinations to predict the infant cognitive scores. Nevertheless, the definition of the brain region connectivity remains a problem. In this work, we propose a crafted layer, the Inter-region Connectivity Module (ICM), to effectively build brain region connections in a data-driven manner. To further leverage the critical cues hidden in the development patterns, we choose path signature as the sequential data descriptor to extract the essential dynamic information of the region-wise growth trajectories. With these region-wise developmental features and the inter-region connectivity, a novel Cortical Developmental Connectivity Network (CDC-Net) is built. Experiments on a longitudinal infant dataset within 3 time points and hundreds of subjects show our superior performance, outperforming classical machine learning based methods and deep learning based algorithms.

show abstract

Section: Ablation Studymentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Learning Infant Brain Developmental Connectivity for Cognitive Score Prediction

Cheng

Zhang

et al. 2021

Machine Learning in Medical Imaging

Self Cite

View full text Add to dashboard Cite

show abstract

“…Further, path signature features, extracted by a differentiable temporal path signature (TPS) layer, can effectively describe dynamic properties of the cortical growth trajectory. With the help of an attention mechanism to automatically weight brain regions at different time points, encouraging results can be obtained [22].…”

Section: Introductionmentioning

confidence: 99%

“…Similarly, based on our previous work [22], we stack the light-weight TPS layers to obtain a global receptive field and aggregate features from all existing visits. Compared to [12], [13], their mapping function is a single-layer nonlinear function, since it is risky to apply its complicated substitutes such as RNNs with the SSS problem, while the hierarchical structure of dynamic patterns may be overlooked.…”

Section: Introductionmentioning

confidence: 99%

Path Signature Neural Network of Cortical Features for Prediction of Infant Cognitive Scores

Cheng

Zhang

et al. 2022

IEEE Trans. Med. Imaging

Self Cite

View full text Add to dashboard Cite

Studies have shown that there is a tight connection between cognition skills and brain morphology during infancy. Nonetheless, it is still a great challenge to predict individual cognitive scores using their brain morphological features, considering issues like the excessive feature dimension, small sample size and missing data. Due to the limited data, a compact but expressive feature set is desirable as it can reduce the dimension and avoid the potential overfitting issue. Therefore, we pioneer the path signature method to further explore the essential hidden dynamic patterns of longitudinal cortical features. To form a hierarchical and more informative temporal representation, in this work, a novel cortical feature based path signature neural network (CF-PSNet) is proposed with stacked differentiable temporal path signature layers for prediction of individual cognitive scores. By introducing the existence embedding in path generation, we can improve the robustness against the missing data. Benefiting from the global temporal receptive field of CF-PSNet, characteristics consisted in the existing data can be fully leveraged. Further, as there is no need for the whole brain to work for a certain cognitive ability, a top K selection module is used to select the most influential brain regions, decreasing the model size and the risk of overfitting. Extensive experiments are conducted on an in-house longitudinal infant dataset within 9 time points. By comparing with several recent algorithms, we illustrate the state-of-the-art performance of our CF-X. Zhang and X.

show abstract

Skeleton-Based Gesture Recognition With Learnable Paths and Signature Features

Cheng,

Shi,

et al. 2024

IEEE Trans. Multimedia

Self Cite

View full text Add to dashboard Cite

For the skeleton-based gesture recognition, graph convolutional networks (GCNs) have achieved remarkable performance since the human skeleton is a natural graph. However, the biological structure might not be the crucial one for motion analysis. Also, spatial differential information like joint distance and angle between bones may be overlooked during the graph convolution. In this paper, we focus on obtaining meaningful joint groups and extracting their discriminative features by the path signature (PS) theory. Firstly, to characterize the constraints and dependencies of various joints, we propose three types of paths, i.e., spatial, temporal, and learnable path. Especially, a learnable path generation mechanism can group joints together that are not directly connected or far away, according to their kinematic characteristic. Secondly, to obtain informative and compact features, a deep integration of PS with few parameters are introduced. All the computational process is packed into two modules, i.e., spatial-temporal path signature module (ST-PSM) and learnable path signature module (L-PSM) for the convenience of utilization. They are plug-and-play modules available for any neural network like CNNs and GCNs to enhance the feature extraction ability. Extensive experiments have conducted on three mainstream datasets (ChaLearn 2013, ChaLearn 2016, and AUTSL). We achieved the state-of-the-art results with simpler framework and much smaller model size. By inserting our two modules into the several GCN-based networks, we can observe clear improvements demonstrating the great effectiveness of our proposed method.

show abstract

Infant Cognitive Scores Prediction with Multi-stream Attention-Based Temporal Path Signature Features

Cited by 5 publications

References 30 publications

Learning Infant Brain Developmental Connectivity for Cognitive Score Prediction

Learning Infant Brain Developmental Connectivity for Cognitive Score Prediction

Path Signature Neural Network of Cortical Features for Prediction of Infant Cognitive Scores

Skeleton-Based Gesture Recognition With Learnable Paths and Signature Features

Contact Info

Product

Resources

About