Application of Convolutional Recurrent Neural Network for Individual Recognition Based on Resting State fMRI Data

Abstract

In most task and resting state fMRI studies, a group consensus is often sought, where individual variability is considered a nuisance. None the less, biological variability is an important factor that cannot be ignored and is gaining more attention in the field. One recent development is the individual identification based on static functional connectome. While the original work was based on the static connectome, subsequent efforts using recurrent neural networks (RNN) demonstrated that the inclusion of temporal features greatly improved identification accuracy. Given that convolutional RNN (ConvRNN) seamlessly integrates spatial and temporal features, the present work applied ConvRNN for individual identification with resting state fMRI data. Our result demonstrates ConvRNN achieving a higher identification accuracy than conventional RNN, likely due to better extraction of local features between neighboring ROIs. Furthermore, given that each convolutional output assembles in-place features, they provide a natural way for us to visualize the informative spatial pattern and temporal information, opening up a promising new avenue for analyzing fMRI data.

Functional Connectome Fingerprinting Using Shallow Feedforward Neural Networks

“…For M = 379 ROIs, N = 100 time points (72s duration) per segment, and K = 256 hidden units, the mean classification accuracies of the corrNN and normNN models were 99.8% and 99.6%, respectively, for an initial set of 100 subjects, and 100.0% and 99.7% for a second independent set of 100 subjects. These accuracies are higher than those reported (94.3% to 98.5%) for RNN models (2, 3). For comparison, the mean classification accuracy using the similarity of the correlation coefficients was 79.4% for 100 time points per segment, which is higher than the 68% mean accuracy reported in (1) using data from a different dataset.…”

“…Functional connectome fingerprinting based on the similarity of correlation coefficient matrices computed from rsfMRI data can identify individuals with high accuracy (> 98%) using long duration (> 12 minute) scans but considerably lower accuracy (≈ 68%) is obtained when the data duration is decreased to 72s (1). Recurrent neural networks (RNN) can achieve high accuracy (98.5%) with short duration (72s) data, presumably reflecting their ability to capture both spatial and temporal features (2, 3). However, it has been shown that high RNN performance can be achieved even when the temporal order of the fMRI data is permuted (4), suggesting that the temporal features are not critical for identification.…”

A New Spiking Convolutional Recurrent Neural Network (SCRNN) With Applications to Event-Based Hand Gesture Recognition

“…Over the past few years, researchers have successfully applied CRNN in medical applications (Wang L. et al, 2019 ), speech processing (Cakir et al, 2017 ; Tan and Wang, 2018 ), and music classification (Choi et al, 2017 ). Adopting a recurrent structure enables the neural network to encapsulate the global information while local features are extracted by the convolution layers.…”

Representation Learning of Resting State fMRI with Variational Autoencoder

“…Most applications are in the regime of supervised learning. Typically, a neural network takes an fMRI-based input data and is trained to generate an output that optimally matches the ground truth for a task, such as individual identification (Chen and Hu, 2018;Wang et al, 2019), prediction of gender, age, or intelligence (Fan et al, 2020;Gadgil et al, 2020;Plis et al, 2014), disease classification (Seo et al, 2019;Suk et al, 2016;Wang et al, 2020;Yang et al, 2019;Zou et al, 2017). The labels required for supervised learning are often orders of magnitude smaller in size than the fMRI data itself, which has a high dimension in both space and time.…”

“…The labels required for supervised learning are often orders of magnitude smaller in size than the fMRI data itself, which has a high dimension in both space and time. As a result, the prior studies often limit the model capacity by using a shallow network and/or limit the input data to activity at the region of interest (ROI) level (Chen and Hu, 2018;Dvornek et al, 2018;Koppe et al, 2019;Matsubara et al, 2019;Suk et al, 2016;Wang et al, 2019;Wang et al, 2020) or reduce it to functional connectivity (D'Souza et al, 2019;Fan et al, 2020;Kawahara et al, 2017;Kim and Lee, 2016;Riaz et al, 2020;Seo et al, 2019;Venkatesh et al, 2019;Yang et al, 2019;Zhao et al, 2018). It is also uncertain to what extent representations learned for a specific task would be generalizable to other tasks.…”