Subject-invariant feature learning for mTBI identification using LSTM-based variational autoencoder with adversarial regularization

Abstract

Developing models for identifying mild traumatic brain injury (mTBI) has often been challenging due to large variations in data from subjects, resulting in difficulties for the mTBI-identification models to generalize to data from unseen subjects. To tackle this problem, we present a long short-term memory-based adversarial variational autoencoder (LSTM-AVAE) framework for subject-invariant mTBI feature extraction. In the proposed model, first, an LSTM variational autoencoder (LSTM-VAE) combines the representation learning ability of the variational autoencoder (VAE) with the temporal modeling characteristics of the LSTM to learn the latent space representations from neural activity. Then, to detach the subject’s individuality from neural feature representations, and make the model proper for cross-subject transfer learning, an adversary network is attached to the encoder in a discriminative setting. The model is trained using the 1 held-out approach. The trained encoder is then used to extract the representations from the held-out subject’s data. The extracted representations are then classified into normal and mTBI groups using different classifiers. The proposed model is evaluated on cortical recordings of Thy1-GCaMP6s transgenic mice obtained via widefield calcium imaging, prior to and after inducing injury. In cross-subject transfer learning experiment, the proposed LSTM-AVAE framework achieves classification accuracy results of 95.8% and 97.79%, without and with utilizing conditional VAE (cVAE), respectively, demonstrating that the proposed model is capable of learning invariant representations from mTBI data.

An LSTM-based adversarial variational autoencoder framework for self-supervised neural decoding of behavioral choices

“…Cortical activity: The relative GCaMP6s fluorescence calcium signal changes (∆F/F%) were computed for each pixel by subtracting the baseline value and dividing it by the same baseline [10,42]. The baseline for each pixel was determined as the average fluorescence intensity across the first 49 frames.…”

“…These locations were kept the same on frames obtained from all recording sessions. Timeseries from each ROI were obtained by computing the average of pixel intensities within the respective ROI, in each frame [42].…”

“…We now describe the variants of the AE models used for neural data representation learning and feature extraction. For comparison with the considered LSTM-based AVAE model, we also consider simple AE, variational AE (VAE), supervised VAE (SVAE), and the AVAE models [42]. 4(a)) is a neural network that reconstructs the input data X as X, usually in an unsupervised manner.…”

“…However, the cross-subject decoding performance in these methods is limited by subject-specific performance in the source domain. To overcome this limitation, generative adversarial networks [37] combined with transfer learning have shown promise in cross-subject representation learning [38][39][40][41][42]. For example, adversarial training has been used to generalize emotion recognition models across subjects and sessions [43].…”

“…The model iteratively refines the assigned clusters with soft clustering assignment loss, and gradually improves the learned feature representations. To address subject variability in decoding brain activity, a model that combines adversarial learning with a long short-term memory-based adversarial variational autoencoder (LSTM-AVAE) is employed [42]. By using an adversary network to constrain the latent representations, the model learns the information shared among the subjects' brain activity, helping to overcome the problem of individual differences.…”