Brain dynamics and temporal trajectories during task and naturalistic processing

Abstract

Human functional Magnetic Resonance Imaging (fMRI) data are acquired while participants engage in diverse perceptual, motor, cognitive, and emotional tasks. Although data are acquired temporally, they are most often treated in a quasi-static manner. Yet, a fuller understanding of the mechanisms that support mental functions necessitates the characterization of dynamic properties. Here, we describe an approach employing a class of recurrent neural networks called reservoir computing, and show the feasibility and potential of using it for the analysis of temporal properties of brain data. We show that reservoirs can be used effectively both for condition classification and for characterizing lower-dimensional "trajectories" of temporal data. Classification accuracy was approximately 90% for short clips of "social interactions" and around 70% for clips extracted from movie segments. Data representations with 12 or fewer dimensions (from an original space with over 300) attained classification accuracy within 5% of the full data. We hypothesize that such low-dimensional trajectories may provide "signatures" that can be associated with tasks and/or mental states. The approach was applied across participants (that is, training in one set of participants, and testing in a separate group), showing that representations generalized well to unseen participants. Taken together, we believe the present approach provides a promising framework to characterize dynamic fMRI information during both tasks and naturalistic conditions.

Using Low-Dimensional Manifolds to Map Relationships Between Dynamic Brain Networks

“…Another weakness of this study is that we did not directly compare our approach to other representations created from fMRI time series. For example, representations generated from the fMRI time series using reservoir computing have been shown to discriminate between 2-back and 0-back task conditions with 77-81% accuracy (Venkatesh et al, 2019). Nevertheless, we feel that the brain networks reveal important neural processes that are captured only by examining the relationships between brain regions.…”

“…There is a growing interest in studying and visualizing brain dynamics in low-dimensional space, but most studies have directly examined the fMRI time series, rather than dynamic networks. Principal components analysis (PCA) has been used to reduce the dimensionality of the raw fMRI data (Shine et al, 2019), and reservoir computing (Venkatesh et al, 2019) has been used to examine temporal state transitions in the raw fMRI data. Both of these studies demonstrated that the patterns of dynamic transitions were unique for distinct task conditions.…”

Neural dynamics of emotion and cognition: From trajectories to underlying neural geometry

“…If the top three dimensions are selected, the evolution of the reservoir can be plotted in this lower-dimensional space. In the present case, the trajectories originated from functional MRI data during the viewing of short movie clips, which were either "scary" or "funny" (Venkatesh et al, 2019). In the example, the trajectories separated quite well.…”

“…But can trajectories be learned from neural data? In this section, we describe an approach to learning temporal information that we recently developed in the context of functional MRI data (Venkatesh et al, 2019).…”

“…However, effectively training RNNs can be challenging, particularly without large amounts of data (Pascanu et al, 2013); but for recent developments see (Martens and Sutskever, 2011;Graves et al, 2013). In our study (Venkatesh et al, 2019), we proposed to use reservoir computing to study temporal properties of brain data. This class of algorithms, which includes liquid-state machines (Maass The reservoir layer contains units with random connections, and provides a memory mechanism such that activation at time t is influenced by past time points.…”