Multi-scale neural decoding and analysis

Abstract

Objective. Complex spatiotemporal neural activity encodes rich information related to behavior and cognition. Conventional research has focused on neural activity acquired using one of many different measurement modalities, each of which provides useful but incomplete assessment of the neural code. Multi-modal techniques can overcome tradeoffs in the spatial and temporal resolution of a single modality to reveal deeper and more comprehensive understanding of system-level neural mechanisms. Uncovering multi-scale dynamics is essential for a mechanistic understanding of brain function and for harnessing neuroscientific insights to develop more effective clinical treatment. Approach. We discuss conventional methodologies used for characterizing neural activity at different scales and review contemporary examples of how these approaches have been combined. Then we present our case for integrating activity across multiple scales to benefit from the combined strengths of each approach and elucidate a more holistic understanding of neural processes. Main results. We examine various combinations of neural activity at different scales and analytical techniques that can be used to integrate or illuminate information across scales, as well the technologies that enable such exciting studies. We conclude with challenges facing future multi-scale studies, and a discussion of the power and potential of these approaches. Significance. This roadmap will lead the readers toward a broad range of multi-scale neural decoding techniques and their benefits over single-modality analyses. This Review article highlights the importance of multi-scale analyses for systematically interrogating complex spatiotemporal mechanisms underlying cognition and behavior.

Multimodal, Multiscale Insights into Hippocampal Seizures Enabled by Transparent, Graphene-Based Microelectrode Arrays

“…Performing simultaneous, multimodal measurements of neural activity remains a substantial challenge in neuroscience. Combining modalities can provide different perspectives on, and thus a more comprehensive view of, neural activity ( Lu et al, 2021 ). Moreover, multimodal measurements can overcome the drawbacks of single modalities and enable observations across scales, from single cells to networks.…”

“…While our implant strategy enabled observations of seizures at the levels of single PCs, ensembles of cells, and the CA1 network, relating observations of neural activity across scales remains a substantial conceptual problem ( Lu et al, 2021 ). Observing neural activity across scales allows for a more complete picture of the neural processes behind phenomena like seizures.…”

Multimodal subspace identification for modeling discrete-continuous spiking and field potential population activity

“…Thus, building dynamical models that simultaneously incorporate multiple observation modalities is important for revealing how different spatiotemporal scales of neural population dynamics explain behavior. Further, such modeling can aggregate information across different spatiotemporal scales of neural activity to improve the performance of brain-machine interfaces (BMIs) [43,44,[48][49][50]. We refer to dynamical modeling with multimodal observations as multiscale dynamical modeling.…”

Multi-block RNN Autoencoders Enable Broadband ECoG Signal Reconstruction

“…Field potentials like ECoG, in particular, are potentially challenging to model because they are spatially and temporally rich. ECoG contains broadband information spanning a wide range of frequencies at varying power scales, and represents larger spatial scales than spiking activity while also capturing highly local spiking activity [13, 3, 14]. These properties also make ECoG a promising modality for neural engineering applications [15, 16].…”

“…Recordings from a given modality exhibit nonlinear dynamics [1] and non-stationary signal statistics [2]. Each neural data modality has different statistical properties [3], which must be accounted for in neural analyses and neural information decoding methods.…”