Autoencoder networks extract latent variables and encode these variables in their connectomes

Farrell, Matthew; Recanatesi, Stefano; Reid, R. Clay; Mihalaş, Ştefan; Shea‐Brown, Eric

doi:10.1016/j.neunet.2021.03.010

Cited by 11 publications

(6 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Methods that seek to find the intrinsic dimension (D) of a nonlinear data manifold rather than using the dimension of a linear embedding (L) are an active and promising area of research [15,17,31,34,57,61,81,85,86,[89][90][91][92][93][94]. Our formalism identifies a natural class of low-dimensional manifolds that should exist in neural data.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Common population codes produce extremely nonlinear neural manifolds

Chaudhuri

2022

Preprint

View full text Add to dashboard Cite

Populations of neurons represent sensory, motor and cognitive variables via patterns of activity distributed across the population. The size of the population used to encode a variable is typically much greater than the dimension of the variable itself, and thus the corresponding neural population activity occupies lower-dimensional subsets of the full set of possible activity states. Given population activity data with such lower-dimensional structure, a fundamental question asks how close the low-dimensional data lies to a linear subspace. The linearity or non-linearity of the low-dimensional structure reflects important computational features of the encoding, such as robustness and generalizability. Moreover, identifying such linear structure underlies common data analysis methods such as Principal Component Analysis. Here we show that for data drawn from many common population codes the resulting point clouds and manifolds are exceedingly nonlinear, with the dimension of the best-fitting linear subspace growing at least exponentially with the true dimension of the data. Consequently, linear methods like Principal Component Analysis fail dramatically at identifying the true underlying structure, even in the limit of arbitrarily many data points and no noise.

show abstract

Section: Discussionmentioning

confidence: 99%

“…This covariance matrix C is circulant, meaning that each row is a shifted copy of the row above. It is well known (and easy to show, see Section 4.4) that the eigenvalues of C are given by 6 the Fourier transform of c, the function used to generate each row [4,31,[59][60][61]. Thus, the pth eigenvalue of C is…”

Section: Translation-invariant Tuningmentioning

confidence: 99%

Common population codes produce extremely nonlinear neural manifolds

Chaudhuri

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…The receptive field characteristic of the retina was first discovered by Gilbert et al, which works as a spatiotemporal filter to process the scenes (Gilbert and Wiesel (1992)). Besides, some studies found the nonlinear rectification function embodied in neurons (Lawlor et al (2018);Farrell, Recanatesi, Reid, Mihalas and Shea-Brown (2021)). Based on these findings, a linear-nonlinear Poisson framework, including a linear filter and a nonlinear function, was designed to model the response of ganglion cells.…”

Section: Biophysical Processing-based Frameworkmentioning

confidence: 99%

SpikeSEE: An Energy-Efficient Dynamic Scenes Processing Framework for Retinal Prostheses

Fang¹,

Zou²,

Yang³

2022

Preprint

View full text Add to dashboard Cite

Intelligent and low-power retinal prostheses are highly demanded in this era, where wearable and implantable devices are used for numerous healthcare applications. In this paper, we propose an energy-efficient dynamic scenes processing framework (SpikeSEE) that combines a spike representation encoding technique and a bio-inspired spiking recurrent neural network (SRNN) model to achieve intelligent processing and extreme low-power computation for retinal prostheses. The spike representation encoding technique could interpret dynamic scenes with sparse spike trains, decreasing the data volume. The SRNN model, inspired by the human retina's special structure and spike processing method, is adopted to predict the response of ganglion cells to dynamic scenes. Experimental results show that the Pearson correlation coefficient of the proposed SRNN model achieves 0.93, which outperforms the state-of-the-art processing framework for retinal prostheses. Thanks to the spike representation and SRNN processing, the model can extract visual features in a multiplication-free fashion. The framework achieves 12 times power reduction compared with the convolutional recurrent neural network (CRNN) processing-based framework. Our proposed SpikeSEE predicts the response of ganglion cells more accurately with lower energy consumption, which alleviates the precision and power issues of retinal prostheses and provides a potential solution for wearable or implantable prostheses.

show abstract

“…This is where less restrictive coordinate transformations often have superior dimensionality reduction en route to classification. VAEs have not previously been used in X-ray spectroscopies, although they have been shown to be superior to PCA in several other contexts [91][92][93][94] .…”

Section: Dimensionality Reduction Algorithmsmentioning

confidence: 99%

Unsupervised Machine Learning for Unbiased Chemical Classification in X-ray Absorption Spectroscopy and X-ray Emission Spectroscopy

Tetef

Govind

Seidler

2021

Preprint

View full text Add to dashboard Cite

We report a comprehensive computational study of unsupervised machine learning for extraction of chemically relevant information in X-ray absorption near edge structure (XANES) and in valence-to-core X-ray emission spectra (VtC-XES) for classification of a broad ensemble of sulforganic molecules. By progressively decreasing the constraining assumptions of the unsupervised machine learning algorithm, moving from principal component analysis to a variational autoencoder to t-distributed stochastic neighbor embedding (t-SNE), we find improved sensitivity to steadily more refined chemical information. Surprisingly, even in merely two dimensions, t-SNE distinguishes not just oxidation state and general sulfur bonding environment but the aromaticity of the bonding radical group with 87% accuracy as well as identifying even finer details in electronic structure within aromatic or aliphatic sub-classes. We find that the chemical information in XANES and VtC-XES is very similar, although they exhibit an unexpected tendency to have different sensitivity within a given molecular class.

show abstract

Autoencoder networks extract latent variables and encode these variables in their connectomes

Cited by 11 publications

References 27 publications

Common population codes produce extremely nonlinear neural manifolds

Common population codes produce extremely nonlinear neural manifolds

SpikeSEE: An Energy-Efficient Dynamic Scenes Processing Framework for Retinal Prostheses

Unsupervised Machine Learning for Unbiased Chemical Classification in X-ray Absorption Spectroscopy and X-ray Emission Spectroscopy

Contact Info

Product

Resources

About