Modern machine learning outperforms GLMs at predicting spikes

Abstract

15Neuroscience has long focused on finding encoding models that effectively ask "what predicts neural 16 spiking?" and generalized linear models (GLMs) are a typical approach. It is often unknown how much of 17 explainable neural activity is captured, or missed, when fitting a GLM. Here we compared the predictive 18 performance of GLMs to three leading machine learning methods: feedforward neural networks, gradient 19 boosted trees (using XGBoost), and stacked ensembles that combine the predictions of several methods. 20We predicted spike counts in macaque motor (M1) and somatosensory (S1) cortices from standard 21 representations of reaching kinematics, and in rat hippocampal cells from open field location and orientation. 22In general, the modern methods (particularly XGBoost and the ensemble) produced more accurate spike 23 predictions and were less sensitive to the preprocessing of features. This discrepancy in performance 24suggests that standard feature sets may often relate to neural activity in a nonlinear manner not captured by 25GLMs. Encoding models built with machine learning techniques, which can be largely automated, more 26 accurately predict spikes and can offer meaningful benchmarks for simpler models.

Brain-state invariant thalamo-cortical coordination revealed by non-linear encoders

“…Using a supervised classifier, so-called gradient boosting [Benjamin et al, 2017, Truccolo andDonoghue, 2007], we show how this method can determine an encoding model for predicting population spike trains knowing the stimulus input. We also show, in line with recently published work [Benjamin et al, 2017], how gradient boosted trees (XGB) can also be used as a very efficient decoding model that is retrieving the stimulus likelihood knowing the spiking activity of a population of neurons. Finally, we demonstrate how it generates a very accurate encoding model for predicting a population spike train conditioned on another, anatomically projected, set of neuronal activity [Harris et al, 2003].…”

“…where T is the total number of leaves and w j the score of leaf j. γ andλ are two free parameters weighting the contribution of the two previous items in the objective function. For the sake of comparison with a related study Benjamin et al [2017], we used the same values: γ = 0.4 and λ = 0.0. However, in the following section detailing the methods, we keep these two parameters as variables.…”

“…A large class of neurons in the brain are modulated by several dimensions of incoming stimuli [Finkelstein et al, 2015, Hardcastle et al, 2017, Rigotti et al, 2013, Sargolini et al, 2006, a property referred to as mixed-selectivity. Untangling the different contributions is sometimes challenging and gradient boosted trees offer a rapid and unequivocal approach to address this issue Benjamin et al [2017], Truccolo and Donoghue [2007]. In fact, there is no intrinsic limit to the dimensionality of the inputs that can be learned.…”

Brain-state invariant thalamo-cortical coordination revealed by non-linear encoders

“…This lack of ground-truth data when performing data analysis is particularly unavoidable in neuroscience [ 25 ]. It has thus become necessary to establish standard, model-free methods that, even if they do not contribute to our understanding of the data, set levels of performance that may be used to benchmark model-based approaches [ 26 , 27 ]. Machine Learning provides a large array of techniques to classify datasets that have demonstrated high level of performance in fields ranging from image processing to astrophysics [ 28 ].…”

Pyglmnet: Python implementation of elastic-net regularized generalized linear models

“…Pyglmnet has already been used in published work (Benjamin et al, 2017;Bertrán et al, 2018;Höfling, Berens, & Zeck, 2019;Rybakken, Baas, & Dunn, 2019). It contains unit tests and includes documentation in the form of tutorials, docstrings and examples that are run through continuous integration.…”