ELVISort: encoding latent variables for instant sorting, an artificial intelligence-based end-to-end solution

Abstract

Objective. The growing number of recording sites of silicon-based probes means that an increasing amount of neural cell activities can be recorded simultaneously, facilitating the investigation of underlying complex neural dynamics. In order to overcome the challenges generated by the increasing number of channels, highly automated signal processing tools are needed. Our goal was to build a spike sorting model that can perform as well as offline solutions while maintaining high efficiency, enabling high-performance online sorting. Approach. In this paper we present ELVISort, a deep learning method that combines the detection and clustering of different action potentials in an end-to-end fashion. Main results. The performance of ELVISort is comparable with other spike sorting methods that use manual or semi-manual techniques, while exceeding the methods which use an automatic approach: ELVISort has been tested on three independent datasets and yielded average F1 scores of 0.96, 0.82 and 0.81, which comparable with the results of state-of-the-art algorithms on the same data. We show that despite the good performance, ELVISort is capable to process data in real-time: the time it needs to execute the necessary computations for a sample of given length is only 1/15.71 of its actual duration (i.e. the sampling time multiplied by the number of the sampling points). Significance. ELVISort, because of its end-to-end nature, can exploit the massively parallel processing capabilities of GPUs via deep learning frameworks by processing multiple batches in parallel, with the potential to be used on other cutting-edge AI-specific hardware such as TPUs, enabling the development of integrated, portable and real-time spike sorting systems with similar performance to offline sorters.

Spike sorting: new trends and challenges of the era of high-density probes

“…The last few years have witnessed a boom in the efforts of using deep learning approaches to tackle the spike sorting problem [20,44,48,64,66,76,87,95,97,98]. Deep learning methods, in fact, have proven so powerful and accurate in many complicated applications, ranging from image classification to natural language processing.…”

“…However, instead of splitting the detection and sorting phase, they used a single 1D CNN architecture to directly detect and sort spikes in the input recording. Racz et al [66] proposed a different architectures that uses a CNN+LSTM-based autoencoder architecture to find a latent space and dense output networks to perform the spike sorting task.…”

From End to End: Gaining, Sorting, and Employing High-Density Neural Single Unit Recordings

“…Unless virtually infinite storage capacity is in our hands, compression is what we should make use of. A more compact dataset is not just efficiently, but also speeds up the computational process (Rokai et al, 2021 ). Several methods have been described for data reduction up to a four-fold rate, including pure compression (Pagin and Ortmanns, 2018 ), thresholded signal transmission (Irwin et al, 2016 ), or the lately introduced on-chip spike sorting procedures (Saeed et al, 2017 ; Xu et al, 2019 ).…”

“…Sequentially constructed algorithms, such as those building upon multiple basic dense layers (Mahallati et al, 2019 ; Yeganegi et al, 2020 ) and convolutional (Li et al, 2020b ) and recurrent layers (Rácz et al, 2020 ) require an expansive repository, although by weights' and activation functions' binarization, complexity may be cut back (Valencia and Alimohammad, 2021 ), or parallelization by graphical processing units may take place (Tam and Yang, 2018 ). These layers may be constructed in different ways, mainly in order to mitigate or abandon the need for hand-labeled neural data throughout training: autoencoders (Weiss, 2019 ; Radmanesh et al, 2021 ; Rokai et al, 2021 ) or networks generated by adversarial (Wu et al, 2019 ; Ciecierski, 2020 ) or reinforcement learning paradigms (Salman et al, 2018 ; Moghaddasi et al, 2020 ) have successfully clustered features originating from noisiest datasets. Likewise, a more sophisticated learning-based method may even incorporate multiple steps of spike sorting, resolving detection, feature extraction, and clustering as a close-packed solution (Eom et al, 2021 ; Rokai et al, 2021 ), although manual curation is advisable (Horváth et al, 2021 ).…”

“…These layers may be constructed in different ways, mainly in order to mitigate or abandon the need for hand-labeled neural data throughout training: autoencoders (Weiss, 2019 ; Radmanesh et al, 2021 ; Rokai et al, 2021 ) or networks generated by adversarial (Wu et al, 2019 ; Ciecierski, 2020 ) or reinforcement learning paradigms (Salman et al, 2018 ; Moghaddasi et al, 2020 ) have successfully clustered features originating from noisiest datasets. Likewise, a more sophisticated learning-based method may even incorporate multiple steps of spike sorting, resolving detection, feature extraction, and clustering as a close-packed solution (Eom et al, 2021 ; Rokai et al, 2021 ), although manual curation is advisable (Horváth et al, 2021 ).…”