Self-organizing neurons: toward brain-inspired unsupervised learning

Abstract

During the last years, Deep Neural Networks have reached the highest performances in image classification. Nevertheless, such a success is mostly based on supervised and off-line learning: they require thus huge labeled datasets for learning, and once it is done, they cannot adapt to any change in the data from the environment. In the context of brain-inspired computing, we apply Kohonen-based Self-Organizing Maps for unsupervised learning without labels, and we explore original extensions such as the Dynamic SOM that enables continuous learning and the Pruning Cellular SOM that includes synaptic pruning in neuromorphic circuits. After presenting the three models and the experimental setup for MNIST classification, we compare different methods for automatic labeling based on very few labeled data (1% of the training dataset), and then we compare the performances of the three Kohonen-based Self-Organizing Maps with STDP-based Spiking Neural Networks in terms of accuracy, dynamicity and scalability.Index Terms-brain-inspired computing, self-organizing maps, unsupervised learning, embedded image classification.

Brain-Inspired Self-Organization with Cellular Neuromorphic Computing for Multimodal Unsupervised Learning

“…However, classification is based on either supervised or semi-supervised learning, as mapping multi-sensory modalities is not sufficient: we need to know the corresponding class to each activation pattern. We proposed in Reference [52] a labeling method summarized in Section 3.1.2 based on very few labeled data, so that we do not use any label in the learning process as explained in Section 3.1. The same approach is used in Reference [48], but the authors rely on the complete labeled dataset, as further discussed in Section 5.4.1.…”

“…In this section, we summarise our previous work on SOM post-labeled unsupervised learning [52], then propose the Reentrant Self-Organizing Map (ReSOM) shown in Figure 2 for learning multimodal associations, labeling one modality based on the other and converge the two modalities with cooperation and competition for a better classification accuracy. We use SOMs and Hebbian-like learning sequentially to perform multimodal learning: first, unimodal representations are obtained with SOMs and, second, multimodal representations develop through the association of unimodal maps via bidirectional synapses.…”

“…The Self-Organizing Map (SOM) proposed by Kohonen [39] is one of the most popular Artificial Neural Networks (ANNs) in the unsupervised learning category [64], inspired from the cortical synaptic plasticity and used in a large range of applications [65] going from high-dimensional data analysis to more recent developments such as identification of social media trends [66], incremental change detection [67] and energy consumption minimization on sensor networks [68]. We introduced in Reference [52] the problem of post-labeled unsupervised learning: no label is available during SOM training then very few labels are available for assigning each neuron the class it represents. The latter is called the labeling phase, which is to distinguish from the fine-tuning process in semi-supervised learning where a labeled subset is used to re-adjust the synaptic weights.…”

“…It can be set to 1 in Algorithm 1. All unimodal trainings were performed over 10 epochs with the same hyper-parameters as in our previous work [52]: for every neuron n in the SOM network do…”

“…The labeling is the step between training and test where we assign each neuron the class it represents in the training dataset. We proposed in Reference [52] a labeling algorithm based on few labeled samples. We randomly took a labeled subset of the training dataset, and we tried to minimize its size while keeping the best classification accuracy.…”