Putting a bug in ML: The moth olfactory network learns to read MNIST

Abstract

We seek to (i) characterize the learning architectures exploited in biological neural networks for training on very few samples, and (ii) port these algorithmic structures to a machine learning context. The Moth Olfactory Network is among the simplest biological neural systems that can learn, and its architecture includes key structural elements and mechanisms widespread in biological neural nets, such as cascaded networks, competitive inhibition, high intrinsic noise, sparsity, reward mechanisms, and Hebbian plasticity. These structural biological elements, in combination, enable rapid learning.MothNet is a computational model of the Moth Olfactory Network, closely aligned with the moth's known biophysics and with in vivo electrode data collected from moths learning new odors. We assign this model the task of learning to read the MNIST digits. We show that MothNet successfully learns to read given very few training samples (1 to 10 samples per class). In this few-samples regime, it outperforms standard machine learning methods such as nearestneighbors, support-vector machines, and neural networks (NNs), and matches specialized one-shot transfer-learning methods but without the need for pretraining.The MothNet architecture illustrates how algorithmic structures derived from biological brains can be used to build alternative NNs that may avoid some of the learning rate limitations of current engineered NNs.

A spiking neural program for sensorimotor control during foraging in flying insects

“…The ability of insects to quickly form associative memories after three to five trials has been demonstrated experimentally (47). However, in general, fewshot learning remains a difficult task for computational models, including insect-inspired models (62). We find that, when compared with learning-dynamics data of real insects (47), our model is able to show realistic learning dynamics that matches with the experimental observations.…”

A spiking neural program for sensory-motor control during foraging in flying insects

“…The ability of insects to quickly form associative memories after 3-5 trials has been demonstrated experimentally (44). However, in general fewshot learning remains a difficult task for computational models including insect inspired models (54). We find that, when compared with learning dynamics data of real insects (44) our model is able to show realistic learning dynamics that matches with the experimental observations.…”

A Spike Time-Dependent Online Learning Algorithm Derived From Biological Olfaction

“…Models of olfactory bulb and piriform cortical activity have been applied to analyze chemosensor array data (Raman and Gutierrez-Osuna, 2005; Raman et al, 2006). Algorithms based on the insect olfactory system have been employed to learn and identify odor-like inputs (Diamond et al, 2016; Delahunt et al, 2018) as well as to identify handwritten digits—visual inputs incorporating additional low-dimensional structure (Huerta and Nowotny, 2009; Delahunt and Kutz, 2018; Diamond et al, 2019). More broadly, insect mushroom bodies in particular have been deeply studied in terms of both their pattern separation and associative learning capacities (Hige, 2018; Cayco-Gajic and Silver, 2019).…”