Brain experiments imply adaptation mechanisms which outperform common AI learning algorithms

Abstract

Attempting to imitate the brain's functionalities, researchers have bridged between neuroscience and artificial intelligence for decades; however, experimental neuroscience has not directly advanced the field of machine learning (ML). Here, using neuronal cultures, we demonstrate that increased training frequency accelerates the neuronal adaptation processes. This mechanism was implemented on artificial neural networks, where a local learning step-size increases for coherent consecutive learning steps, and tested on a simple dataset of handwritten digits, MNIST. Based on our on-line learning results with a few handwriting examples, success rates for brain-inspired algorithms substantially outperform the commonly used ML algorithms. We speculate this emerging bridge from slow brain function to ML will promote ultrafast decision making under limited examples, which is the reality in many aspects of human activity, robotic control, and network optimization. IntroductionMachine learning is based on Donald Hebb's pioneering work; seventy years ago, he suggested that learning occurs in the brain through synaptic (link) strength modifications (1). A synaptic strength modification typically lasts tens of minutes (2) while the clock speed of a neuron (node) ranges around one second (3). Although the brain is comparatively slow, its computational capabilities outperform typical state-of-the-art artificial intelligence algorithms. Following this speed/capability paradox, we experimentally derive accelerated learning mechanisms based on small datasets, where their utilization on gigahertz processors is expected to lead to ultrafast decision making.Unlike modern computers, a well-defined global clock does not govern brain dynamics; instead, they are a function of relative event timing (e.g., stimulations and evoked spikes) (4).According to neuronal computational, using decaying input summation via its ramified dendritic trees, each neuron sums the asynchronous incoming electrical signals and generates a short electrical pulse (spike) when its threshold is reached. For each neuron, synaptic strength is slowly modified based on the relative timing of inputs from other synapses; if a signal is induced from a synapse without generating a spike, its associated strength is modified based on the relative timing to adjacent spikes from other synapses on the same neuron (5).Recently it was experimentally demonstrated that each neuron functions as a collection of independent threshold units (6). After signals arrive via one of the dendritic trees, each threshold unit is activated. Additionally, a new type of adaptive rule was experimentally observed based on dendritic signal arrival timing (7), which is similar to the slow adaptation mechanism currently attributed to synapses (links). This dendritic adaptation occurs on a faster timescale: it requires approximately five minutes, while synaptic modification requires tens of minutes or more. ResultsIn this study, dendritic adaptation was experimentally examined at a higher stimulation fr...

Power-law scaling to assist with key challenges in artificial intelligence

“…Accelerated strategy: power-law with many epochs. An accelerated BP method is based on a recent new bridge between experimental neuroscience and advanced artificial intelligence learning algorithms, in which an increased training frequency has been able to significantly accelerate neuronal adaptation processes 24 . This accelerated brain-inspired mechanism involves time-dependent step-size, η t , associated with each weight, such that coherent consecutive gradients of weight, that is, with the same sign, increase the conjugate η .…”

“…1a). The presented dataset of examples for the algorithm involves the following initial preprocessing and steps (see Supplementary Appendix A): (a) Balanced set of examples: The small dataset consists of an equal number of random examples per label 24 . (b) Input bias: The bias of each example is subtracted and the standard deviation of its 784 pixels is normalized to unity.…”

Efficient shallow learning as an alternative to deep learning

“…Another possible mechanism is the addition of a super-linear number of cross-weights to the filters. This represents a biological realization because cross-weights result as a byproduct of dendritic nonlinear amplification 17,29,34,35 . Nevertheless, these possible enhanced ρ mechanisms significantly increase computational complexity and are mentioned for their potential biological relevance, limited number of layers, and the natural emergence of many cross-weights.…”

Ideography Leads Us to the Field of Cognition: A Radical-Guided Associative Model for Chinese Text Classification

“…No matter in the early days or now, imitating human cognitive principles has always been the original intention of deep learning (Bezdek 1992;Sardi et al 2020). Initially, the fullyconnected edges designed in artificial neural networks ideally mimic the numerous dendrites of nerve cells.…”