Synapse maintenance in the Where-What Networks

The 2011 International Joint Conference on Neural Networks

2011

Self Cite

Abstract-Finite Automata (FA) is a base net for many sophisticated probability-based systems of artificial intelligence. However, an FA processes symbols, instead of images that the brain senses and produces (e.g., sensory images and motor images). Of course, many recurrent artificial neural networks process images. However, their non-calibrated internal states prevent generalization, let alone the feasibility of immediate and error-free learning. I wish to report a general-purpose Developmental Program (DP) for a new type of, brain-anatomy inspired, networks -Developmental Networks (DNs). The new theoretical results here are summarized by three theorems. (1) From any complex FA that demonstrates human knowledge through its sequence of the symbolic inputs-outputs, the DP incrementally develops a corresponding DN through the image codes of the symbolic inputs-outputs of the FA. The DN learning from the FA is incremental, immediate and error-free. (2) After learning the FA, if the DN freezes its learning but runs, it generalizes optimally for infinitely many image inputs and actions based on the embedded inner-product distance, state equivalence, and the principle of maximum likelihood. (3) After learning the FA, if the DN continues to learn and run, it "thinks" optimally in the sense of maximum likelihood based on its past experience.

Section: Resultsmentioning

confidence: 99%

Three theorems: Brain-like networks logically reason and optimally generalize

The 2011 International Joint Conference on Neural Networks

2011

Self Cite

“…In the second experiment, DN is applied to a set of local templates is derived from two object recognition datasets, NORB and CIFAR-10 using the idea of local patch extraction from foreground in [11] and the recognition rate is compared to some major local-feature learning algorithms [5], [6], [8], [9]. The NORB dataset with elimination of complex background [13] is a set of images of 50 toys, of size 96 × 96, belonging to 5 classes namely, four-legged animals, human figures, airplanes, trucks and cars, imaged by two cameras under 6 lighting conditions, 9 elevations and 18 azimuths, of which 24300 images were used for training and testing each.…”

Section: B Local Template Based Methodsmentioning

confidence: 99%

“…The DN (Developmental Network) framework [10] has used global templates from image datasets [10], as well as local templates from cluttered scenes [11].…”

Section: B Characteristics Of Dnmentioning

confidence: 99%

“…Second, relatively higher variation through a synapse gives information for cellular synaptic pruning in all neurons, to delete their links to irrelevant components. This was used in synapse maintenance [11], but this capability has been turned off because the public image datasets, that we use here, do not present variation of background pixels like a dynamic physical world would.…”

Section: E Properties Of Dnmentioning

confidence: 99%

See 1 more Smart Citation

Developing dually optimal LCA features in sensory and action spaces for classification

Wagle

2012 IEEE International Conference on Development and Learning and Epigenetic Robotics (ICDL)

2012

Self Cite

Abstract-Appearance based methods have utilized a variety of techniques such as Linear Discriminant Analysis (LDA), Support Vector Machine (SVM), k-means clustering, and sparse autoencoders to extract training-data dependent features from static images -by static images, we mean, foreground objects and backgrounds are present in static images as snapshots. The Developmental Networks (DN) use Lobe Component Analysis (LCA) features developed not only from the image space X but also the action space Z. Since, Z can be taught to represent a set of trainer specified meanings (e.g., type and location), a DN treats these meanings in a unified way for both detection and recognition for objects in dynamic cluttered backgrounds. However, the DN method has not been applied to publicly available datasets and compared with well-known major techniques. In this work, we fill this void. We describe how the Z information enables the features to be more sensitive to trainer specified output meanings (e.g., type and location). The reported experiments fall into two extensively studied categories -global template based object recognition and local template based scene classification. For the datasets used, the performance of the DN method is better or comparable to some major local template based methods but the DNs also provide statistics-based location information about the object in a cluttered scene.

“…The mechanism of synaptic maintenance was first proposed to automatically adapt the shape of receptive field of a neuron. The result shows that the adaptive receptive fields can reduce the interference from leakedin background pixels which have larger fluctuations than the foreground pixels [12]. Then the synaptic maintenance was extended from sensory domain to multiple domains in WWN-9 and was named cross-domain synaptic maintenance.…”

Section: Introductionmentioning

confidence: 99%

Cross-domain and within-domain synaptic maintenance for autonomous development of visual areas

Guo

2015 Joint IEEE International Conference on Development and Learning and Epigenetic Robotics (ICDL-EpiRob)

2015

Self Cite

Abstract-Where-What Networks (WWNs) is a series of developmental networks for the recognition and attention of complex visual scenes. One of the most critical challenges of autonomous development is task non-specificity, namely, the network is meant to learn a variety of open-ended task skills without pre-defined tasks. Then how does a brain-like network develop skills for object relation that can generalize using implicit symbol-like rules? A preliminary scheme of uniform synaptic maintenance, which works across a neuron's sensory and motor domains, has been proposed in our WWN-9. In the new work here, we show that cross-domain and within-domain synaptic maintenance gains superior generalization than using the uniform synaptic maintenance scheme. This generalization enables the WWN to automatically discover symbol-like but implicit rules -detecting object groups from new combinations of object locations that were never observed. By "symbol-like but implicit rules", we mean that the development program has no symbols and explicit rules, but symbol-like concepts (location, type) and implicit rule (two specific type objects must present concurrently -group) emerge as the firing patterns of the motor area and are used by the control. Moreover, the process of synaptic maintenance corresponds to the genesis (and adaptation) of cell connections and our model autonomously develops the Y area into two subarea, early area and later area, in charge of pattern recognition and symbolic reasoning respectively.