Fuzzy ARTMAP: an adaptive resonance architecture for incremental learning of analog maps

Abstract

A neural network architecture is introduced for incremental supervised learning of recognition categories and multidimensional maps in response to arbitrary sequences of analog or binary input vectors. The architecture, called Fuzzy ARTMAP, achieves a synthesis of fuzzy logic and Adaptive Resonance Theory (ART) neural networks. Fuzzy ARTMAP realizes a new Minimax Learning Rule that conjointly minimizes predictive error and maxi@zes code compression, or generalization. This is achieved by a match tracking process that increases the ART vigilance parameter by the minimum amount needed to correct a predictive error.As a result, the system automatically learns a minimal number of recognition categories, or "hidden units," to meet accuracy criteria. A normalization procedure called complement coding leads to a symmetric theory in which the MIN operator (A) and the MAX operator (v) of fuzzy logic play complementary roles. Improved prediction is achieved by training the system several times using different orderings of the input set, then voting. This voting strategy can also be used to assign probability estimates to competing predictions given small, noisy, or incomplete training sets. Simulations illustrate Fuzzy ARTMAP performance as compared to benchmark back propagation and genetic algorithm systems. These simulations include (i) finding points inside versus outside a circle; (ii) learning to tell two spirals apart; (iii) incremental approximation of a piecewise continuous function; (iv) a letter recognition database; and (v) a medical database.

A neural network model of adaptively timed reinforcement learning and hippocampal dynamics

“…The vigilance parameter is computed in the orienting subsystem A, where it may be increased by punishing events or other unexpected consequences (Carpenter and Grossberg, 1987a;Carpenter, Grossberg, Markuzon, Reynolds, and Rosen, 1991). Vigilance weighs how close the input exemplar I must be to the top-down prototype V in order for resonance to occur.…”

A biologically motivated visual memory architecture for online learning of objects

“…Adaptive resonance (ART) networks are another common approach to perform one-shot and online learning. Many applications of ART and its relative Fuzzy ARTMAP have so far concentrated on representation spaces with much lower dimensionality (Carpenter, Grossberg, Markuzon, Reynolds, & Rosen 1992). The necessity of complement coding (see discussion in Sect.3.2), doubling the input space dimensionality, and problems with sparse vectors make ART networks not very suitable for representing the feature activations of the visual hierarchy we use here.…”

“…One established neuronal network architecture that is able to learn online with the same performance as for offline training is the adaptive resonance theory (ART) and especially Fuzzy ARTMAP (Carpenter, Grossberg, Markuzon, Reynolds, & Rosen 1992). The relation of this network architecture to our short-term memory model will be discussed later (see Sect.…”

“…One major drawback of Fuzzy ART is related to the sparsity of feature vectors, which essentially requires complement coding to avoid that too many adaptive weights become zero. A large amount of zero weights is an unattractive condition for Fuzzy ART networks that should be prevented (Carpenter, Grossberg, Markuzon, Reynolds, & Rosen 1992), because in such a case the "choice function" used for calculating the winner node always results in nearly perfect matches, which results in choosing a winner node independent of the input. Additionally the already very high-dimensional feature vectors are doubled in size by this coding schema.…”

“…Based on the description of our oVQ algorithm the relation to Fuzzy ARTMAP (Carpenter, Grossberg, Markuzon, Reynolds, & Rosen 1992) and Fuzzy ART (Carpenter, Grossberg, & Rosen 1991) will be discussed in the following. Both architectures have the common feature that they can immediately recognize a specific object view after a single occurrence ("one shot learning"), which makes them suitable for online learning.…”