Additive Fuzzy Systems: From Generalized Mixtures to Rule Continua

Abstract

A generalized probability mixture density governs an additive fuzzy system. The fuzzy system's if‐then rules correspond to the mixed probability densities. An additive fuzzy system computes an output by adding its fired rules and then averaging the result. The mixture's convex structure yields Bayes theorems that give the probability of which rules fired or which combined fuzzy systems fired for a given input and output. The convex structure also results in new moment theorems and learning laws and new ways to both approximate functions and exactly represent them. The additive fuzzy system itself is just the first conditional moment of the generalized mixture density. The output is a convex combination of the centroids of the fired then‐part sets. The mixture's second moment defines the fuzzy system's conditional variance. It describes the inherent uncertainty in the fuzzy system's output due to rule interpolation. The mixture structure gives a natural way to combine fuzzy systems because mixing mixtures yields a new mixture. A separation theorem shows how fuzzy approximators combine with exact Watkins‐based two‐rule function representations in a higher‐level convex sum of the combined systems. Two mixed Gaussian densities with appropriate Watkins coefficients define a generalized mixture density such that the fuzzy system's output equals any given real‐valued function if the function is bounded and not constant. Statistical hill‐climbing algorithms can learn the generalized mixture from sample data. The mixture structure also extends finite rule bases to continuum‐many rules. Finite fuzzy systems suffer from exponential rule explosion because each input fires all their graph‐cover rules. The continuum system fires only a special random sample of rules based on Monte Carlo sampling from the system's mixture. Users can program the system by changing its wave‐like meta‐rules based on the location and shape of the mixed densities in the mixture. Such meta‐rules can help mitigate rule explosion. The meta‐rules grow only linearly with the number of mixed densities even though the underlying fuzzy if‐then rules can have high‐dimensional if‐part and then‐part fuzzy sets.

Uniform Mixture Convergence of Continuously Transformed Fuzzy Systems

“…The jth rule associates the then-part fuzzy set B j with the if-part fuzzy set A j . The fuzzy system's corresponding governing probability mixture p n (y|x) mixes m rule likelihood probabilities p Bj (y|x) with m convex mixing weights or prior densities p j (x): p n (y|x) = p 1 (x)p B1 (y|x) + • • • + p m (x)p Bm (y|x) [12]. The fuzzy systems F n can converge to some sampled neural classifier or to any other black-box approximator.…”

“…Figure 5 shows how randomly sampling from the Gaussian fuzzy system's trained 2-bell-curve mixture p(y|x) can reproduce the transformed target function sin 2 (x) through Monte Carlo averaging. This amounts to drawing a finite number of new if-then rules for each input x from a virtual rule continuum [12]. The rule histograms in Fig.…”

“…The new mixture convergence theorem exploits the fact that a generalized probability mixture p(y|x) of just two Gaussian bell curves can exactly represent any bounded real function f as the average of the mixture: [12,13]. The conditional expectation E[Y |X = x] integrates or sums with respect to the conditional Gaussian mixture p(y|x):…”

“…The weights w j may depend on x or on other quantities in some applications. Then normalized rule firings define a generalized probability mixture [12] because we assume that the then-part set functions are nonnegative and integrable:…”

“…Figure 5 shows how such virtual rules can estimate the transformed target function sin 2 x by Monte Carlo averaging. The mixture becomes a compound in this case because the discrete rule index j becomes the continuous θ [12]:…”