A survey on universal approximation and its limits in soft computing techniques

Abstract

This paper deals with the approximation behaviour of soft computing techniques. First, we give a survey of the results of universal approximation theorems achieved so far in various soft computing areas, mainly in fuzzy control and neural networks. We point out that these techniques have common approximation behaviour in the sense that an arbitrary function of a certain set of functions (usually the set of continuous function, C) can be approximated with arbitrary accuracy e on a compact domain. The drawback of these results is that one needs unbounded numbers of ''building blocks'' (i.e. fuzzy sets or hidden neurons) to achieve the prescribed e accuracy. If the number of building blocks is restricted, it is proved for some fuzzy systems that the universal approximation property is lost, moreover, the set of controllers with bounded number of rules is nowhere dense in the set of continuous functions. Therefore it is reasonable to make a trade-off between accuracy and the number of the building blocks, by determining the functional relationship between them. We survey this topic by showing the results achieved so far, and its inherent limitations. We point out that approximation rates, or constructive proofs can only be given if some characteristic of smoothness is known about the approximated function.

A Multi-view Comparison of Various Metaheuristic and Soft Computing Algorithms

“…Universal approximator [10] Number of rules grows exponentially for increased accuracy [10] Suffers from existential theorem [18]. Difficulty in interpreting the results of defuzzification process [19].…”

Improving the protection of aquatic ecosystems by dynamically constraining reservoir operation via direct policy conditioning

“…In addition to overcome the dual DP curse of dimensionality and modeling, DPS problems can be solved using multiobjective evolutionary algorithms (MOEAs) to obtain an approximation of the Pareto front in a single run of the algorithm. The e↵ectiveness of this approach depends on the flexibility of the selected class of functions used to define the policy (Tikk et al, 2003) and on the ability of the optimization algorithm to deal with a large number of objectives. In this work, we use gaussian Radial Basis Functions (RBFs) to parameterize the policies as they are capable of representing functions for a large class of problems (Busoniu et al, 2011).…”

Neural Networks and Static Modelling

“…The model produces another error called the test error, which is normally higher than the training error. Tikk et al (2003) and Mhaskar (1996) have provided a detailed survey on the evolution of approximation theory and the use of neural networks for approximation purposes. In both papers, the mathematical background is being provided, which has been used as the background for other researchers who studied the approximation properties of various neural networks.…”