Selecting the architecture of a class of back-propagation neural networks used as approximators

Abstract

This paper examines the architecture of back-propagation neural networks used as approximators by addressing the interrelationship between the number of training pairs and the number of input, output, and hidden layer nodes required for a good approximation. It concentrates on nets with an input layer, one hidden layer, and one output layer. It shows that many of the currently proposed schemes for selecting network architecture for such nets are deficient. It demonstrates in numerous examples that overdetermined neural networks tend to give good approximations over a region of interest, while underdetermined networks give approximations which can satisfy the training pairs but may give poor approximations over that region of interest. A scheme is presented that adjusts the number of hidden layer nodes in a neural network so as to give an overdetermined approximation. The advantages and disadvantages of using multiple output nodes are discussed. Guidelines for selecting the number of output nodes are presented.

Parameter identification by neural network for intelligent deep drawing of axisymmetric workpieces

“…If the undetermined parameters of the network are too few, which will lead to excess fitting to the network, the learning process may easily plunge into poor convergence. Experiments show that good computation results will be obtained when sample quantities have the following relationship with undetermined parameters of the network [17]:…”

Guidelines for the selection of network architecture

“…Different values of these parameters may give different approximations. The dangers of using underdetermined nets as approximators is discussed in some detail by Carpenter and Hoffman (1997). Therefore, in the present study, only overdetermined networks are examined.…”

“…Therefore, in the present study, only overdetermined networks are examined. Carpenter and Hoffman (1997) pointed out that, to make multiple approximations, it is better in general to use multiple neural networks that each have one output node than to use one neural network with multiple output nodes. Thus, in the present study, neural networks with a single output node are used to make one approximation.…”

“…The advantage of using overdetermined polynomial approximations is discussed by Box and Draper (1987), Khuri and Cornell (1987), Myers (1971), andCarpenter (1993). The advantage of using overdetermined neural network approximations has been discussed by Carpenter and Hoffman (1997). If training and test data are expensive or difficult to obtain either computationally or experimentally, then it is desirable to use as few a number of training pairs as possible.…”

“…In other words, they can be used to relate responses, or output variables, to a number of input variables. Considerable research in this area of response surface methodology using neural networks has occurred in recent years (Rumelhart & McClelland, 1986;Anderson & Rosenfeld, 1988;Hornik et al, 1989;Hajela & Berke, 1990;Swift &Batill, 1991;Carpenter &Barthelemy, 1993;Wang etal., 1994;Carpenter & Hoffman, 1995,1997. Likewise, neural networks approximations have been used in all areas of engineering.…”