Constructing Multilayer Feedforward Neural Networks to Approximate Nonlinear Functions in Engineering Mechanics Applications

Abstract

This paper presents a major step in the development and validation of a systematic prototype-based methodology for designing multilayer feedforward neural networks to model nonlinearities common in engineering mechanics. The applications of this work include (but are not limited to) system identification of nonlinear dynamic systems and neural-network-based damage detection. In this and previous studies (Pei, J. S., 2001, “Parametric and Nonparametric Identification of Nonlinear Systems,” Ph.D. thesis, Columbia University; Pei, J. S., and Smyth, A. W., 2006, “A New Approach to Design Multilayer Feedforward Neural Network Architecture in Modeling Nonlinear Restoring Forces. Part I: Formulation,” J. Eng. Mech., 132(12), pp. 1290–1300; Pei, J. S., and Smyth, A. W., 2006, “A New Approach to Design Multilayer Feedforward Neural Network Architecture in Modeling Nonlinear Restoring Forces. Part II: Applications,” J. Eng. Mech., 132(12), pp. 1301–1312; Pei, J. S., Wright, J. P., and Smyth, A. W., 2005, “Mapping Polynomial Fitting Into Feedforward Neural Networks for Modeling Nonlinear Dynamic Systems and Beyond,” Comput. Methods Appl. Mech. Eng., 194(42–44), pp. 4481–4505), the authors do not presume to provide a universal method to approximate any arbitrary function. Rather the focus is given to the development of a procedure which will consistently lead to successful approximations of nonlinear functions within the specified field. This is done by examining the dominant features of the function to be approximated and exploiting the strength of the sigmoidal basis function. As a result, a greater efficiency and understanding of both neural network architecture (e.g., the number of hidden nodes) as well as weight and bias values is achieved. Through the use of illuminating mathematical insights and a large number of training examples, this study demonstrates the simplicity, power, and versatility of the proposed prototype-based initialization methodology. A clear procedure for initializing neural networks to model various nonlinear functions commonly seen in engineering mechanics is provided. The proposed methodology is compared with the widely used Nguyen–Widrow initialization to demonstrate its robustness and efficiency in the specified applications. Future work is also identified.

Mapping some functions and four arithmetic operations to multilayer feedforward neural networks

“…The decomposition and pseudo-decomposition techniques 16,17 have enabled the inclusion of more hidden nodes within one hidden layer, while the layer condensation technique substantiated in this study will allow for a further inclusion of one more hidden layer. By using these three techniques, the analytic work and/or neural network prototypes developed by the authors can be concatenated repetitively as basic "building blocks" to form larger and more complex multilayer feedforward neural networks.…”

“…16 Guided by this view of a normalized division operation, Fig. 3(a) illustrates three individual neural networks of small size, labeled Neural Networks 1 to 3, to approximate the two multipliers and multiplication operation individually and the normalized division operation, together.…”

“…For other more complex functions, the authors have proposed using these prototypes in conjunction with decomposition and pseudodecomposition techniques. 16,17 The approximation of a reciprocal function is the key to mapping the division operation to a multilayer feedforward neural network. The three variants of Prototype 3 (labeled 3a, 3b and 3c, each with only two hidden nodes) 16 will be adopted here to approximate a normalized reciprocal function, 1 100p , for a normalized input p ∈ [−1, +1].…”

“…A procedure for the initial design of multilayer feedforward neural networks to approximate a collection of ten types of nonlinear functions 25,26 was developed by the authors, 16 where three prototypes and their variants were proposed and utilized, either individually or combinatorially. In short, the prototypes and their variants are neural networks with clearly defined numbers of hidden nodes and derived/recommended values for the weights and biases based on the approximation capability of linear sums of sigmoidal functions.…”

“…This study is part of a continuing development of a heuristic methodology as presented previously. [11][12][13][14][15][16][17] Presented in this paper is a condensed version of another full length paper by the authors. 18 The target functions being studied by the authors are all for the modeling of nonlinear functions in engineering mechanics applications.…”