Encoding a priori information in feedforward networks

Joerding, Wayne; Meador, J.

doi:10.1016/0893-6080(91)90063-b

Cited by 55 publications

(23 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Unfortunately, it has also been common practise that the lack of transparency of such b l a c k-box a p p r o a c hes has led to neglection of the available prior knowledge, and the resulting ill-conditioning has been handled implicitly using regularization methods like stopped training Ljung 1992, Ljung, Sj oberg andMcKelvey 1993). However, simple explicit regularization methods like default models (Thompson and Kramer 1994, Su et al 1992, Kramer et al 1992, Johansen and Foss 1992c, constraints (Joerding andMeador 1991, Thompson andKramer 1994), penalty on parameter magnitude (Weigend, Huberman and Rumelhart 1990), and smoothness regularization (Bishop 1991, Girosi et al 1994, has also been suggested. The optimization framework presented here will be useful for regularizing such complex parameter identi cation problems, and is useful as a complementary technique applied together with other approaches such as neural networks, in order to reduce the sensitivity with respect to the a priori choice of model structure.…”

Section: Discussionmentioning

confidence: 99%

Operating regime based process modeling and identification

Johansen

Foss

1997

Computers & Chemical Engineering

136

View full text Add to dashboard Cite

SummaryThe development of mathematical models is a major bottleneck for the application of advanced model-based techniques for control, optimization, scheduling, automatic fault detection and diagnosis in the process industries. Hence, there is a potential for improved product quality and pro tability, a s w ell as improved production exibility and safety if the cost of model development can be reduced. In this thesis we address some aspects of non-linear modeling and identi cation using a combination of empirical process data and prior knowledge. The major part of this thesis is concerned with a modeling framework based on an operating regime decomposition of the system's operating range. Within each operating regime, the system is modeled with a simple local model. The local models are patched together using a smooth interpolation technique. This framework supports the development of transparent semi-empirical or semi-mechanistic models, and is exible with respect to the prior knowledge and empirical data required. We argue that the cost of model development c a n b e l o w, yet the quality of the model can be high, through the application of this modeling framework, in some cases. These cases are characterized by a limited amount of process knowledge, and the availability of a reasonable amount of process data. Identi cation of model structure and parameters on the basis of process data in this framework is discussed in detail. The properties of the modeling framework is illustred with some semi-realistic examples, both simulated and experimental. In addition, the applicability of the modeling framework for model based control is investigated both through simulation examples and analysis. A minor part of this thesis is an optimization formulation of the modeling and identi cation problem, where the idea is to minimize a criterion that penalizes mismatch b e t ween model behavior and empirical data, and inconsistency with the prior knowledge. The motivation is that in some cases, the application of prior knowledge may be simpler and more transparent than through the direct speci cation of a parameterized model structure, which is the traditional approach. In addition, this thesis contains some theoretical results of more general interest. Some of these results are related to properties of some model structure identi cation criteria, while others are related to the stability and robustness of adaptive control loops based on non-linear models.iii iv

show abstract

Section: Discussionmentioning

confidence: 99%

Operating regime based process modeling and identification

Johansen

Foss

1997

Computers & Chemical Engineering

136

View full text Add to dashboard Cite

show abstract

“…Constraints involving inputs are called infinite constraints because they must hold for all (infinitely many) possible input combinations. Joerding and Meador (1991) transformed infinite constraints into finite constraints, which involve only the network weights. In this way, they enforced monotonicity, convexity, or concavity on a network output with respect to the network inputs.…”

Section: Introductionmentioning

confidence: 99%

“…The network estimated the specific growth rate, which was input into the component mass balances. Joerding and Meador (1991) used the parametric model as a normalization post-processor to force the outputs of the network to sum to one, which is important when estimating quantities such as component mole fractions in mixtures or the probabilities of mutually exclusive events.…”

Section: Introductionmentioning

confidence: 99%

Modeling chemical processes using prior knowledge and neural networks

1994

View full text Add to dashboard Cite

“…These conditions can come from information or beliefs about the data generating process a network seeks to model, Joerding and Meador (1991), Joerding, Li, Hu, and Meador (1992), or may arise from their ability to improve network generalization, Bishop (1990), Psaltis and Neifield (1988). Geman, Bienenstock, and Doursat (1992) argues for imposing a priori constraints on neural networks as a way to circumvent the tradeoff between bias and variance in nonparametric estimation.…”

Section: Introductionmentioning

confidence: 98%

Global estimation of feedforward networks with a priori constraints

Joerding

Liu²

1994

Comput Econ

View full text Add to dashboard Cite

Abstract. This paper presents a feedforward network estimation algorithm that addresses two issues, (i) avoiding local inferior minima to the performance criteria, and (ii) imposing a priori constraints to improve generalization and test economic hypotheses. The algorithm combines methods either previously developed or obviously beneficial but not yet combined. These involve combining linear least squares with simulated annealing, along with weight space reducing methods, to considerably improve its speed relative to pure simulated annealing. We present evidence on the algorithm's reliability at finding a global minimum. We also demonstrate how to constrain the estimation process to find networks that satisfy a given a priori condition. We provide examples of imposing a priori information to (i) prevent the trained network from making fundamental errors and (ii) to test economically interesting hypotheses.

show abstract

Encoding a priori information in feedforward networks

Cited by 55 publications

References 6 publications

Operating regime based process modeling and identification

Operating regime based process modeling and identification

Modeling chemical processes using prior knowledge and neural networks

Global estimation of feedforward networks with a priori constraints

Contact Info

Product

Resources

About