Statistical analysis of constrained data sets

Abstract

Statistical Analysis of Constrained Data SetsPractical, comprehensive computer-based methods for analyzing data sets are developed. Methods for calculating data adjustment, parameter values, variance-covariance of adjusted data and parameters and for detecting aberrant data are presented. A simple calculation algorithm and application of the methods for design of experimental measurements are proposed. JAY Research and Development Napenille, IL 60540Least squares analysis, commonly used to fit regression equations to sets of experimental data, also provides powerful techniques for analyzing the measured data themselves, when these data can be interrelated through constraining physical laws. Least squares parameter estimation problems are typically "parameter-rich in the sense that the data set is constrained by a single regression equation. (In this article, the term "parameter" is used in the engineering sense of a quantity to be estimated from other data, rather than in the statistical sense, wherein it applies to all estimated quantities.)In another frequently met situation of particular interest in this work, a large set of data is interrelated by physical laws such as heat balances, material balances or kinetic equations. This situation may be thought of as being "comtraint-rich", and is typical of many laboratory experiments, pilot unit tests, and commercial unit performance tests. Constraints, together with estimates of the variances of the measured data, can be used to adjust data to more accurate values and to draw conclusions about their credibility.The data estimation techniques discussed here greatly facilitate analysis of constrained data sets by providing maximum likelihood estimates of the meawred data and any parameters, by assessing the probability that there are extraordinary errors, and by providing error information about the calculated quantities for use in subsequent analysis. Further, the techniques can be used to develop experiments producing data with improved accuracy. The method can be thought of as an extension of the averaging process to situations, in which each measured quantity may enter into one or a number of physical constraints, and by which alternative values can be inferred. It can be applied, if necessary, to detect gross measurement errors and to isolate any such by using data redundancies, much as a skilled analy$t would. And, replicates or near replicates are not required to judge credibility of the data values.The net effect can be significantly increased efficiency of experimentation, offering the happy choice of either obtaining more accurate data for a given cost, or of achieving the same accuracy in the final results with less experimental cost. Because the techniques are general, they can handle data sets described as a typical linear or nonlinear regression problem, a parameter-free data adjustment problem, or any combination of these.The basic data correction algorithm was first derived for and applied to surveying problems (Demming 1946). Data adjustment techniq...

Data reconciliation and parameter estimation in plant performance analysis

“…The rigorous and constant-direction methods of Knepper and Gorman (1980) were also tested. The rigorous method is identical to that of Britt and Leucke.…”

“…Plant performance analysis was first considered by Kuehn and Davidson (1961). Britt and Leucke (1973) and Knepper and Gorman (1980) provided an algorithm that can be used to adjust plant data to meet the constraints. In the last ten years, considerable effort has been expended to develop procedures to handle very large flowsheets.…”

Application of broyden's method to reconciliation of nonlinearly constrained data

“…Therefore, the computing time for step 2 will likely be prohibitively large. This prompted Knepper and Gorman (1980) to use old values of derivatives until the constraints are satisfied. However, this approach leads to slow convergence rate and possibly a small radius of convergence.…”

“…Unfortunately, this procedure is too specific to be useful for more general cases. Knepper and Gorman (1980) proposed a Gauss-Newton iterative algorithm and, in order to reduce computational effort, suggested using old derivatives of constraints until the constraints are satisfied (i.e., the constant-direction approach). HLowever, their algorithm is limited to problems with no more constraints than measured variables and the constant-direction approach is characterized by slow convergence.…”

Several techniques to detect and identify systematic biases when process constraints are bilinear

“…Several p^ers have described ways to improve computational speed, which would be especially beneficial for large process networks (Britt and Leucke, 1973;Knepper and Gorman, 1980;Stephenson and Shewchuk, 1986;Pai and Fisher, 1988;Tjoa and Biegler, 1991).…”