Gross errors detection of industrial data by neural network and cluster techniques

Abstract

-This article describes the application of a three-layer feed-forward neural network to analyze industrial plant data. To adjust mathematical models (for control or optimization purposes) from plant data, it is necessary to analyze and detect outliers and systematic errors and to remove them. The system studied is the feed preparation of an isoprene production unit and represents a multivariable problem. To detect outliers in a multivariable system is not an easy task. The technique used in this paper is able to identify this kind of error. The methodology employed involves construction of a reliable neural network model to represent the process and its training with a few iterations (a few thousand). Thus, the points at which errors between the experimental and calculated data appear to be scattered far from the majority of the values are probably outliers. In some cases, outlier points can be easily detected, but in others, they are not so obvious. In these cases, they are separated and a cluster with other similar data is built. After analyzing these clusters based on the similarity principle or by hypothesis tests for means, it is then decided whether or not these points can be excluded. At the same time the process is checked for any abnormalities recorded during the specific period. Three year's worth of process data were analyzed and about 30% of the data were excluded.

Knowledge in Imperfect Data

“…This is why, firstly, this operation focuses on identifying the outliers within the set of the collected data. In the literature, one can find reference to numerous techniques of identifying pieces of the data which are classified as outliers (Alves & Nascimento, 2002;Ben-Gal, 2005;Cateni et al, 2008;Fan et al, 2006;Mohamed et al, 2007). This is an important issue, since it is only in the case of outliers that their removal from the set may be justified.…”

Analysis and Detection of Outliers and Systematic Errors in Industrial Plant Data

“…Outliers -although different from the majority of the sample -may be indicative of characteristics of the population that would not be discovered in the normal procedure of analysis. Outliers may simply be extreme values in a probability distribution of a random variable that occur quite naturally but not frequently and should not be rejected (Alves and Nascimento, 2002). In contrast, the major part of the outliers can be considered to be measurements with gross errors.…”

“…This task becomes especially complicated for complex processes. The influencing parameters may be not directly accessible or large stochastic deviations of the process variables may lead to a considerable scatter in the measured data (Alves and Nascimento, 2002). For this reason, many approaches have been proposed.…”

“…In this step, the hypothesis test for means was employed, which involves a confidence interval estimate and a hypothesis test, with 95% as the confidence level (Himmelblau, 1970). Details of this procedure are found in Alves and Nascimento (2002).…”

Detection of outliers in a gas centrifuge experimental data

“…Correct use of these experimental data requires a careful analysis of the associated e rrors, which indicates the reproducibility, the representativeness and the reliability of the data obtained. However, this analysis can be an arduous task in the case of a very complex process, where uncontrolled or unmonitored parameters influence the process or where large stochastic deviations are observed, resulting in a very dispersed data set (Alves and Nascimento, 2002) Several procedures have been proposed for the treatment and analysis of experimental data sets, based on statistics or on the physics of the process (Himmelblau, 1970;Plácido and Loureiro, 1998). Nonetheless, those techniques can be difficult to apply.…”