An Algorithm for on-Line Outlier Rejection by Sequence-Analysis in Data Acquisition

Pavese, Francesco; Ichim, Daniela; Colonnetti, G.; Ciarlini, P.; Picone, Marco

doi:10.1142/9789812811684_0034

Cited by 2 publications

(4 citation statements)

References 2 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The algorithm was initially implemented in FORTRAN 77 as a subroutine [3], and later also in MATLAB R . The FORTRAN 77 subroutine takes a few microseconds and the MATLAB R one takes about 3.3 ms to run on a slow PC (clock rate <500 MHz).…”

Section: Code Implementation For the Saodr Algorithm And Test Resultsmentioning

confidence: 99%

“…The analysis of the distance subsequences is the core of the algorithm, see [3]. Its construction is based on the analysis of all possible situations involving outliers in the reading sequence.…”

Section: The 'Truth Table'mentioning

confidence: 99%

“…A two-step procedure has been introduced in [3]: for each sample y i , (i) a pre-processing step rejects the outlying readings y ij then, (ii) the baseline drift is suppressed by means of a regression applied to the 'cleaned' readings, to obtain the valid output sample value y i . The first step consists of a pre-processing algorithm, named 'sequence-analysis outlier data rejection' (SAODR), aimed at being easily fitted into the instrument firmware.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

SAODR: sequence analysis for outlier data rejection

Pavese¹,

Ichim²

2004

Meas. Sci. Technol.

View full text Add to dashboard Cite

In automatic data acquisition, a sample is generally made up of several instrumental readings. A series of readings is generally reduced to a single value by simple methods, such as averaging. However, outlying values can affect the series. The paper introduces an algorithm, named ‘sequence-analysis outlier data rejection’ (SAODR), which takes into account one of the most common problems affecting the measurand during the acquisition, i.e. a nonlinear drift with embedded sequences of outliers due to pulse-noise peaks. The algorithm uses a time-ordering procedure and the ‘distances’ between successive readings. The frequent case of constant sampling rate is discussed. The reported tests show the results obtained with Fortran 77 and MATLAB® implementations of the algorithm. A rejection efficiency higher than 99% was obtained.

show abstract

Section: Code Implementation For the Saodr Algorithm And Test Resultsmentioning

confidence: 99%

Section: The 'Truth Table'mentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

SAODR: sequence analysis for outlier data rejection

Pavese¹,

Ichim²

2004

Meas. Sci. Technol.

View full text Add to dashboard Cite

show abstract

“…These outliers can be due either to pulse-noise (for their elimination during data acquisition see [3]) or to outlying characteristics of a thermometer (either intrinsic or due to its mounting on the calibration apparatus). The outlier rejection has been simply obtained by applying a threshold criterium, not critical for the driftevaluation purpose.…”

Section: Compensation Of Thermal Driftmentioning

confidence: 99%

Tools for Quality Testing of Batches of Artifacts: The Cryogenic Thermometers for the LHC

Ichim¹,

Pavese²,

Balle³

et al. 2001

Advanced Mathematical and Computational Tools in Metrology V

Self Cite

View full text Add to dashboard Cite

In the processing of data series, such as in the case of the resistance R vs. temperature T calibrations of the thermometers (several thousands) necessary for the LHC new accelerator at CERN, it is necessary to use automatic methods for determining the quality of the acquired data and the degree of uniformity of the thermometer characteristics, that are of the semiconducting type. In addition, it must be determined if the calibration uncertainties comply with the specifications in the wide temperature range 1,6 -300 K. Advantage has been taken of the fact that these thermometers represent a population with limited variability, to apply a Least Squares Method with Fixed Effect. This allows to fit the data of all the thermometers together, by taking into account the individuality of each thermometer in the model as a deviation from one of them taken as reference.. where f(T i ) is the model valid for all i data and all k thermometers, while the subsequent part is the "fixed effect" model for the k-th thermometer, where g(T) is a suitable function of T. This method is shown in the paper applied to different stages of the data processing. First, for efficient compensation for the thermal drift occurring during acquisition, robust against the occurrence of outliers. Second, for detection of clusters of thermometers with inherently different characteristics. Finally, for optimisation of the calibration-point distribution. Geneva, Switzerland E-mail: juan.casas-cubillos@cern.ch In the processing of data series, such as in the case of the resistance R vs. temperature T calibrations of the thermometers (several thousands) necessary for the LHC new accelerator at CERN, it is necessary to use automatic methods for determining the quality of the acquired data and the degree of uniformity of the thermometer characteristics, that are of the semiconducting type. In addition, it must be determined if the calibration uncertainties comply with the specifications in the wide temperature range 1,6 -300 K. Advantage has been taken of the fact that these thermometers represent a population with limited variability, to apply a Least Squares Method with Fixed Effect. This allows to fit the data of all the thermometers together, by taking into account the individuality of each thermometer in the model as a deviation from one of them taken as reference:.. where f(Ti) is the model valid for all i data and all k thermometers, while the subsequent part is the "fixed effect" model for the k-th thermometer, where g(T) is a suitable function of T. This method is shown in the paper applied to different stages of the data processing. First, for efficient compensation for the thermal drift occurring during acquisition, robust against the occurrence of outliers. Second, for detection of clusters of thermometers with inherently different characteristics. Finally, for optimisation of the calibration-point distribution.

show abstract

An Algorithm for on-Line Outlier Rejection by Sequence-Analysis in Data Acquisition

Cited by 2 publications

References 2 publications

SAODR: sequence analysis for outlier data rejection

SAODR: sequence analysis for outlier data rejection

Tools for Quality Testing of Batches of Artifacts: The Cryogenic Thermometers for the LHC

Contact Info

Product

Resources

About