Outlier detection by means of robust regression estimators for use in engineering science

Hekimoğlu, Şerif; Erenoǧlu, Ramazan Cüneyt; Kalina, Jan

doi:10.1631/jzus.a0820140

Cited by 15 publications

(13 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…After several tests, we determined that it is safe to discard abundances with assinged weights smaller than 0.90 (the weight scale ranges from zero to one). Robust regression estimators are shown to be more reliable than sigma clipping (Hekimoglu et al 2009). After the initial abundance outliers are discarded, the leastsquares algorithm minimizes three values: the two slopes of the linear models fitted as a function of the excitation potential and the reduced equivalent width, and the difference in abundances from neutral and ionized iron lines.…”

Section: Equivalent Width Methodsmentioning

confidence: 99%

Determining stellar atmospheric parameters and chemical abundances of FGK stars with iSpec

Blanco-Cuaresma

Soubiran

Heiter

et al. 2014

A&A

401

301

View full text Add to dashboard Cite

Context. An increasing number of high-resolution stellar spectra is available today thanks to many past and ongoing extensive spectroscopic surveys. Consequently, the scientific community needs automatic procedures to derive atmospheric parameters and individual element abundances. Aims. Based on the widely known SPECTRUM code by R.O. Gray, we developed an integrated spectroscopic software framework suitable for the determination of atmospheric parameters (i.e., effective temperature, surface gravity, metallicity) and individual chemical abundances. The code, named iSpec and freely distributed, is written mainly in Python and can be used on different platforms. Methods. iSpec can derive atmospheric parameters by using the synthetic spectral fitting technique and the equivalent width method. We validated the performance of both approaches by developing two different pipelines and analyzing the Gaia FGK benchmark stars spectral library. The analysis was complemented with several tests designed to assess other aspects, such as the interpolation of model atmospheres and the performance with lower quality spectra. Results. We provide a code ready to perform automatic stellar spectral analysis. We successfully assessed the results obtained for FGK stars with high-resolution and high signal-to-noise spectra.

show abstract

Section: Equivalent Width Methodsmentioning

confidence: 99%

Determining stellar atmospheric parameters and chemical abundances of FGK stars with iSpec

Blanco-Cuaresma

Soubiran

Heiter

et al. 2014

A&A

401

301

View full text Add to dashboard Cite

show abstract

“…The cutoff value 2.5 is chosen with regard to the situation with normal errors; in such case only 1.24 % of residuals are classified as outliers [11]. Still, it is necessary to admit that each robust estimator would require its own particular value of the threshold [5].…”

Section: Estimation Of σ 2 and Outlier Detectionmentioning

confidence: 99%

“…In general, outliers typically appear in real data in across various disciplines, e.g. in engineering applications [5] or image analysis based on markers measured within images [6]. Outliers appear practically always in measurements of molecular genetic and metabolomic biomarkers, for which severe measurement errors are immanent to the measurement technology [7], [8].…”

Section: Introductionmentioning

confidence: 99%

On Robust Estimation of Error Variance in (Highly) Robust Regression

Kalina

Tichavský

2020

Measurement Science Review

Self Cite

View full text Add to dashboard Cite

The linear regression model requires robust estimation of parameters, if the measured data are contaminated by outlying measurements (outliers). While a number of robust estimators (i.e. resistant to outliers) have been proposed, this paper is focused on estimating the variance of the random regression errors. We particularly focus on the least weighted squares estimator, for which we review its properties and propose new weighting schemes together with corresponding estimates for the variance of disturbances. An illustrative example revealing the idea of the estimator to down-weight individual measurements is presented. Further, two numerical simulations presented here allow to compare various estimators. They verify the theoretical results for the least weighted squares to be meaningful. MM-estimators turn out to yield the best results in the simulations in terms of both accuracy and precision. The least weighted squares (with suitable weights) remain only slightly behind in terms of the mean square error and are able to outperform the much more popular least trimmed squares estimator, especially for smaller sample sizes.

show abstract

“…So far, Finland Outotec company still has more than 80% market share. In China, the Beijing Institute of Mining and Metallurgy following its analytical principles, developed in 2014 with the same function grade analyzer -BOXA type on-stream x-ray fluorescence analyzer (Hekimoglu, Eernoglu, & Kalina, 2009). According to foreign reports, the measurement accuracy of the analyzer increased by 1%, will effectively improve the metal recovery rate of 0.2 or more, and now from the hardware to improve measurement accuracy has been very difficult, or input-output serious disproportionate, so more scholars turn to the analysis of the modeling technology to improve research.…”

Section: Applicationmentioning

confidence: 99%

Online Outlier Detection for Time-varying Time Series on Improved ARHMM in Geological Mineral Grade Analysis Process

Zhao¹,

Zhoub²,

Su³

et al. 2017

Earth sci. res. j.

View full text Add to dashboard Cite

Given the difficulty of accurate online detection for massive data collecting real-timely in a strong noise environment during the complex geological mineral grade analysis process, an order self-learning ARHMM (Autoregressive Hidden Markov Model) algorithm is proposed to carry out online outlier detection in the geological mineral grade analysis process. The algorithm utilizes AR model to fit the time series obtained from "Online x -ray Fluorescent Mineral Analyzer" and makes use of HMM as a basic detection tool, which can avoid the deficiency of presetting the threshold in traditional detection methods. The structure of traditional BDT (Brockwell-Dahlhaus-Trindade) algorithm is improved to be a double iterative structure in which iterative calculation from both time and order is applied respectively to update parameters of ARHMM online. With the purpose of reducing the influence of outlier on parameter update of ARHMM, the strategies of detection-before-update and detection-based-update are adopted, which also improve the robustness of the algorithm. Subsequent simulation by model data and practical application verify the accuracy, robustness, and property of online detection of the algorithm. According to the result, it is obvious that new algorithm proposed in this paper is more suitable for outlier detection of mineral grade analysis data in geology and mineral processing. Existe gran dificultad para la detección en tiempo real para series de datos masivos con altos niveles de ruido de valores atípicos. Se propone un algoritmo de autoaprendizaje ARHMM (Modelo autoregresivo oculto de Markov) para llevar a cabo la detección de dichos valores atípicos en el proceso de análisis del grado mineral geológico. El algoritmo usa un modelo AR para ajustar la serie de tiempo obtenida del "analizador de fluorescencia de rayos X" y hace uso del HMM como una herramienta básica de detección, la cual puede evitar la deficiencia de predeterminar el umbral en métodos tradicionales de detección. Para actualizar los parámetros del ARHMM en tiempo real, la estructura del algoritmo BDT (Brockwell-Dahlhaus-Trindade) tradicional se mejora para ser una doble estructura iterativa en la que se aplica el cálculo iterativo en tiempo y en orden respectivamente. Con el propósito de reducir la influencia de valores atípicos (o extremos) en la actualización del parámetro de ARHMM, se adoptan las estrategias de detección-antes-que-actualización y la detección-basada-en-actualización, lo que también aumenta la robustez del algoritmo. La subsiguiente simulación por modelos de datos y aplicación práctica comprueba la precisión, fortaleza y capacidad de la detección en línea del algoritmo. De acuerdo con el resultado, es evidente que el nuevo algoritmo propuesto en este artículo es más apropiado para la detección de datos de valores atipicos para el análisis del grado mineral en geología y el procesamiento mineral. ABSTRACT RESUMEN IntroductionMineral composition analysis is a key factor in determining whether or not to carry out mining. Over t...

show abstract

Outlier detection by means of robust regression estimators for use in engineering science

Cited by 15 publications

References 18 publications

Determining stellar atmospheric parameters and chemical abundances of FGK stars with iSpec

Determining stellar atmospheric parameters and chemical abundances of FGK stars with iSpec

On Robust Estimation of Error Variance in (Highly) Robust Regression

Online Outlier Detection for Time-varying Time Series on Improved ARHMM in Geological Mineral Grade Analysis Process

Contact Info

Product

Resources

About