Principal component analysis for data containing outliers and missing elements

Serneels, Sven; Verdonck, Tim

doi:10.1016/j.csda.2007.05.024

Cited by 141 publications

(85 citation statements)

References 36 publications

(63 reference statements)

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…For example, the influence of a single outlier can be critical for the PCA (Walczak and Massart, 1995;Stanimirova et al, 2007;Serneels and Verdonck, 2008). Another problem of outlier detection is that even when applying automatic outlier handling pre-processing steps, usually the results have to be validated manually by the model developer.…”

Section: Data Outliersmentioning

confidence: 99%

Data-driven Soft Sensors in the process industry

Kadlec

Gabrys

Strandt

2009

Computers & Chemical Engineering

1,441

725

View full text Add to dashboard Cite

In the last two decades Soft Sensors established themselves as a valuable alternative to the traditional means for the acquisition of critical process variables, process monitoring and other tasks which are related to process control. This paper discusses characteristics of the process industry data which are critical for the development of data-driven Soft Sensors. These characteristics are common to a large number of

show abstract

Section: Data Outliersmentioning

confidence: 99%

Data-driven Soft Sensors in the process industry

Kadlec

Gabrys

Strandt

2009

Computers & Chemical Engineering

1,441

725

View full text Add to dashboard Cite

show abstract

“…As PCA is sensitive to outliers within the data, methods to perform a more robust PCA are also considered, e.g., in Serneels & Verdonck (2008). However, for analysis of SPECT or PET images the underlying data usually contains a manageable amount of subjects and can therefore be sorted manually or by applying tests as presented in Section 5.…”

Section: Principal Component Analysismentioning

confidence: 99%

Principal Component Analysis Applied to SPECT and PET Data of Dementia Patients – A Review

Stühler¹,

Merhof²

2012

Principal Component Analysis - Multidisciplinary Applications

View full text Add to dashboard Cite

“…Outliers may also cause the so-called masking effect: due to their presence, the model is distorted in such a way that, based on the principal components, no outliers are detected (Serneels & Verdonck, 2008) With respect to the method of time series, the effect of outliers is associated with structural changes that may be related to unexpected events, such as economic crisis, strikes or wars, measurement errors or inadequate recording of information.…”

Section: Introductionmentioning

confidence: 99%

<b>Principal components in the discrimination of outliers: A study in simulation sample data corrected by Pearson's and Yates´s chi-square distance

Veloso

Cirillo

2016

Acta Sci. Technol.

View full text Add to dashboard Cite

ABSTRACT. Current study employs Monte Carlo simulation in the building of a significance test to indicate the principal components that best discriminate against outliers. Different sample sizes were generated by multivariate normal distribution with different numbers of variables and correlation structures. Corrections by chi-square distance of Pearson´s and Yates's were provided for each sample size. Pearson´s correlation test showed the best performance. By increasing the number of variables, significance probabilities in favor of hypothesis H 0 were reduced. So that the proposed method could be illustrated, a multivariate time series was applied with regard to sales volume rates in the state of Minas Gerais, obtained in different market segments.Keywords: contaminated samples, Monte Carlo, significance test, p-value.Componentes principais na discriminação de outliers: estudo de simulação em dados amostrais corrigidos pelas distâncias qui-quadrado de Pearson's and Yates.RESUMO. Este trabalho tem por objetivo realizar um estudo, utilizando simulação Monte Carlo na construção de um teste de significância para indicar os componentes principais que melhor discriminam as discrepâncias. Neste contexto, diferentes tamanhos amostrais foram gerados pela distribuição normal multivariada com diferentes números de variáveis e estruturas de correlação. Para cada tamanho amostral, procedeu-se com as correções dadas pela distância qui-quadrado de Pearson e Yates. Concluiu-se ao considerar a correção de Pearson o teste apresentou melhor desempenho, entretanto, aumentando o número de variáveis as probabilidades de significância a favor a hipótese H 0 foram reduzidas. Por fim, para ilustrar a metodologia proposta realizou-se uma aplicação em uma série temporal multivariada referente a índices de volumes de vendas do estado de Minas Gerais obtidos em diferentes segmentos de mercados.

show abstract

Principal component analysis for data containing outliers and missing elements

Cited by 141 publications

References 36 publications

Data-driven Soft Sensors in the process industry

Data-driven Soft Sensors in the process industry

Principal Component Analysis Applied to SPECT and PET Data of Dementia Patients – A Review

<b>Principal components in the discrimination of outliers: A study in simulation sample data corrected by Pearson's and Yates´s chi-square distance

Contact Info

Product

Resources

About