Dealing with temporal and spatial correlations to classify outliers in geophysical data streams

Appice, Annalisa; Guccione, Pietro; Malerba, Donato; Ciampi, Anna

doi:10.1016/j.ins.2013.12.009

Cited by 24 publications

(15 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…This augmented representation makes each vertex "aware" of the values of the vertices connected to it and informs the process of learning about patterns of autocorrelation existing within the network (Network autocorrelation in Figure 2). To the best of our knowledge this has been studied only for classification problems [7,8]. 2.…”

Section: Contributionsmentioning

confidence: 99%

“…The classification algorithm uses the information of the vertices related to the vertex to be classified and also the past connections of the related vertices. The method proposed in [7] accounts for autocorrelation to detect outliers in data streams generated by geophysical sensors. In particular, a forecasting model is trained incrementally by accounting for temporal correlation of the data that exhibit a spatial correlation in the recent past.…”

Section: Temporal Autocorrelation For Predictive Inferencementioning

confidence: 99%

See 1 more Smart Citation

Leveraging temporal autocorrelation of historical data for improving accuracy in network regression

Loglisci

Malerba

2017

Statistical Analysis

Self Cite

View full text Add to dashboard Cite

Temporal data describe processes and phenomena that evolve over time. In many real-world applications temporal data are characterized by temporal autocorrelation, which expresses the dependence of time-stamped data over a certain a time lag. Often such processes and phenomena are characterized by evolving complex entities, which we can represent with evolving networks of data. In this scenario, a task that deserves attention is regression inference in temporal network data. In this paper, we investigate how to improve the predictive inference on network data by accommodating temporal autocorrelation of the historical data in the learning process of the prediction models. Historical data is a type of temporal data where most part of the elements has been already stored. In practice, we study how to explicitly consider the influence of data of a network observed in the past, to enhance the prediction on the same network observed at the present. The proposed approach relies on a model ensemble built with individual predictors learned on historical network data. The predictors are trained from summary networks, which synthesize the effect of the autocorrelation in distinct sequences of network observations. Summary networks are identified with a sliding window model. Finally, the model ensemble combines together the predictors with a weighting schema, which reflects the degree of influence of a predictor with respect to the network observed at the present. So, we aim at accommodating the temporal autocorrelation both in the data and in the prediction model. Empirical evaluation demonstrates that the proposed approach can boost regression performance in real-world network data.

show abstract

Section: Contributionsmentioning

confidence: 99%

Section: Temporal Autocorrelation For Predictive Inferencementioning

confidence: 99%

Leveraging temporal autocorrelation of historical data for improving accuracy in network regression

Loglisci

Malerba

2017

Statistical Analysis

Self Cite

View full text Add to dashboard Cite

show abstract

“…Metodologias para a detecção de outliers vêm sendo criadas para atender às demandas das diversas áreas do conhecimento científico, como proposto por Barua e Alhajj (2007) para processamento de imagens, Qiao, Haibo e Hong (2013) para dados provenientes de satélites e Appice et al (2014) para fluxo de dados geofísicos. Santos et al (2017) propuseram um método de detecção de outliers para dados geoespaciais contínuos através da geoestatística e teoremas da estatística clássica, independentemente da causa geradora das inconsistências.…”

Section: Introductionunclassified

Estimação Do Tamanho Amostral Na Geoestatística Usando Um Modelo De Variograma Gaussiano Na Presença De Outliers

Mendes

Santos

Emiliano³

et al. 2019

EnergAgric

View full text Add to dashboard Cite

RESUMO:A determinação de um tamanho de amostragem que seja adequado para a reconstrução da população, na análise de dados espaciais, é algo que tem sido estudado em vários trabalhos. Independentemente da área de estudo, qualquer variável pode conter outlier. Conforme sugerido por alguns pesquisadores, no intuito de eliminar tais dados discrepantes, metodologias vêm sendo criadas para atender às demandas das diversas áreas do conhecimento científico. O objetivo deste trabalho é utilizar o teorema da taxa de Nyquist para determinar um tamanho ideal para amostras georreferenciadas contendo outliers, oriundas de uma grade quadrática regular, no qual o modelo de dependência espacial é o Gaussiano. O que se pretende atingir é uma densidade de amostragem necessária para a reconstrução de mapas populacionais de variáveis nas quais as condições de regularidade necessárias em geoestatística foram verificadas. Como resultado pode-se concluir que o tamanho ideal de amostragem obtido na ausência de outliers, 115 pontos, foi bem inferior aos 228 pontos obtidos na presença dos outliers.Palavras-chave: geoestatística, amostragem, taxa Nyquist, outliers. ABSTRACT:The determination of suitable sample size for population reconstruction in the analysis of spatial data is something that has been studied in several scientific papers. Regardless study area, any variable may contain outlier. As suggested by some researchers, in order to eliminate such discrepant data, methodologies have been created to meet the demands of various scientific areas knowledge. The purpose of this work is use the Nyquist Rate Theorem to determine an ideal size for georeferenced samples containing outliers from a regular quadratic grid in which the spatial dependence model is Gaussian. What we intend to achieve is a sampling density necessary for the population maps reconstruction of variables in which the necessary regularity conditions in Geostatistics were verified. As a result, it can be concluded that, the ideal sampling size obtained in the outliers absence, 115 points, was well below the 228 points obtained in the outliers presence.

show abstract

“…Some of these authors assert that the concern about disparate data is as old as the first attempts at analysis of a set of data, as in the case of comments of Bernoulli in 1777 about the existence of such data. Recently, new methods for handling outliers have been developed to meet the demands of various areas of scientific knowledge, as in the case of (Hongxing, et al, 2001) for spatial data distributed in irregular grids, (Barua and Alhajj, 2007) for processing images, (Qiao et al, 2013) for data from satellites and (Appice et al, 2014) for geophysical data stream. Studying the detection of outliers is important because the first step in data analysis consists in the evaluation of data quality.…”

Section: Introductionmentioning

confidence: 99%

Detection of Inconsistencies in Geospatial Data With Geostatistics

Santos

Emiliano

et al. 2017

Bol. Ciênc. Geod.

View full text Add to dashboard Cite

Abstract:Almost every researcher has come through observations that "drift" from the rest of the sample, suggesting some inconsistency. The aim of this paper is to propose a new inconsistent data detection method for continuous geospatial data based in Geostatistics, independently from the generative cause (measuring and execution errors and inherent variability data). The choice of Geostatistics is based in its ideal characteristics, as avoiding systematic errors, for example. The importance of a new inconsistent detection method proposal is in the fact that some existing methods used in geospatial data consider theoretical assumptions hardly attended. Equally, the choice of the data set is related to the importance of the LiDAR technology (Light Detection and Ranging) in the production of Digital Elevation Models (DEM). Thus, with the new methodology it was possible to detect and map discrepant data. Comparing it to a much utilized detections method, BoxPlot, the importance and functionality of the new method was verified, since the BoxPlot did not detect any data classified as discrepant. The proposed method pointed that, in average, 1,2% of the data of possible regionalized inferior outliers and, in average, 1,4% of possible regionalized superior outliers, in relation to the set of data used in the study. Keywords: Outliers, geoprocessing, LiDAR technology. Resumo:Provavelmente todo pesquisador já se deparou com observações que se "afastam" das demais, sugerindo a existência de inconsistências. O objetivo desse trabalho é propor um novo método de detecção de dados inconsistentes para dados geoespaciais contínuos baseando-se na Geoestatística, independentemente da causa geradora (erros de medição, execução e variabilidade inerente aos dados). A escolha pela Geoestatística está baseada em suas características ideais, como evitar erros

show abstract

Dealing with temporal and spatial correlations to classify outliers in geophysical data streams

Cited by 24 publications

References 18 publications

Leveraging temporal autocorrelation of historical data for improving accuracy in network regression

Leveraging temporal autocorrelation of historical data for improving accuracy in network regression

Estimação Do Tamanho Amostral Na Geoestatística Usando Um Modelo De Variograma Gaussiano Na Presença De Outliers

Detection of Inconsistencies in Geospatial Data With Geostatistics

Contact Info

Product

Resources

About