A composite statistical method for the detection of multiple undocumented abrupt changes in the mean value within a time series

Abstract

ABSTRACT:The time series of measurements of hydro-meteorological variables often suffer from imperfections such as missing data, outliers and discontinuities in the mean values. The discontinuity in the mean can be the effect of: instrumental offsets and of their corrections, of changes in the monitoring station or in the surrounding environment. If the discontinuities can be identified with a reasonable precision, a correction of the erroneous data can be made. Several authors have put their great effort into developing techniques to identify non-climatic inhomogeneities; the resulting statistical methods are especially effective when the series contains a single change point, while their performances decline when the series contains multiple change points or inhomogeneous segments (a portion of the series bounded by two complementary shifts). These limitations also affect the standard normal homogeneity test (SNHT), one of the most effective and widely applied tests. We present a composite method of homogeneity testing, standard normal homogenization composite method (SNHCM), including the SNHT as one component, which improves the SNHT performances with multiple change points and inhomogeneous segments. A number of comparisons among the new method, the SNHT and a powerful optimal segmentation method (OSM-CM), are illustrated in the paper. SNHCM demonstrates their performances in change-point detection similar to, or better than, the SNHT and very close to the OSM-CM. The SNHCM is effective in recognizing complex patterns of discontinuities, especially inhomogeneous segments, which represent a severe problem for SNHT; on the contrary, SNHT performs slightly better only when the series contains a single change point, but the difference between the two methods is negligible. Compared to the OSM-CM, SNHCM provides very similar performances, with some favourable features deriving from the fact that it is computationally lighter, simpler to implement, can easily handle very long series and is based on statistical hypothesis tests with a well-defined and adjustable significance level.

Homogeneidade de séries climatológicas em Minas Gerais

“…As não homogeneidades podem ser alterações causadas, por exemplo, pelo crescimento de vegetação ou pela urbanização na proximidade das estações ou, ainda, por mudança na localização ou descalibração nos instrumentos de medida e até mesmo por hábitos de observação (Aguilar et al, 2003;Caussinus & Mestre, 2004;Brunet et al, 2006;Costa & Soares, 2009;Beaulieu et al, 2010;Pandžiae & Likso, 2010;Rienzner & Gandolfi, 2011).…”

Critical values improvement for the standard normal homogeneity test by combining Monte Carlo and regression approaches

“…As a matter of fact, in most cases "single change-point" tests applications work fine also in case multiple change-points are included in the series (which is the common case), provided that such steps form a so-called hierarchic pattern (Hawkings 2001). On the other hand, the single change-point tests lose in reliability (Hawkings 2001;Rienzner and Gandolfi 2011) in case the steps follow a non-hierarchic pattern, as the platform-like inhomogeneity, i.e. two subsequent changes having equal amplitude and opposite sign.…”

“…On the other hand, multiple change-point detection methods are computationally cumbersome and their application to time series with many thousands of elements is often unfeasible. To this aim, Rienzner and Gandolfi (2011) proposed the SNHCM (standard normal homogeneity composite method), a composite multiple change-point detection method including SNHT and SNH2T (this test is peculiar to identify the platform-like inhomogeneity). Therefore, SNHCM is an intermediate point between the complete multiple change-point detection methods and the single change-point tests.…”

“…The Standard Normal Homogeneity Test (SNHT -Alexandersson 1986), is a single change-point detection test widely adopted in climate time-series analysis for the homogeneity assessment and the change-point detection (see Khaliq and Ouarda 2007;Venema et al 2013). SNHT is known for its power, simplicity and robustness (Aguilar et al 2003;Ducrè-Robitaille et al 2003;Beaulieu et al 2008;Domonkos 2011;Rienzner and Gandolfi 2011;Venema et al 2012). For identifying further change-points, repeated applications are performed on pieces of the series, which is cut according to the identified change-points.…”

“…All the analyses were carried out in MATLAB ® (R2011b, version 7.13), along with the MATLAB Optimization Toolbox ® (R2011b, version 6.1), the MATLAB Statistics Toolbox ® (R2011b, version 7.6), and the routines implementing SNHT and SNH2T as described in Rienzner and Gandolfi (2011).…”