Tail inference: where does the tail begin?

Abstract

Abstract. The quality of estimation of tail parameters, such as tail index in the univariate case, or the spectral measure in the multivariate case, depends crucially on the part of the sample included in the estimation. A simple approach involving sequential statistical testing is proposed in order to choose this part of the sample. This method can be used both in the univariate and multivariate cases. It is computationally efficient, and can be easily automated. No visual inspection of the data is required. We establish consistency of the Hill estimator when used in conjunction with the proposed method, as well describe its asymptotic fluctuations. We compare our method to existing methods in univariate and multivariate tail estimation, and use it to analyze Danish fire insurance data.

Initial Results of Testing Some Statistical Properties of Hard Disks Workload in Personal Computers in Terms of Non-Extensive Entropy and Long-Range Dependencies

“…The estimation of the heavy tail indexα by means of the Hill method is quite often used by researchers. It is believed that this is one of the best methods [56]. The estimator is based on the approach where, for a set of Y i of recorded data, a new ordered, in a decreasing fashion, set X i is given, with X 1 being the highest value from Y i .…”

“…The most common technique applied to solve this problem is the use of plotα vs. k as well as choosing a part of the plot that looks stable. There are also effective techniques and approaches based on statistical testing [56].…”

Multivariate Tail Estimation With Application to Analysis of Covar

“…Danielsson et al (2001) and Drees and Kaufmann (1998). In our previous work (Nguyen and Samorodnitsky, 2012), we proposed a sequential testing method for deciding "where the tail begins" in which the significance level increases with the sample size. Our approach is based on a simple idea.…”

“…The choice of k described above (stop the first time the exponentiality hypothesis is rejected) is only one of several possibilities; another possibility is to stop the last time the exponentiality hypothesis is accepted. We use the choice of k for which the asymptotic theory in Nguyen and Samorodnitsky (2012) applies.…”

“…de Haan and Ferreira (2006). With a random choice of k as in our procedure, we needed to use the second-order regular variation even to obtain consistency of the estimator, even in the one-dimensional case; see Nguyen and Samorodnitsky (2012). In fact, the behavior of the random variable N n above seems to undergo a phase transition if the condition λ n = (n 2|ρ| 1+2|ρ| ) breaks down.…”