Basic statistics for distributional symbolic variables: a new metric-based approach

Abstract

In data mining it is usual to describe a group of measurements using summary statistics or through their empirical distribution functions. Each summary of a group of measurements is the representation of a typology of individuals (sub-populations) or of the evolution of the observed variable for each individual. Therefore, typologies or individuals are expressible through multi-valued descriptions (intervals, frequency distributions). Symbolic Data Analysis, a relatively new statistical approach, aims at the treatment of such kinds of data. In the conceptual framework of Symbolic Data Analysis, the paper aims at presenting new basic statistics for numeric multi-valued data. First of all, we propose how to consider all numerical multi-valued descriptions as special cases of distributional data, i.e. as data described by distributions. Secondly, we extend some classic univariate (mean, variance, standard deviation) and bivariate (covariance and correlation) basic statistics taking into account the nature, the source and the interpretation of the variability of such data. As opposed to those proposed in the literature, the novel statistics are based on a distance between distributions, the ℓ2 Wasserstein distance. Using a clinic dataset, we compare the proposed approach to the existing one showing the main differences in terms of interpretation of results.

Fuzzy clustering of distributional data with automatic weighting of variable components

“…Finally, it is noteworthy that, according to [23] and the therein references, a closed form of the distance in Eq. (1) can be obtained under some alternative conditions: there exists closed forms for the integral of Q kj , Q 2 kj and Q kj ·Q k j ; if Q kj is a quantile function associated with a histogram [22]; or if the probability density function associated with Q kj depends only on its means and its standard deviation (for example, if Q kj and Q kj are the quantile functions of two Gaussian or two Uniform distributional data).…”

“…. , P ) element of the matrix of prototypes G is obtained as follows, for each cluster, setting the partial derivatives w.r.t.ḡ ij and Q c gij equal to zero, and according to [23], the quantile function associated with the corresponding probability density function (pdf ) g kj is:…”

“…All the cited methods rely on the choice of a suitable dissimilarity between DD. More recently, the use of Wasserstein distances [35] has been investigated and a new set of statistical indexes has been proposed [23] for DV.…”

“…We remark that g ij is a DD because the corresponding quantile function Qg ij is a weighted average quantile function (for further details see[16] and[23]) and, thus is a quantile function, whose derivative of its inverse is a probability density function.…”

Principal axes analysis of symbolic histogram variables

“…). Irpino and Verde also noticed that the two distances that give a single center in the form of histogram are the L 2 Euclidean distance and the L 2 Wasserstein distance. If and are the two density functions associated with the histogram and , respectively, and and their corresponding quantile functions, the L 2 Wasserstein distance (for a continuous distribution) is defined as follows: The mean quantile function is …”