Detecting changes of clustering structures using normalized maximum likelihood coding

Abstract

We are concerned with the issue of detecting changes of clustering structures from multivariate time series. From the viewpoint of the minimum description length (MDL) principle, we propose an algorithm that tracks changes of clustering structures so that the sum of the code-length for data and that for clustering changes is minimum. Here we employ a Gaussian mixture model (GMM) as representation of clustering, and compute the code-length for data sequences using the normalized maximum likelihood (NML) coding. The proposed algorithm enables us to deal with clustering dynamics including merging, splitting, emergence, disappearance of clusters from a unifying view of the MDL principle. We empirically demonstrate using artificial data sets that our proposed method is able to detect cluster changes significantly more accurately than an existing statistical-test based method and AIC/BIC-based methods. We further use real customers' transaction data sets to demonstrate the validity of our algorithm in market analysis. We show that it is able to detect changes of customer groups, which correspond to changes of real market environments.

A Short Review on Minimum Description Length: An Application to Dimension Reduction in PCA

“…Ref. [ 49 ] presents a study concerning the use of MDL, specifically, the normalized maximum likelihood (NML) version, in the dynamic model selection. The aim is to track changes of clustering structures so that the sum of the data’s code-length and clustering changes’ code-length is minimized.…”

Kernel Complexity for Nonparametric Distributions and Detection of Its Changes

“…Yamanishi and Maruyama proposed a dynamic model selection (DMS) algorithm based on the MDL principle to detect the changes in statistical models [14,15]. As an extension of this algorithm, Hirai and Yamanishi proposed the sequential DMS (SDMS) algorithm, which is used to apply DMS to a sequential setting [16]. Spiliopoulou et al conducted research that adds a qualitative consideration to the change in clustering by proposing a method to identify the nature of the transition of a cluster on the basis of data movement between clusters [17].…”

“…In this study, the AUC is calculated in relation to the benefit. Using these evaluation scores, we evaluated the proposed algorithm in comparison with the density ratio estimation (DRE) algorithm [19], SDMS algorithm [16], SE algorithm [27], tracking the best expert (TBE) algorithm [12], and the entropy-based method (abbreviated as entropy), which is described in Algorithm 2. When using the DRE algorithm, we process the two-dimensional data X t ∈ R n×m at each time as one-dimensional data X ′ t ∈ R nm and use them as the input for the DRE algorithm.…”

Data Discovery and Anomaly Detection Using Atypicality for Real-Valued Data

“…Finally, we will mention change point detection and quickest change detection [ 35 , 36 , 37 , 38 , 39 , 40 , 41 , 42 ]. The goal here is to find a point in time where the distribution of data changes from one to another.…”

“…In principle one could use atypicality for change point detection, but since it is not optimized for this application, the comparison is not that relevant, and atypicality might not perform well. We refer to [ 35 , 36 ] for how to use MDL for change point detection.…”