A Distance for HMMs Based on Aggregated Wasserstein Metric and State Registration

Abstract

We propose a framework, named Aggregated Wasserstein, for computing a dissimilarity measure or distance between two Hidden Markov Models with state conditional distributions being Gaussian. For such HMMs, the marginal distribution at any time spot follows a Gaussian mixture distribution, a fact exploited to softly match, aka register, the states in two HMMs. We refer to such HMMs as Gaussian mixture model-HMM (GMM-HMM). The registration of states is inspired by the intrinsic relationship of optimal transport and the Wasserstein metric between distributions. Specifically, the components of the marginal GMMs are matched by solving an optimal transport problem where the cost between components is the Wasserstein metric for Gaussian distributions. The solution of the optimization problem is a fast approximation to the Wasserstein metric between two GMMs. The new Aggregated Wasserstein distance is a semi-metric and can be computed without generating Monte Carlo samples. It is invariant to relabeling or permutation of the states. This distance quantifies the dissimilarity of GMM-HMMs by measuring both the di↵erence between the two marginal GMMs and the difference between the two transition matrices. Our new distance is tested on the tasks of retrieval and classification of time series. Experiments on both synthetic data and real data have demonstrated its advantages in terms of accuracy as well as e ciency in comparison with existing distances based on the Kullback-Leibler divergence.

A Wasserstein-Type Distance in the Space of Gaussian Mixture Models

“…Observe that the point of view followed here in our paper is quite different from these works, since M W 2 is defined in a completely continuous setting as an optimal transport between GMMs with a restriction on couplings, following the same kind of approach as in [3]. The fact that this restriction leads to an explicit discrete formula, the same as the one proposed independently in [8,9] and [6,7], is quite striking. Observe also that thanks to the "identifiability property" of GMMs, this continuous formulation (4.1) is obviously non ambiguous, in the sense that the value of the minimium is the same whatever the parametrization of the Gaussian mixtures µ 0 and µ 1 .…”

“…It happens that the discrete form (4.4), which can be seen as an aggregation of simple Wasserstein distances between Gaussians, has been recently proposed as an ingenious alternative to W 2 in the machine learning literature, both in [8,9] and [6,7]. Observe that the point of view followed here in our paper is quite different from these works, since M W 2 is defined in a completely continuous setting as an optimal transport between GMMs with a restriction on couplings, following the same kind of approach as in [3].…”

“…The complexity of the corresponding discrete optimization problem does not depend on the space dimension, but only on the number of components in the different mixtures, which makes it particularly suitable in practice for high dimensional problems. Observe that this equivalent discrete formulation has been proposed twice recently in the machine learning literature, but with a very different point of view, by two independent teams [8,9] and [6,7].…”

Multisource Single-Cell Data Integration by MAW Barycenter for Gaussian Mixture Models

“…In this article, we follow the terminology of Chen et al. (2016) and call $$\widetilde{W}_2$$ MAW distance. The notation $$GW$$ was used by Delon and Desolneux (2020), which we do not adopt here to avoid confusion with the Gromov Wasserstein distance (Peyré et al.…”

Evaluating generative networks using Gaussian mixtures of image features

“…Constraining the class of joint distributions is a relaxation that has been done before (Bion-Nadal et al, 2019) due to the difficulty of considering arbitrary joint distributions. This metric, MW 2 , appears in a few different sources in the literature (Chen et al, 2016;2018;2019) and has been studied theoretically (Delon & Desolneux, 2020); recently, implementations of this quantity have emerged. 1…”