Simultaneous Ground Metric Learning and Matrix Factorization with Earth Mover's Distance

Zen, Gloria; Ricci, Elisa; Sebe, Nicu

doi:10.1109/icpr.2014.634

Cited by 13 publications

(13 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The second option is to consider a parametrization of the function C(X, Y ) in order to take into account the structures of the considered graphs. This corresponds to the problem of ground metric learning and has been studied in a line of works, such as [56], [57], [58], [59], among others. We believe that learning an appropriate cost from the data could offer a promising direction for future work.…”

Section: Parametrization Of Flowpool -Learning the Ground Costmentioning

confidence: 99%

FlowPool: Pooling Graph Representations with Wasserstein Gradient Flows

Simou¹

2021

Preprint

View full text Add to dashboard Cite

In several machine learning tasks for graph structured data, the graphs under consideration may be composed of a varying number of nodes. Therefore, it is necessary to design pooling methods that aggregate the graph representations of varying size to representations of fixed size which can be used in downstream tasks, such as graph classification. Existing graph pooling methods offer no guarantee with regards to the similarity of a graph representation and its pooled version. In this work we address this limitation by proposing FlowPool, a pooling method that optimally preserves the statistics of a graph representation to its pooled counterpart by minimizing their Wasserstein distance. This is achieved by performing a Wasserstein gradient flow with respect to the pooled graph representation. We propose a versatile implementation of our method which can take into account the geometry of the representation space through any ground cost. This implementation relies on the computation of the gradient of the Wasserstein distance with recently proposed implicit differentiation schemes. Our pooling method is amenable to automatic differentiation and can be integrated in end-to-end deep learning architectures. Further, FlowPool is invariant to permutations and can therefore be combined with permutation equivariant feature extraction layers in GNNs in order to obtain predictions that are independent of the ordering of the nodes. Experimental results demonstrate that our method leads to an increase in performance compared to existing pooling methods when evaluated in graph classification tasks.

show abstract

Section: Parametrization Of Flowpool -Learning the Ground Costmentioning

confidence: 99%

FlowPool: Pooling Graph Representations with Wasserstein Gradient Flows

Simou¹

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…It is possible to restrict the class of ground metrics, for instance using Mahalanobis [61,31] or geodesic distances [26] to develop more efficient learning schemes. [64] simultaneously perform ground metric learning and matrix factorization, and this finds applications to NLP [28]. Metric learning can also be performed through adversarial optimization, where the metric is maximized over to perform generative model training [19], discriminant analysis [17] and to define robust transportation distances [45,43].…”

Section: Previous Workmentioning

confidence: 99%

Unsupervised Ground Metric Learning using Wasserstein Singular Vectors

Huizing¹,

Cantini²,

Peyré³

2021

Preprint

View full text Add to dashboard Cite

Optimal Transport (OT) defines geometrically meaningful "Wasserstein" distances, used in machine learning applications to compare probability distributions. However, a key bottleneck is the design of a "ground" cost which should be adapted to the task under study. In most cases, supervised metric learning is not accessible, and one usually resorts to some ad-hoc approach. Unsupervised metric learning is thus a fundamental problem to enable data-driven applications of Optimal Transport. In this paper, we propose for the first time a canonical answer by computing the ground cost as a positive eigenvector of the function mapping a cost to the pairwise OT distances between the inputs. This map is homogeneous and monotone, thus framing unsupervised metric learning as a non-linear Perron-Frobenius problem. We provide criteria to ensure the existence and uniqueness of this eigenvector. In addition, we introduce a scalable computational method using entropic regularization, which -in the large regularization limit -operates a principal component analysis dimensionality reduction. We showcase this method on synthetic examples and datasets. Finally, we apply it in the context of biology to the analysis of a high-throughput single-cell RNA sequencing (scRNAseq) dataset, to improve cell clustering and infer the relationships between genes in an unsupervised way.

show abstract

“…Wang et al [47] follow GML's approach but drop the requirement that the learned cost must be a metric. Zen et al [55] use GML to enhance previous results on Nonnegative Matrix Decomposition with a Wasserstein loss (EMD-NMF) [39], by alternatively learning the matrix decomposition and the ground metric. Dupuy et al [20] learn a similarity matrix from the observation of a fixed transport plan, and use this to propose factors explaining weddings across groups in populations.…”

Section: Ground Metric Learningmentioning

confidence: 99%

“…We search for metrics that are geodesic distances on graphs, via a non-linear diffusion equation. Previous methods use formulations such as a full symmetrical distance matrix that can be constrained to satisfy triangle inequalities [47,17,55], a linear transformation of an existing embedding [27], a bilinear form parameterized by an affinity matrix [20], or a combination of local Mahalanobis distances and a global one [52].…”

Section: Ground Metric Learningmentioning

confidence: 99%

“…Our method is similar to the work of Zen et al [55], in the sense that we both aim to reconstruct the input data given a model, however they only use OT distances as loss functions to compare inputs with linear reconstructions, while we use OT distances to synthesize the reconstructions themselves. The difference between our method and that of Dupuy et al [20] lies in the available observations: they learn from a fixed matching (transport plan) whereas we learn from a sequence of mass displacements from which we infer both the metric and an optimal transport plan.…”

Section: Ground Metric Learningmentioning

confidence: 99%

See 1 more Smart Citation

Ground Metric Learning on Graphs

Heitz,

Bonneel,

Coeurjolly

et al. 2019

Preprint

View full text Add to dashboard Cite

Optimal transport (OT) distances between probability distributions are parameterized by the ground metric they use between observations. Their relevance for real-life applications strongly hinges on whether that ground metric parameter is suitably chosen. Selecting it adaptively and algorithmically from prior knowledge, the so-called ground metric learning (GML) problem, has therefore appeared in various settings. We consider it in this paper when the learned metric is constrained to be a geodesic distance on a graph that supports the measures of interest. This imposes a rich structure for candidate metrics, but also enables far more efficient learning procedures when compared to a direct optimization over the space of all metric matrices. We use this setting to tackle an inverse problem stemming from the observation of a density evolving with time: we seek a graph ground metric such that the OT interpolation between the starting and ending densities that result from that ground metric agrees with the observed evolution. This OT dynamic framework is relevant to model natural phenomena exhibiting displacements of mass, such as for instance the evolution of the color palette induced by the modification of lighting and materials.

show abstract

Simultaneous Ground Metric Learning and Matrix Factorization with Earth Mover's Distance

Cited by 13 publications

References 19 publications

FlowPool: Pooling Graph Representations with Wasserstein Gradient Flows

FlowPool: Pooling Graph Representations with Wasserstein Gradient Flows

Unsupervised Ground Metric Learning using Wasserstein Singular Vectors

Ground Metric Learning on Graphs

Contact Info

Product

Resources

About