Semi-Supervised Learning in Network-Structured Data via Total Variation Minimization

Abstract

We provide an analysis and interpretation of total variation (TV) minimization for semi-supervised learning from partially-labeled network-structured data. Our approach exploits an intrinsic duality between TV minimization and network flow problems. In particular, we use Fenchel duality to establish a precise equivalence of TV minimization and a minimum cost flow problem. This provides a link between modern convex optimization methods for non-smooth Lasso-type problems and maximum flow algorithms. We show how a primal-dual method for TV minimization can be interpreted as distributed network optimization. Moreover, we derive a novel condition on the network structure and available label information that ensures that TV minimization accurately learns (approximately) piecewise constant graph signals. This condition depends on the existence of sufficiently large network flows between labeled data points. We verify our analysis in numerical experiments.

Statistical Analysis of Multi-Relational Network Recovery

“…Besides the latent space models, sparsity [24] or clustering assumptions [25] have been used to impose low-dimensional structures in single-relational networks. An MRN can be seen as a combination of several heterogeneous single-relational networks.…”

Local Graph Clustering With Network Lasso

“…We have recently explored the relation between network flow problems and TV minimization [17], [18]. Loosely speaking, the solution of TV minimization is piece-wise constant over clusters whose boundaries have a small total weight.…”

“…The special case of (6) when α = 0 is studied in [17]. We can also interpret (6) as TV minimization using soft constraints instead of hard constraints [18]. While [18] enforcesx i for each seed node i ∈ S k , (6) uses soft constraints such that typicallŷ x i < 1 at seed nodes i ∈ S k .…”

“…We can also interpret (6) as TV minimization using soft constraints instead of hard constraints [18]. While [18] enforcesx i for each seed node i ∈ S k , (6) uses soft constraints such that typicallŷ x i < 1 at seed nodes i ∈ S k . Spectral methods use optimization problems similar to (6) but with the Laplacian quadratic form {i,j}∈E W i,j (x i − x j ) 2 instead of TV (7).…”

“…Building on our recent work on the duality between TV minimization and network flow optimization [17], [18], we show that the proposed nLasso clustering method can be interpreted (in a precise sense) as a flow-based clustering method [19], [31], [32], [34]. As detailed in Section III, our nLasso problem (and its dual flow optimization problem) is similar but different from the TV minimization problems (and its dual flow optimization problems) studied in [17], [18].…”