A Tight Analysis of Bethe Approximation for Permanent

Anari, Nima; Rezaei, Alireza

doi:10.1109/focs.2019.000-3

Cited by 8 publications

(7 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…PROOF. The proof is similar to derivation of Equation ( 4) in [3], except their result is for the case that Q = P . Let A be the adjacency matrix of G. Then for a perfect matching M note that…”

Section: 2mentioning

confidence: 80%

Sequential importance sampling for estimating expectations over the space of perfect matchings

Alimohammadi¹,

Diaconis²,

Roghani³

et al. 2021

Preprint

View full text Add to dashboard Cite

This paper makes three contributions to estimating the number of perfect matching in bipartite graphs. First, we prove that the popular sequential importance sampling algorithm works in polynomial time for dense bipartite graphs. More carefully, our algorithm gives a (1 − )-approximation for the number of perfect matchings of a λ-dense bipartite graph, using) samples. With size n on each side and for 1 2 > λ > 0, a λ-dense bipartite graph has all degrees greater than (λ + 1 2 )n. Second, practical applications of the algorithm requires many calls to matching algorithms. A novel preprocessing step is provided which makes significant improvements.Third, three applications are provided. The first is for counting Latin squares, the second is a practical way of computing the greedy algorithm for a card guessing game with feedback, and the third is for stochastic block models. In all three examples, sequential importance sampling allows treating practical problems of reasonably large sizes.

show abstract

Section: 2mentioning

confidence: 80%

Sequential importance sampling for estimating expectations over the space of perfect matchings

Alimohammadi¹,

Diaconis²,

Roghani³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Our results improve the analysis of the Bethe permanent for such structured matrices. Previously, the best known analysis of the Bethe permanent showed an √ 2 N -approximation factor to the permanent [AR18]. The analysis in [AR18] is tight for general non-negative matrices and the authors showed that this bound cannot be improved without leveraging further structure.…”

Section: Resultsmentioning

confidence: 99%

“…Previously, the best known analysis of the Bethe permanent showed an √ 2 N -approximation factor to the permanent [AR18]. The analysis in [AR18] is tight for general non-negative matrices and the authors showed that this bound cannot be improved without leveraging further structure. Our next result is of similar flavor, and we provide an asymptotically tight example for Theorem 3.1 and Corollary 3.2.…”

Section: Resultsmentioning

confidence: 99%

“…Finally, we use the fact that the KL-divergence between µ and ν is non-negative to get the required upper bound on the scaled Sinkhorn approximation with a doubly stochastic witness Q (obtained from P ). This proof technique is inspired by the recent work of Anari and Rezaei [AR18] that gives a tight √ 2 N bound on the approximation ratio of the Bethe approximation for the permanent of an N × N non-negative matrix. Though this bound on the quality of the Bethe and scaled Sinkhorn approximations for nonnegative matrices with k distinct columns suffices for our PML applications, interestingly we show that it can be extended to non-negative matrices with bounded rank.…”

Section: Overview Of Techniquesmentioning

confidence: 99%

See 1 more Smart Citation

The Bethe and Sinkhorn Permanents of Low Rank Matrices and Implications for Profile Maximum Likelihood

Anari,

Charikar,

Shiragur

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

In this paper we consider the problem of computing the likelihood of the profile of a discrete distribution, i.e., the probability of observing the multiset of element frequencies, and computing a profile maximum likelihood (PML) distribution, i.e., a distribution with the maximum profile likelihood. For each problem we provide polynomial time algorithms that given n i.i.d. samples from a discrete distribution, achieve an approximation factor of exp (−O( √ n log n)), improving upon the previous best-known bound achievable in polynomial time of exp(−O(n 2/3 log n)) (Charikar, Shiragur and Sidford, 2019). Through the work of Acharya, Das, Orlitsky and Suresh (2016), this implies a polynomial time universal estimator for symmetric properties of discrete distributions in a broader range of error parameter.We achieve these results by providing new bounds on the quality of approximation of the Bethe and Sinkhorn permanents (Vontobel, 2012 and2014). We show that each of these are exp(O(k log(N/k))) approximations to the permanent of N × N matrices with non-negative rank at most k, improving upon the previous known bounds of exp(O(N )). To obtain our results on PML, we exploit the fact that the PML objective is proportional to the permanent of a certain Vandermonde matrix with √ n distinct columns, i.e. with non-negative rank at most √ n. As a by-product of our work we establish a surprising connection between the convex relaxation in prior work (CSS19) and the well-studied Bethe and Sinkhorn approximations.

show abstract

“…Definition B.3. (Vontobel, 2010;Anari & Rezaei, 2019) Let A ∈ R + n×n . The bethe permanent of A is defined as follows:…”

Section: B Matrix Permanent and Its Approximation With Bethe Permanentmentioning

confidence: 99%

Discovering Non-monotonic Autoregressive Orderings with Variational Inference

Li¹,

Trabucco²,

Park³

et al. 2021

Preprint

View full text Add to dashboard Cite

The predominant approach for language modeling is to process sequences from left to right, but this eliminates a source of information: the order by which the sequence was generated. One strategy to recover this information is to decode both the content and ordering of tokens. Existing approaches supervise content and ordering by designing problem-specific loss functions and pre-training with an ordering pre-selected. Other recent works use iterative search to discover problemspecific orderings for training, but suffer from high time complexity and cannot be efficiently parallelized. We address these limitations with an unsupervised parallelizable learner that discovers high-quality generation orders purely from training data-no domain knowledge required. The learner contains an encoder network and decoder language model that perform variational inference with autoregressive orders (represented as permutation matrices) as latent variables. The corresponding ELBO is not differentiable, so we develop a practical algorithm for end-to-end optimization using policy gradients. We implement the encoder as a Transformer with non-causal attention that outputs permutations in one forward pass. Permutations then serve as target generation orders for training an insertionbased Transformer language model. Empirical results in language modeling tasks demonstrate that our method is context-aware and discovers orderings that are competitive with or even better than fixed orders.

show abstract

A Tight Analysis of Bethe Approximation for Permanent

Cited by 8 publications

References 38 publications

Sequential importance sampling for estimating expectations over the space of perfect matchings

Sequential importance sampling for estimating expectations over the space of perfect matchings

The Bethe and Sinkhorn Permanents of Low Rank Matrices and Implications for Profile Maximum Likelihood

Discovering Non-monotonic Autoregressive Orderings with Variational Inference

Contact Info

Product

Resources

About