IMP: A message-passing algorithm for matrix completion

Abstract

A new message-passing (MP) method is considered for the matrix completion problem associated with recommender systems. We attack the problem using a (generative) factor graph model that is related to a probabilistic low-rank matrix factorization. Based on the model, we propose a new algorithm, termed IMP, for the recovery of a data matrix from incomplete observations. The algorithm is based on a clustering followed by inference via MP (IMP). The algorithm is compared with a number of other matrix completion algorithms on real collaborative filtering (e.g., Netflix) data matrices. Our results show that, while many methods perform similarly with a large number of revealed entries, the IMP algorithm outperforms all others when the fraction of observed entries is small. This is helpful because it reduces the well-known cold-start problem associated with collaborative filtering (CF) systems in practice.

Bilinear Generalized Approximate Message Passing—Part I: Derivation

“…Applying these approximations to (26) and absorbing -invariant terms into the term, we obtain (27) where we used the relationship (28) and defined…”

“…In our framework, would be chosen to induce sparsity, would represent the noiseless observations, and would model the (possibly noisy) observation mechanism. While a plethora of approaches to these problems have been proposed based on optimization techniques (e.g., [5]- [15]), greedy methods (e.g., [16]- [20]), Bayesian sampling methods (e.g., [21], [22]), variational methods (e.g., [23]- [27]), and discrete message passing (e.g., [28]), ours is based on the Approximate Message Passing (AMP) framework, an instance of loopy belief propagation (LBP) [29] that was recently developed to tackle linear [30]- [32] and generalized linear [33] inference problems encountered in the context of compressive sensing (CS). In the generalized-linear CS problem, one estimates from observations that are statistically coupled to the transform outputs through a separable likelihood function , where in this case the transform is fixed and known.…”

Graphical models concepts in compressed sensing

“…For higher values of the rank r, rigidity of the graph is related to uniqueness of the solution of the reconstruction problem [SC09]. Finally, message passing algorithms for this problem were studied in [KYP10,KM10a].…”

Parametric Bilinear Generalized Approximate Message Passing

“…In our framework, {p x nl } would be chosen to induce sparsity, Z = AX would represent the noiseless observations, and {p y ml |z ml } would model the (possibly noisy) observation mechanism. While a plethora of approaches to these problems have been proposed based on optimization techniques (e.g., [4]- [14]), greedy methods (e.g., [15]- [19]), Bayesian sampling methods (e.g., [20], [21]), variational methods (e.g., [22]- [26]), and discrete message passing (e.g., [27]), ours is based on the Approximate Message Passing (AMP) framework, an instance of loopy belief propagation (LBP) [28] that was recently developed to tackle linear [29]- [31] and generalized linear [32] inference problems encountered in the context of compressive sensing (CS). In the generalized-linear CS problem, one estimates x ∈ R N from observations y ∈ R M that are statistically coupled to the transform outputs z = Ax through a separable likelihood function p y|z (y|z), where in this case the transform A is fixed and known.…”