Bilinear Generalized Approximate Message Passing—Part II: Applications

Abstract

Abstract-In this paper, we extend the generalized approximate message passing (G-AMP) approach, originally proposed for highdimensional generalized-linear regression in the context of compressive sensing, to the generalized-bilinear case. In Part I of this two-part paper, we derived our Bilinear G-AMP (BiG-AMP) algorithm as an approximation of the sum-product belief propagation algorithm in the high-dimensional limit, and proposed an adaptive damping mechanism that aids convergence under finite problem sizes, an expectation-maximization (EM)-based method to automatically tune the parameters of the assumed priors, and two rankselection strategies. Here, in Part II, we discuss the specializations of BiG-AMP to the problems of matrix completion, robust PCA, and dictionary learning, and present the results of an extensive empirical study comparing BiG-AMP to state-of-the-art algorithms on each problem. Our numerical results, using both synthetic and real-world datasets, demonstrate that EM-BiG-AMP yields excellent reconstruction accuracy (often best in class) while maintaining competitive runtimes.

Bilinear Generalized Approximate Message Passing—Part I: Derivation

“…In addition, we proposed an adaptive damping mechanism to aid convergence under realistic problem sizes, an expectation-maximization (EM)-based method to automatically tune the parameters of the assumed priors, and two rank-selection strategies. In Part II [1] of this two-part work, we detail the application of BiG-AMP to matrix completion, robust PCA, and dictionary learning, and we present the results of an extensive numerical investigation into the performance of BiG-AMP on both synthetic and real-world datasets. The results in [1] demonstrate that BiG-AMP yields excellent reconstruction 10 In some cases the singular values of could be used instead.…”

“…In high-dimensional inference problems, exact implementation of the SPA is impractical, motivating approximations of the 1 Another worthwhile objective could be to compute the joint MAP estimate ; we leave this to future work. [33] to solve the generalized CS problem, which exploits the "blessings of dimensionality" that arise when is a sufficiently large and dense and which was rigorously analyzed in [35].…”

“…Applying this methodology to EM-BiG-AMP, where , we obtain the rank-selection rule (113) Since depends on the application (e.g., matrix completion, robust PCA, dictionary learning), detailed descriptions of are postponed to [1].…”

“…In the case that , , and/or are completely unknown, they can be modeled as Gaussian-mixtures with mean/variance/weight parameters learned via EM [38]. In Part II [1], we detail the application of BiG-AMP to matrix completion, robust PCA, and dictionary learning, and present the results of an extensive numerical investigation into the performance of BiG-AMP in each application. Those empirical results demonstrate that BiG-AMP yields an excellent combination of estimation accuracy and runtime when compared to existing state-of-the-art algorithms for each application.…”

“…Finally, Section VI concludes. In Part II [1], we detail the application of BiG-AMP to matrix completion, robust PCA, and dictionary learning, and present the results of an extensive numerical investigation into the performance of BiG-AMP in each application.…”