Bilinear Generalized Approximate Message Passing—Part I: Derivation

Abstract

Abstract-In this paper, we extend the generalized approximate message passing (G-AMP) approach, originally proposed for high-dimensional generalized-linear regression in the context of compressive sensing, to the generalized-bilinear case, which enables its application to matrix completion, robust PCA, dictionary learning, and related matrix-factorization problems. Here, in Part I of a two-part paper, we derive our Bilinear G-AMP (BiG-AMP) algorithm as an approximation of the sum-product belief propagation algorithm in the high-dimensional limit, where central-limit theorem arguments and Taylor-series approximations apply, and under the assumption of statistically independent matrix entries with known priors. In addition, we propose an adaptive damping mechanism that aids convergence under finite problem sizes, an expectation-maximization (EM)-based method to automatically tune the parameters of the assumed priors, and two rank-selection strategies. In Part II of the paper, we will discuss the specializations of EM-BiG-AMP to the problems of matrix completion, robust PCA, and dictionary learning, and we will present the results of an extensive empirical study comparing EM-BiG-AMP to state-of-the-art algorithms on each problem.

Joint User Identification, Channel Estimation, and Signal Detection for Grant-Free NOMA

“…Finally, the JUICESD-RiGm algorithm is obtained by replacing (24) and (26) l,t = 0; for q = 1, 2, ..., Q Calculate v z l,t andẑ l,t , ∀l, t, based on (13)-(16); Calculate v y k,t andŷ k,t , ∀k, t, based on (17)- (22); end \\ CCE module Calculate the message of g k in each time slot according to (23); Calculate the message of g k by combining multiple messages in multiple time slots and the a-priori distribution of g k according to (27) and (25);…”

A nonconvex splitting method for symmetric nonnegative matrix factorization: Convergence analysis and optimality

“…Well-known algorithms include the multiplicative update [6], alternating projected gradient methods [16], alternating nonnegative least squares (ANLS) with the active set method [17] and a few recent methods such as the bilinear generalized approximate message passing [18], [19], as well as methods based on the block coordinate descent [20]. These methods often possess strong convergence guarantees (to Karush-Kuhn-Tucker (KKT) points of the NMF problem) and most of them lead to satisfactory performance in practice; see [8] and the references therein for detailed comparison and comments for different algorithms.…”

Bilinear Generalized Approximate Message Passing—Part II: Applications

“…1 To initialize for EM-BiG-AMP, we adapt the procedure outlined in [15] to our matrix-completion problem, giving the EM initializations and…”

“…(1) (2) and where the likelihood function is known and separable, i.e., (3) In Part I of the work [1], we proposed and derived the BiG-AMP algorithm, whose general form is summarized in [1,TABLE III]. We also uncovered special cases under which the general approach can be simplified, such as the scalar-variance BiG-AMP under possibly incomplete additive white Gaussian noise (PIAWGN) observations, as summarized in [1,TABLE IV], and its specialization to Gaussian priors, as summarized by the BiG-AMP-Lite algorithm in [1, Table V].…”

“…We also uncovered special cases under which the general approach can be simplified, such as the scalar-variance BiG-AMP under possibly incomplete additive white Gaussian noise (PIAWGN) observations, as summarized in [1,TABLE IV], and its specialization to Gaussian priors, as summarized by the BiG-AMP-Lite algorithm in [1, Table V]. In addition, Part I proposed an adaptive damping mechanism, an expectation-maximization (EM)-based method of tuning the parameters of , , and (in case they are unknown), and two methods to select the rank (in case it is unknown).…”