A Nonconvex Splitting Method for Symmetric Nonnegative Matrix Factorization: Convergence Analysis and Optimality

Lu, Songtao; Mei, Hong; Wang, Zhengdao

doi:10.1109/tsp.2017.2679687

Cited by 40 publications

(25 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Developing stochastic optimization algorithms for both stages amenable to large-scale implementations is also pertinent. Generalizing SPARTA and our analytical results to robust sparse PR and matrix recovery with outliers constitute worthwhile future directions too [60,61,20].…”

Section: Discussionmentioning

confidence: 88%

Sparse Phase Retrieval via Truncated Amplitude Flow

Wang

Zhang

Giannakis

et al. 2018

IEEE Trans. Signal Process.

127

178

View full text Add to dashboard Cite

This paper develops a novel algorithm, termed SPARse Truncated Amplitude flow (SPARTA), to reconstruct a sparse signal from a small number of magnitude-only measurements. It deals with what is also known as sparse phase retrieval (PR), which is NP-hard in general and emerges in many science and engineering applications. Upon formulating sparse PR as an amplitudebased nonconvex optimization task, SPARTA works iteratively in two stages: In stage one, the support of the underlying sparse signal is recovered using an analytically well-justified rule, and subsequently a sparse orthogonality-promoting initialization is obtained via power iterations restricted on the support; and, in the second stage, the initialization is successively refined by means of hard thresholding based gradient-type iterations. SPARTA is a simple yet effective, scalable, and fast sparse PR solver. On the theoretical side, for any n-dimensional k-sparse (k n) signal x with minimum (in modulus) nonzero entries on the order of (1/ √ k) x 2, SPARTA recovers the signal exactly (up to a global unimodular constant) from about k 2 log n random Gaussian measurements with high probability. Furthermore, SPARTA incurs computational complexity on the order of k 2 n log n with total runtime proportional to the time required to read the data, which improves upon the state-of-the-art by at least a factor of k. Finally, SPARTA is robust against additive noise of bounded support. Extensive numerical tests corroborate markedly improved recovery performance and speedups of SPARTA relative to existing alternatives. and ptychography, astronomy, optics, as well as array and coherent diffraction imaging. In these settings, optical sensors and detectors such as charge-coupled device cameras, photosensitive films, and human eyes record only the intensity (squared magnitude) of a light wave, but not the phase. In particular, solution to PR has led to significant accomplishments, including the discovery in 1953 of DNA double helical structure from diffraction patterns, and the characterization of aberrations in the Hubble Space Telescope from measured point spread functions [1]. Due to the absence of Fourier phase information, the one-dimensional (1D) Fourier PR problem is generally ill-posed. It can be shown that there are in fact exponentially many non-equivalent solutions beyond trivial ambiguities in the 1D PR case [2]. A common approach to overcome this ill-posedness is exploiting additional information on the unknown signal such as non-negativity, sparsity, or bounded magnitude [3,4,5]. Other viable solutions consist of introducing redundancy into the measurement transforming system to obtain over-sampled and short-time Fourier transform (STFT) measurements [6], random Gaussian measurements [7,8,9], and coded diffraction patterns using structured illumination and random masks [10,11,7], just to name a few; see [10] for contemporary reviews on the theory and practice of PR.Past PR approaches can be mainly categorized as convex and nonconvex ones. A popular class of ...

show abstract

Section: Discussionmentioning

confidence: 88%

Sparse Phase Retrieval via Truncated Amplitude Flow

Wang

Zhang

Giannakis

et al. 2018

IEEE Trans. Signal Process.

127

178

View full text Add to dashboard Cite

show abstract

“…To address the optimization problem of the CDE model, inspired by Non-convex Spiting framework [Lu et al, 2017], we design an effective NS-Alternating algorithm. In this algorithm, we first reformulate the problem (6) and alternately solve the subproblems of the reformulation.…”

Section: Optimization: Ns-alternatingmentioning

confidence: 99%

“…L(H, V; Λ) is the augmented Lagrangian. Note that, proximal term ||V − V (t) || 2 F and penalty parameter ξ (t) are added according to the study [Lu et al, 2017]. The optimization w.r.t.…”

Section: Representation Matrix Subproblemmentioning

confidence: 99%

MASTER: across Multiple social networks, integrate Attribute and STructure Embedding for Reconciliation

Zhang

et al. 2018

Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence

View full text Add to dashboard Cite

Recently, reconciling social networks receives significant attention. Most of the existing studies have limitations in the following three aspects: multiplicity, comprehensiveness and robustness. To address these three limitations, we rethink this problem and propose the MASTER framework, i.e., across Multiple social networks, integrate Attribute and STructure Embedding for Reconciliation. In this framework, we first design a novel Constrained Dual Embedding model by simultaneously embedding and reconciling multiple social networks to formulate our problem into a unified optimization. To address this optimization, we then design an effective algorithm called NS-Alternating. We also prove that this algorithm converges to KKT points. Through extensive experiments on real-world datasets, we demonstrate that MASTER outperforms the state-of-the-art approaches.

show abstract

“…Then, according to Point 1, the step-size should show an upward trend during the iterative process. However, the traditional approach and some related approaches [29,30] all make the step-size increasingly smaller during the iterative process, which does not meet our requirements.…”

mentioning

confidence: 92%

Nonparametric Tensor Completion Based on Gradient Descent and Nonconvex Penalty

Xiong

2019

Symmetry

View full text Add to dashboard Cite

Existing tensor completion methods all require some hyperparameters. However, these hyperparameters determine the performance of each method, and it is difficult to tune them. In this paper, we propose a novel nonparametric tensor completion method, which formulates tensor completion as an unconstrained optimization problem and designs an efficient iterative method to solve it. In each iteration, we not only calculate the missing entries by the aid of data correlation, but consider the low-rank of tensor and the convergence speed of iteration. Our iteration is based on the gradient descent method, and approximates the gradient descent direction with tensor matricization and singular value decomposition. Considering the symmetry of every dimension of a tensor, the optimal unfolding direction in each iteration may be different. So we select the optimal unfolding direction by scaled latent nuclear norm in each iteration. Moreover, we design formula for the iteration step-size based on the nonconvex penalty. During the iterative process, we store the tensor in sparsity and adopt the power method to compute the maximum singular value quickly. The experiments of image inpainting and link prediction show that our method is competitive with six state-of-the-art methods.which is similar to Tucker decomposition. In the second approach, Nuclear Norm Minimization (NNM) is often used to replace RM because RM is NP-hard [14]. Reference [15] directly minimized the tensor nuclear norm for tensor completion. Reference [16] proposed a dual frame for low-rank tensor completion via nuclear norm constraints. Since high-order tensors represent a higher dimensional space, some works [17,18] used the Riemannian manifold for tensor completion, which is still closely linked to RM. In addition, some works [19,20] converted the tensor into matrices and realized tensor completion by means of matrix completion, but they ignored the inner structure and correlation of the data.The aforementioned methods of tensor completion all require some hyperparameters, such as the upper bound of the rank in the low-rank constraint and the penalty coefficient for norms. However, the selection of these hyperparameters not only consumes a substantial amount of time but also determines the performance of the methods. To address this issue, we propose a novel Nonparametric Tensor Completion (NTC) method based on gradient descent and nonconvex penalty. We use gradient descent to solve the optimization problem of tensor completion and build a gradient tensor with tensor matricizations and Singular Value Decomposition (SVD). We select the optimal direction based on the scaled latent nuclear norm in each iteration. The step-size in gradient descent is regarded as a penalty parameter for the singular value, and we design a nonconvex penalty for it. Furthermore, during the iterative process, we store the tensor in sparsity and adopt the power method to compute the maximum singular value quickly. Experiments of image inpainting and link prediction show that our metho...

show abstract

A Nonconvex Splitting Method for Symmetric Nonnegative Matrix Factorization: Convergence Analysis and Optimality

Cited by 40 publications

References 47 publications

Sparse Phase Retrieval via Truncated Amplitude Flow

Sparse Phase Retrieval via Truncated Amplitude Flow

MASTER: across Multiple social networks, integrate Attribute and STructure Embedding for Reconciliation

Nonparametric Tensor Completion Based on Gradient Descent and Nonconvex Penalty

Contact Info

Product

Resources

About