Parallel Algorithms for Constrained Tensor Factorization via Alternating Direction Method of Multipliers

Liavas, Athanasios P.; Sidiropoulos, Nicholas D.

doi:10.1109/tsp.2015.2454476

Cited by 103 publications

(63 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There, stochastic gradient descent (SGD) could be a promising candidate for such a family of SDF solvers due to their simplicity and aptitude for large data sets. In addition, substantial progress has recently been made in the computation of tensor decompositions on Hadoop with algorithms based on alternating least squares (ALS) [69]- [71]. Third, let denote the column-wise vectorization of its argument and let be the number of elements in a tensor .…”

Section: Structured Data Fusionmentioning

confidence: 99%

Structured Data Fusion

Sorber

Barel

Lathauwer

2015

IEEE J. Sel. Top. Signal Process.

172

192

View full text Add to dashboard Cite

We present structured data fusion (SDF) as a framework for the rapid prototyping of knowledge discovery in one or more possibly incomplete data sets. In SDF, each data set-stored as a dense, sparse, or incomplete tensor-is factorized with a matrix or tensor decomposition. Factorizations can be coupled, or fused, with each other by indicating which factors should be shared between data sets. At the same time, factors may be imposed to have any type of structure that can be constructed as an explicit function of some underlying variables. With the right choice of decomposition type and factor structure, even well-known matrix factorizations such as the eigenvalue decomposition, singular value decomposition and QR factorization can be computed with SDF. A domain specific language (DSL) for SDF is implemented as part of the software package Tensorlab, with which we offer a library of tensor decompositions and factor structures to choose from. The versatility of the SDF framework is demonstrated by means of four diverse applications, which are all solved entirely within Tensorlab's DSL.Index Terms-Big data, tensor, data fusion, structured matrices, canonical polyadic decomposition, block term decomposition, structured factors, domain specific language.

show abstract

Section: Structured Data Fusionmentioning

confidence: 99%

Structured Data Fusion

Sorber

Barel

Lathauwer

2015

IEEE J. Sel. Top. Signal Process.

172

192

View full text Add to dashboard Cite

show abstract

“…Nevertheless, it has been observed by many researchers that the ADMM works extremely well for various applications involving nonconvex objectives, such as the nonnegative matrix factorization [37,38], phase retrieval [39], distributed matrix factorization [40], distributed clustering [41], sparse zero variance discriminant analysis [42], polynomial optimization [43], tensor decomposition [44], matrix separation [45], matrix completion [46], asset allocation [47], sparse feedback control [48] and so on. However, to the best of our knowledge, existing convergence analysis of ADMM for nonconvex problems is very limited -all known global convergence analysis needs to impose uncheckable conditions on the sequence generated by the algorithm.…”

mentioning

confidence: 99%

Convergence Analysis of Alternating Direction Method of Multipliers for a Family of Nonconvex Problems

Luo²,

2016

View full text Add to dashboard Cite

Abstract. The alternating direction method of multipliers (ADMM) is widely used to solve large-scale linearly constrained optimization problems, convex or nonconvex, in many engineering fields. However there is a general lack of theoretical understanding of the algorithm when the objective function is nonconvex. In this paper we analyze the convergence of the ADMM for solving certain nonconvex consensus and sharing problems. We show that the classical ADMM converges to the set of stationary solutions, provided that the penalty parameter in the augmented Lagrangian is chosen to be sufficiently large. For the sharing problems, we show that the ADMM is convergent regardless of the number of variable blocks. Our analysis does not impose any assumptions on the iterates generated by the algorithm, and is broadly applicable to many ADMM variants involving proximal update rules and various flexible block selection rules.

show abstract

“…Therefore, rough estimates of its computational complexity can be easily derived [24]. The estimate for the update of B according to Eq.…”

Section: Time Complexitymentioning

confidence: 99%

t-BNE: Tensor-based Brain Network Embedding

Cao

Wei

et al. 2017

Proceedings of the 2017 SIAM International Conference on Data Mining

View full text Add to dashboard Cite

Brain network embedding is the process of converting brain network data to discriminative representations of subjects, so that patients with brain disorders and normal controls can be easily separated. Computer-aided diagnosis based on such representations is potentially transformative for investigating disease mechanisms and for informing therapeutic interventions. However, existing methods either limit themselves to extracting graph-theoretical measures and subgraph patterns, or fail to incorporate brain network properties and domain knowledge in medical science. In this paper, we propose t-BNE, a novel Brain Network Embedding model based on constrained tensor factorization. t-BNE incorporates 1) symmetric property of brain networks, 2) side information guidance to obtain representations consistent with auxiliary measures, 3) orthogonal constraint to make the latent factors distinct with each other, and 4) classifier learning procedure to introduce supervision from labeled data. The Alternating Direction Method of Multipliers (ADMM) framework is utilized to solve the optimization objective. We evaluate t-BNE on three EEG brain network datasets. Experimental results illustrate the superior performance of the proposed model on graph classification tasks with significant improvement 20.51%, 6.38% and 12.85%, respectively. Furthermore, the derived factors are visualized which could be informative for investigating disease mechanisms under different emotion regulation tasks.

show abstract

Parallel Algorithms for Constrained Tensor Factorization via Alternating Direction Method of Multipliers

Cited by 103 publications

References 31 publications

Structured Data Fusion

Structured Data Fusion

Convergence Analysis of Alternating Direction Method of Multipliers for a Family of Nonconvex Problems

t-BNE: Tensor-based Brain Network Embedding

Contact Info

Product

Resources

About