On Optimizing Distributed Tucker Decomposition for Dense Tensors

Chakaravarthy, Venkatesan T.; Choi, Jee; Joseph, Douglas J.; Liu, Xing; Murali, Prakash; Sabharwal, Yogish; Sreedhar, D.

doi:10.1109/ipdps.2017.86

Cited by 29 publications

(25 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…MET (Memory Efficient Tucker) [14] tackles this challenge by adaptively ordering computations and performing them in a piecemeal [19] discuss a shared and distributed memory parallelization of an ALS-based TF for sparse tensors. [41] proposes optimizations of HOOI for dense tensors on distributed systems. The above methods depend on SVD for updating factor matrices, while P-TUCKER utilizes a row-wise update rule.…”

Section: Related Workmentioning

confidence: 99%

Scalable Tucker Factorization for Sparse Tensors - Algorithms and Discoveries

Park

Sael

et al. 2018

2018 IEEE 34th International Conference on Data Engineering (ICDE)

View full text Add to dashboard Cite

Given sparse multi-dimensional data (e.g., (user, movie, time; rating) for movie recommendations), how can we discover latent concepts/relations and predict missing values? Tucker factorization has been widely used to solve such problems with multi-dimensional data, which are modeled as tensors. However, most Tucker factorization algorithms regard and estimate missing entries as zeros, which triggers a highly inaccurate decomposition. Moreover, few methods focusing on an accuracy exhibit limited scalability since they require huge memory and heavy computational costs while updating factor matrices.In this paper, we propose P-TUCKER, a scalable Tucker factorization method for sparse tensors. P-TUCKER performs alternating least squares with a row-wise update rule in a fully parallel way, which significantly reduces memory requirements for updating factor matrices. Furthermore, we offer two variants of P-TUCKER: a caching algorithm P-TUCKER-CACHE and an approximation algorithm P-TUCKER-APPROX, both of which accelerate the update process. Experimental results show that P-TUCKER exhibits 1.7-14.1× speed-up and 1.4-4.8× less error compared to the state-of-the-art. In addition, P-TUCKER scales near linearly with the number of observable entries in a tensor and number of threads. Thanks to P-TUCKER, we successfully discover hidden concepts and relations in a large-scale real-world tensor, while existing methods cannot reveal latent features due to their limited scalability or low accuracy.

show abstract

Section: Related Workmentioning

confidence: 99%

Scalable Tucker Factorization for Sparse Tensors - Algorithms and Discoveries

Park

Sael

et al. 2018

2018 IEEE 34th International Conference on Data Engineering (ICDE)

View full text Add to dashboard Cite

show abstract

“…The TTM component of the framework is a special case of the MET approach, wherein no intermediate tensors are computed. For the Tucker decomposition of dense tensors, MATLAB [2], single-machine [30] and distributed [1,6] implementations have been proposed. Prior work has also studied the Tucker decomposition on the MapReduce platform [11].…”

Section: Procedurementioning

confidence: 99%

On Optimizing Distributed Tucker Decomposition for Sparse Tensors

Chakaravarthy¹,

Choi²,

Joseph³

et al. 2018

Proceedings of the 2018 International Conference on Supercomputing

Self Cite

View full text Add to dashboard Cite

The Tucker decomposition generalizes the notion of Singular Value Decomposition (SVD) to tensors, the higher dimensional analogues of matrices. We study the problem of constructing the Tucker decomposition of sparse tensors on distributed memory systems via the HOOI procedure, a popular iterative method. The scheme used for distributing the input tensor among the processors (MPI ranks) critically influences the HOOI execution time. Prior work has proposed different distribution schemes: an offline scheme based on sophisticated hypergraph partitioning method and simple, lightweight alternatives that can be used real-time. While the hypergraph based scheme typically results in faster HOOI execution time, being complex, the time taken for determining the distribution is an order of magnitude higher than the execution time of a single HOOI iteration. Our main contribution is a lightweight distribution scheme, which achieves the best of both worlds. We show that the scheme is nearoptimal on certain fundamental metrics associated with the HOOI procedure and as a result, near-optimal on the computational load (FLOPs). Though the scheme may incur higher communication volume, the computation time is the dominant factor and as the result, the scheme achieves better performance on the overall HOOI execution time. Our experimental evaluation on large real-life tensors (having up to 4 billion elements) shows that the scheme outperforms the prior schemes on the HOOI execution time by a factor of up to 3x. On the other hand, its distribution time is comparable to the prior lightweight schemes and is typically lesser than the execution time of a single HOOI iteration.

show abstract

“…For an N -dimensional tensor, the traditional methods necessitate the multiplication of the input tensor with O(N 2 ) matrices in each iteration of these algorithm, which gets very expensive as the dimensionality of tensor increases. Efficiently carrying them out for higher dimensional tensors has been the focus of recent work [8,15,18]. Specifically, [14] and [18] investigate the use of a data structure called dimension tree in order to reduce the number of such multiplications to N log N within an iteration of HOOI and CP-ALS algorithms, respectively, for sparse tensors.…”

Section: Introductionmentioning

confidence: 99%

“…Specifically, [14] and [18] investigate the use of a data structure called dimension tree in order to reduce the number of such multiplications to N log N within an iteration of HOOI and CP-ALS algorithms, respectively, for sparse tensors. For dense tensors, the sheer number of multiplications is not the most precise cost metric, and for this reason Choi et al [8] investigate the use of an optimal dimension tree structure, computed in O(4 N ) time using O(3 N ) space, that potentially performs more TTMs in total, yet yields the lowest actual operation count possible.…”

Section: Introductionmentioning

confidence: 99%

“…We conjecture that finding an optimal tree in the general case stays NP-hard. We introduce a fast greedy algorithm for finding the optimal order for a series of TTMs, which in turn enables us designing an exact algorithm for finding an optimal binary dimension tree in O(3 N ) time using O(2 N ) memory using a dynamic programming formulation, a significant improvement with respect to the state of the art requiring O(4 N ) time and O(3 N ) space [8]. We show, however, with a counter-example that an optimal dimension tree is not necessarily binary in the Tucker case.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Computing Dense Tensor Decompositions with Optimal Dimension Trees

Kaya

Robert

2018

Algorithmica

View full text Add to dashboard Cite

Dense tensor decompositions have been widely used in many signal processing problems including analyzing speech signals, identifying the localization of signal sources, and many other communication applications. Computing these decompositions poses major computational challenges for big datasets emerging in these domains. CANDECOMP/PARAFAC (CP) and Tucker formulations are the prominent tensor decomposition schemes heavily used in these fields, and the algorithms for computing them involve applying two core operations, namely tensor-times-matrix (TTM) and tensor-timesvector (TTV) multiplication, which are executed repetitively within an iterative framework. In the recent past, efficient computational schemes using a data structure called dimension tree, are employed to significantly reduce the cost of these two operations, through storing and reusing partial results that are commonly used across different iterations of these algorithms. This framework has been introduced for sparse CP and Tucker decompositions in the literature, and a recent work investigates using an optimal binary dimension tree structure in computing dense Tucker decompositions. In this paper, we investigate finding an optimal dimension tree for both CP and Tucker decompositions. We show that finding an optimal dimension tree for an N -dimensional tensor is NP-hard for both decompositions, provide faster exact algorithms for finding an optimal dimension tree in O(3 N ) time using O(2 N ) space for the Tucker case, and extend the algorithm to the case of CP decomposition with the same time and space complexities.

show abstract

On Optimizing Distributed Tucker Decomposition for Dense Tensors

Cited by 29 publications

References 14 publications

Scalable Tucker Factorization for Sparse Tensors - Algorithms and Discoveries

Scalable Tucker Factorization for Sparse Tensors - Algorithms and Discoveries

On Optimizing Distributed Tucker Decomposition for Sparse Tensors

Computing Dense Tensor Decompositions with Optimal Dimension Trees

Contact Info

Product

Resources

About