Fast Search of the Optimal Contraction Sequence in Tensor Networks

Liang, Ling; Xu, Jinhong; Deng, Lei; Yan, Mingyu; Hu, Xing; Zhang, Zheng; Li, Guoqi; Xie, Yuan

doi:10.1109/jstsp.2021.3051231

Cited by 9 publications

(4 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We perform this filtering step as early in the pipeline as possible to reduce the burden on subsequent stages. There has been extensive research on heuristics to restrict the minimum maximum depth in tensor network contraction orders, but our focus is somewhat different [14,25,31]. We are interested in enumerating every program of minimum depth, and our programs are quite small.…”

Section: Filter By Maximum Nesting Depthmentioning

confidence: 99%

“…Our efforts differ from such approaches as our decisions are made offline, without examining matrix inputs. In the dense case, our problem looks quite similar to the tensor contraction ordering problem, where the most important cost function is simply the loop nesting depth and storage cost of dense temporaries [14,25,31]. However, since we consider the sparsity of our tensors and kernels, our algorithm is more similar to query optimizers for databases [10,19,20], which use the theory of conjunctive query containment to reduce an input query to its smallest equivalent.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

An Asymptotic Cost Model for Autoscheduling Sparse Tensor Programs

Ahrens¹,

Kjølstad²,

Amarasinghe³

2021

Preprint

View full text Add to dashboard Cite

While loop reordering and fusion can make big impacts on the constant-factor performance of dense tensor programs, the effects on sparse tensor programs are asymptotic, often leading to orders of magnitude performance differences in practice. Sparse tensors also introduce a choice of compressed storage formats that can have asymptotic effects. Research into sparse tensor compilers has led to simplified languages that express these tradeoffs, but the user is expected to provide a schedule that makes the decisions. This is challenging because schedulers must anticipate the interaction between sparse formats, loop structure, potential sparsity patterns, and the compiler itself. Automating this decision making process stands to finally make sparse tensor compilers accessible to end users.We present, to the best of our knowledge, the first automatic asymptotic scheduler for sparse tensor programs. We provide an approach to abstractly represent the asymptotic cost of schedules and to choose between them. We narrow down the search space to a manageably small "Pareto frontier" of asymptotically undominated kernels. We test our approach by compiling these kernels with the TACO sparse tensor compiler and comparing them with those generated with the default TACO schedules. Our results show that our approach reduces the scheduling space by orders of magnitude and that the generated kernels perform asymptotically better than those generated using the default schedules.

show abstract

Section: Filter By Maximum Nesting Depthmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

An Asymptotic Cost Model for Autoscheduling Sparse Tensor Programs

Ahrens¹,

Kjølstad²,

Amarasinghe³

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Large einsum-related research is going on in tensor networks applicable in machine learning or quantum physics calculations, see e.g. [19,27,25,12,17].…”

Section: Introductionmentioning

confidence: 99%

Fast Evaluation of Finite Element Weak Forms Using Python Tensor Contraction Packages

Cimrman

2021

Preprint

View full text Add to dashboard Cite

In finite element calculations, the integral forms are usually evaluated using nested loops over elements, and over quadrature points. Many such forms (e.g. linear or multi-linear) can be expressed in a compact way, without the explicit loops, using a single tensor contraction expression by employing the Einstein summation convention. To automate this process and leverage existing high performance codes, we first introduce a notation allowing trivial differentiation of multi-linear finite element forms. Based on that we propose and describe a new transpiler from Einstein summation based expressions, augmented to allow defining multi-linear finite element weak forms, to regular tensor contraction expressions. The resulting expressions are compatible with a number of Python scientific computing packages, that implement, optimize and in some cases parallelize the general tensor contractions. We assess the performance of those packages, as well as the influence of operand memory layouts and tensor contraction paths optimizations on the elapsed time and memory requirements of the finite element form evaluations. We also compare the efficiency of the transpiled weak form implementations to the C-based functions available in the finite element package SfePy.

show abstract

“…On the other hand, the work in [16] proposed a polynomial algorithm that determines the optimal time complexity (the largest number of multiplications of elements among every single pairwise contraction, which is also adopt in [17]) on any tensor tree algorithm. Other works such as [18] and [19] contributed to prompting the efficiency on finding the optimal sequence, but they did not determine the intrinsic hardness of this problem. Therefore, a huge gap occurs between [15] and [16].…”

Section: Introductionmentioning

confidence: 99%

Towards a polynomial algorithm for optimal contraction sequence of tensor networks from trees

Liang

Deng

et al. 2019

Phys. Rev. E

View full text Add to dashboard Cite

The computational cost of contracting a tensor network depends on the sequence of contractions, but to decide the sequence of contractions with a minimal computational cost on an arbitrary network has been proved to be an NP-complete problem. In this work, we conjecture that the problem may be a polynomial one if we consider the computational complexity instead. We propose a polynomial algorithm for the optimal contraction complexity of tensor tree network, which is a specific and widely applied network structure. We prove that for any tensor tree network, the proposed algorithm can achieve a sequence of contractions that guarantees the minimal time complexity and a linear space complexity simultaneously. To illustrate the validity of our idea, numerical simulations are presented that evidence the significant benefits when the network scale becomes large. This work will have great potential for the efficient processing of various physical simulations and pave the way for the further exploration of the computational complexity of tensor contraction on arbitrary tensor networks.

show abstract

Fast Search of the Optimal Contraction Sequence in Tensor Networks

Cited by 9 publications

References 26 publications

An Asymptotic Cost Model for Autoscheduling Sparse Tensor Programs

An Asymptotic Cost Model for Autoscheduling Sparse Tensor Programs

Fast Evaluation of Finite Element Weak Forms Using Python Tensor Contraction Packages

Towards a polynomial algorithm for optimal contraction sequence of tensor networks from trees

Contact Info

Product

Resources

About