The polyadic structure of factorable function tensors with applications to high-order minimization techniques

Jackson, Richard H F; McCormick, Garth P.

doi:10.1007/bf00938603

Cited by 27 publications

(21 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A function f : R n → R is a factorable function [5] if it can be represented as the last function in a finite sequence of functions…”

Section: Factorable Functions and Algorithmic Differentiationmentioning

confidence: 99%

“…The formulation of large-scale problems in many scientific applications naturally give rise to "structured" representation. Examples of computationally useful structures arising in large-scale problems include "unary functions" [1], "partially separable functions" [2], and "factorable functions" [5]. The main purpose of this note is to show that when the function is provided in a suitable form, these structures can be exploited automatically.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Structured computation in optimization and algorithmic differentiation

Hascoët

Hossain

Steihaug

2013

ACM Commun. Comput. Algebra

View full text Add to dashboard Cite

“…A function f : R n → R is a factorable function [5] if it can be represented as the last function in a finite sequence of functions…”

Section: Factorable Functions and Algorithmic Differentiationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Structured computation in optimization and algorithmic differentiation

Hascoët

Hossain

Steihaug

2013

ACM Commun. Comput. Algebra

View full text Add to dashboard Cite

“…Several previous studies (within the basic framework of SOAM) have focused on detecting and exploiting this symmetry, but most of them fall short in their ability to exploit the symmetry to its fullest performance benefit [3][4][5][6]. To be precise, a SOAM method actually computes a Hessian-vector product as a basic unit, and the full Hessian is then obtained via multiple Hessian-vector products.…”

Section: Introductionmentioning

confidence: 99%

Capitalizing on live variables: new algorithms for efficient Hessian computation via automatic differentiation

2016

View full text Add to dashboard Cite

We revisit an algorithm [called Edge Pushing (EP)] for computing Hessians using Automatic Differentiation (AD) recently proposed by Gower and Mello (Optim Methods Softw 27(2): 233-249, 2012). Here we give a new, simpler derivation for the EP algorithm based on the notion of live variables from data-flow analysis in compiler theory and redesign the algorithm with close attention to general applicability and performance. We call this algorithm Livarh and develop an extension of Livarh that incorporates preaccumulation to further reduce execution time-the resulting algorithm is called Livarhacc. We engineer robust implementations for both algorithms Livarh and Livarhacc within ADOL-C, a widely-used operator overloading based AD software tool. Rigorous complexity analyses for the algorithms are provided, and the performance of the algorithms is evaluated using a mesh optimization application and several kinds of synthetic functions as testbeds. The results show that the new algorithms outperform state-of-the-art sparse methods (based on sparsity pattern detection, coloring, compressed matrix evaluation, and recovery) in some cases by orders of magnitude. We have made our implementation available online as open-source software and it will be included in a future release of ADOL-C. B Alex Pothen

show abstract

“…For α = 0 the method is Chebyshev's method (the improved method of tangent hyperbolas, [9,18]). When α = 1/2 the method is the original Halley's method, see [16] and [17].…”

Section: Introductionmentioning

confidence: 99%

“…Assumption 2 The Hessian ∇ 2 f (x * ) is positive definite. We solve problem (1) with the third-order methods [6,7,9,10,12,16,17].…”

Section: Introductionmentioning

confidence: 99%

On the Halley class of methods for unconstrainedoptimization problems

Zhang

2010

Optimization Methods and Software

View full text Add to dashboard Cite

Third-order methods can be used to solve efficiently the unconstrained optimization problems, and they, in most cases, use fewer iterations but more computational cost per iteration than a second-order method to reach the same accuracy. Recently, it has been shown by an article that under some conditions the ratio of the number of arithmetic operations of a third-order method (the Halley class of methods) and Newton's method is constant (at most 5) per iteration. Automatic differentiation (AD) can compute fast and accurate derivatives such as the Jacobian, Hessian matrix and the tensor of the function. The Halley class of methods includes these high-order derivatives. In this paper, we apply AD efficiently to the methods and investigate the computational complexity of them. The results show that under general conditions even including the computation of the function and its derivative terms, the upper bound of the ratio can be reduced to 3.5.

show abstract

The polyadic structure of factorable function tensors with applications to high-order minimization techniques

Cited by 27 publications

References 16 publications

Structured computation in optimization and algorithmic differentiation

Structured computation in optimization and algorithmic differentiation

Capitalizing on live variables: new algorithms for efficient Hessian computation via automatic differentiation

On the Halley class of methods for unconstrainedoptimization problems

Contact Info

Product

Resources

About