An efficient method for the numerical evaluation of partial derivatives of arbitrary order

Abstract

For any typical multivariable expression f , point a in the domain of f , and positive integer maxorder, this method produces the numerical values of all partial derivatives at a up through order maxorder. By the technique known as automatic differentiation, theoretically exact results are obtained using numerical (as opposed to symbolic) manipulation. The key ideas are a hyperpyramid data structure and a generalized Leibniz's rule. Any expression in n variables corresponds to a hyperpyramid array, in n -dimensional space, containing the numerical values of all unique partial derivatives (not wasting space on different permutations of derivatives). The arrays for simple expressions are combined by hyperpyramid operators to form the arrays for more complicated expressions. These operators are facilitated by a generalized Leibniz's rule which, given a product of multivariable functions, produces any partial derivative by forming the minimum number of products (between two lower partials) together with a product of binomial coefficients. The algorithms are described in abstract pseudo-code. A section on implementation shows how these ideas can be converted into practical and efficient programs in a typical computing environment. For any specific problem, only the expression itself would require recoding.

Introduction to Automatic Differentiation and MATLAB Object-Oriented Programming

“…>> fgradf (20,44,9) ans = 56.0461 1.0717 1.9505 1.4596 One can even compute an automatic Jacobian of a function F : R n → R n and use it in a multivariable Newton's method to solve a system of nonlinear equations. For a numerical analysis class, this makes a nice AD and MATLAB programming challenge.…”

“…However, the convolutions use rectangular subarrays taken from the corners. Addressing these parts of the nonrectangular n-dimensional data structures (usually implemented in a linear array) becomes a focus for successful implementation of such a method [2], [20].…”

Directions for computing truncated multivariate Taylor series

“…For n, d = 10, this is 30,045,015 flops for every function or operation in the expression. Moreover, much of the effort in implementing such a scheme goes into finding the array addresses of pyramid entries a ϕ and b ψ−ϕ [1], [11], [12], [15], [18]. The interpolation idea will avoid these complexities by doing automatic differentiation only on univariate coefficient lists.…”

Nonlocal automated comparative static analysis

“…A similar approach is suggested in refs. [19], [22], and [27, ments Feed in ADA, Jerrell [8] implements Feed in C++, and Wexler [34] implements Feed in C. One advantage of operator overloading is that expressions to be diiFerentiated can be written in a natural way; user-provided Wengert lists are not needed. Nevertheless, for clarity, the imple mentation of Feed for example (8) will be illustrated here using the simpler approach proposed in [16].…”

“…However, algorithms designed specific^y for the evaluation of higher-order partial derivatives are proposed in refs. [16,22,35], and software for this purpose is available from the authors upon request. As suggested by the application reviewed in section 2, there are good reasons to urge the continued development of efficient, user-friendly, portable modules for the automatic evaluation of higherorder partial derivatives.…”