Error bounds for overdetermined and underdetermined generalized centred simplex gradients

Abstract

Using the Moore-Penrose pseudoinverse, this work generalizes the gradient approximation technique called centred simplex gradient to allow sample sets containing any number of points. This approximation technique is called the generalized centred simplex gradient. We develop error bounds and, under a full-rank condition, show that the error bounds have order O(∆ 2 ), where ∆ is the radius of the sample set of points used. We establish calculus rules for generalized centred simplex gradients, introduce a calculus-based generalized centred simplex gradient and confirm that error bounds for this new approach are also order O(∆ 2 ). We provide several examples to illustrate the results and some benefits of these new methods.

Adaptive Finite-Difference Interval Estimation for Noisy Derivative-Free Optimization

“…Most recently, Moré and Wild proposed a heuristic for estimating the second derivative for determining the forward-difference interval that checks two conditions: (1) if the noise dominates the second-order derivative; and (2) if the forward and backward difference is too large relative to the function values [19]. A comparison of the resulting errors between finite-difference and simplex gradients were analyzed in [3,12].…”

Approximating the diagonal of a Hessian: which sample set of points should be used

“…We now present a process to obtain a formula to approximate the diagonal entries of the Hessian. This process is assuming that the gradient is approximated via the GCSG [13]. Let…”

“…The approach is called generalized centered simplex gradient (GCSG) and it is created by retaining the k original points in the sample set and adding their reflection through the point of interest (see Definition 3.1). An error bound which applies to the underdetermined, determined and overdetermined cases is introduced in [13]. The error bound shows that the GCSG is O(∆ 2 S ) accurate.…”

Error Analysis of Surrogate Models Constructed through Operations on Sub-models

“…In [10,22], the authors introduce calculus rules (product, quotient, chain) for simplex gradients. Based on these rules, gradient approximations techniques are developed [9,13]. The authors of [11] present a Hessian approximation technique and develop another Hessian approximation technique based on calculus rules.…”