We identify a class of root-searching methods that surprisingly outperform the bisection method on the average performance while retaining minmax optimality. The improvement on the average applies for any continuous distributional hypothesis. We also pinpoint one specific method within the class and show that under mild initial conditions it can attain an order of convergence of up to 1.618, i.e., the same as the secant method. Hence, we attain both an improved average performance and an improved order of convergence with no cost on the minmax optimality of the bisection method. Numerical experiments show that, on regular functions, the proposed method requires a number of function evaluations similar to current state-of-the-art methods, about 24% to 37% of the evaluations required by the bisection procedure. In problems with non-regular functions, the proposed method performs significantly better than the state-of-the-art, requiring on average 82% of the total evaluations required for the bisection method, while the other methods were outperformed by bisection. In the worst case, while current state-of-the-art commercial solvers required two to three times the number of function evaluations of bisection, our proposed method remained within the minmax bounds of the bisection method.

No abstract

No abstract

Current state-of-the-art multi-objective optimization solvers, by computing gradients of all 𝑚 objective functions per iteration, produce after 𝑘 iterations a measure of proximity to critical conditions that is upperbounded by 𝑂 (1/ √ 𝑘) when the objective functions are assumed to have 𝐿−Lipschitz continuous gradients; i.e. they require 𝑂 (𝑚/𝜖 2 ) gradient and function computations to produce a measure of proximity to critical conditions bellow some target 𝜖. We reduce this to 𝑂 (1/𝜖 2 ) with a method that requires only a constant number of gradient and function computations per iteration; and thus, we obtain for the first time a multi-objective descent-type method with a query complexity cost that is unaffected by increasing values of 𝑚. For this, a brand new multi-objective descent direction is identified, which we name the central descent direction, and, an incremental approach is proposed. Robustness properties of the central descent direction are established, measures of proximity to critical conditions are derived, and, the incremental strategy for finding solutions to the multi-objective problem is shown to attain convergence properties unattained by previous methods. To the best of our knowledge, this is the first method to achieve this with no additional a-priori information on the structure of the problem, such as done by scalarizing techniques, and, with no pre-known information on the regularity of the objective functions other than Lipschitz continuity of the gradients.

Backtracking is an inexact line search procedure that selects the first value in a sequence 𝑥 0 , 𝑥 0 𝛽, 𝑥 0 𝛽 2 ... that satisfies 𝑔(𝑥) ≤ 0 on R + with 𝑔(𝑥) ≤ 0 iff 𝑥 ≤ 𝑥 * . This procedure is widely used in descent direction optimization algorithms with Armijo-type conditions. It both returns an estimate in (𝛽𝑥 * , 𝑥 * ] and enjoys an upper-bound ⌈log 𝛽 𝜖/𝑥 0 ⌉ on the number of function evaluations to terminate, with 𝜖 a lower bound on 𝑥 * . The basic bracketing mechanism employed in several root-searching methods is adapted here for the purpose of performing inexact line searches, leading to a new class of inexact line search procedures. The traditional bisection algorithm for root-searching is transposed into a very simple method that completes the same inexact line search in at most ⌈log 2 log 𝛽 𝜖/𝑥 0 ⌉ function evaluations. A recent bracketing algorithm for root-searching which presents both minmax function evaluation cost (as the bisection algorithm) and superlinear convergence is also transposed, asymptotically requiring ∼ log log log 𝜖/𝑥 0 function evaluations for sufficiently smooth functions. Other bracketing algorithms for root-searching can be adapted in the same way. Numerical experiments suggest time savings of 50% to 80% in each call to the inexact search procedure.CCS Concepts: • Mathematics of computing → Solvers; Numerical analysis.

We find a searching method on ordered lists that surprisingly outperforms binary searching with respect to average query complexity while retaining minmax optimality. The method is shown to require O(log 2 log 2 n) queries on average while never exceeding log 2 n queries in the worst case, i.e. the minmax bound of binary searching. Our average results assume a uniform distribution hypothesis similar to those of previous authors under which the expected query complexity of interpolation search of O(log 2 log 2 n) is known to be optimal.Hence our method turns out to be optimal with respect to both minmax and average performance. We further provide robustness guarantees and perform several numerical experiments with both artificial and real data. Our results suggest that time savings range roughly from a constant factor of 10% to 50% to a logarithmic factor spanning orders of magnitude when different metrics are considered.

No abstract