A Local Search for Numerical Optimisation Based on Covariance Matrix Diagonalisation

Abstract

Pattern Search is a family of optimisation algorithms that improve upon an initial solution by performing moves along the directions of a basis of vectors. In its original definition Pattern Search moves along the directions of each variable. Amongst its advantages, the algorithm does not require any knowledge of derivatives or analytical expression of the function to optimise. However, the performance of Pattern Search is heavily problem dependent since the search directions can be very effective on some problems and lead to poor performance on others. The present article proposes a novel enhancement of Pattern Search that explores the space by using problem-dependent search directions. Some points are sampled within the basin of attraction and the diagonalisation of the covariance matrix associated with the distribution of these points is performed. The proposed Covariance Pattern Search improves upon an initial point by varying it along the directions identified by the eigenvectors of the covariance matrix.

Generalised Pattern Search Based on Covariance Matrix Diagonalisation

“…In the specific case, the proposed pattern search aims to detect a preferential pattern that suits the specific problems. This article extends our previous work presented in [42]. More specifically, while in [42] we introduced a specific implementation of a pattern search method that uses a fitness landscape analysis, we extensively present here the fitness landscape analysis method and its applicability to the entire pattern search family.…”

“…This article extends our previous work presented in [42]. More specifically, while in [42] we introduced a specific implementation of a pattern search method that uses a fitness landscape analysis, we extensively present here the fitness landscape analysis method and its applicability to the entire pattern search family. Furthermore, we investigate for the first time the theoretical standpoint of the proposed class of methods and provide a theoretical justification of the method.…”

Teaching Mathematics to Computer Scientists: Reflections and a Case Study

“…If only a distribution of points whose objective function value is below a threshold (in a minimisation problem) are saved in the data set, then this distribution describes the geometry of the optimisation problem, see [31]. Like for the case of the PCA, the diagonalisation of the associated covariance matrix, that is the detection of its eigenvectors provides the optimisation algorithm with a set of preferential search directions to perform the search for the the optimum, see [30]. However, unlike the case of the PCA, the most important direction (variable) is the least represented one as it would correspond to the direction with maximum directional gradient.…”

Covariance Pattern Search with Eigenvalue-determined Radii

“…In [25] a FLA based LS was proposed. The FLA in [25] samples points in the domain to generate a data set of candidate solutions whose objective function value is below a prearranged threshold (in a minimisation scenario). This data set provides some useful pieces of information about the geometry of the problem.…”

“…These eigenvectors are then used as a basis (reference system) to explore the space. More specifically, in [25], [26] the eigenvectors of the calculated covariance matrix are used to build the Pattern Matrix of an implementation of generalised Patter Search (PS) [27]. In [28], the same FLA has been successfully implemented in three LS algorithms composing Multiple Trajectory Search [29].…”