Large Scale Problems in Practice: The Effect of Dimensionality on the Interaction Among Variables

Abstract

Abstract. This article performs a study on correlation between pairs of variables in dependence on the problem dimensionality. Two tests, based on Pearson and Spearman coefficients, have been designed and used in this work. In total, 86 test problems ranging between 10 and 1000 variables have been studied. If the most commonly used experimental conditions are used, the correlation between pairs of variables appears, from the perspective of the search algorithm, to consistently decrease. This effect is not due to the fact that the dimensionality modifies the nature of the problem but is a consequence of the experimental conditions: the computational feasibility of the experiments imposes an extremely shallow search in case of high dimensions. An exponential increase of budget and population with the dimensionality is still practically impossible. Nonetheless, since real-world application may require that large scale problems are tackled despite of the limited budget, an algorithm can quickly improve upon initial guesses if it integrates the knowledge that an apparent weak correlation between pairs of variables occurs, regardless the nature of the problem.

A Local Search with a Surrogate Assisted Option for Instance Reduction

“…Techniques that perturb the variables separately, just like that used in this article, are known to be effective for large scale problems, see [24,17,15]. This observation was reported in the experimental study in [2]. Large scale problems are by no means easier than low-dimensional problems.…”

“…Under these experimental conditions, the algorithm "sees" the problem as separable: average Pearson and Spearman coefficients of the variables approach zero independently on the problem when the dimensionality grows, see [2].…”

Predicting Cervical Hyperextension Injury: A Covariance Guided Sine Cosine Support Vector Machine

“…When dealing with some particular optimization problems, the gradient information and hessian matrix are requirements for the deterministic optimization method to converge to the optimal solution [47]. Furthermore, in some scenarios [48]- [51], we find that the use of deterministic algorithms does not require gradient information. As for the swarm intelligence algorithms, it is hard to obtain the accurate hessian matrix of the objective function [52].…”

On the use of single non-uniform mutation in lightweight metaheuristics

“…Quite surprisingly though, recent evidence (Caraffini et al 2012) has shown that even non-separable functions can be handled by perturbing separately each variable at a time: while this approach does not necessarily lead to the detection of the optimum, it may still be able to detect promising areas of the decision space. In the same context, another study has shown that the correlation between pairs of variables appears to consistently decrease when the problem dimensionality increases (Caraffini et al 2017). In other words, non-separable problems in high dimensionalities can be tackled as if they are separable.…”

“…Based on the aforementioned evidence on the effect of separable operators on non-separable problems (Caraffini et al 2012(Caraffini et al , 2017, and being motivated by the results concerning the non-uniform mutation (Tang and Tseng 2013) in Genetic Algorithms, in the present work we aim to address the following research questions:…”