Instance Space Analysis of Combinatorial Multi-objective Optimization Problems

Abstract

In recent years, there has been a continuous stream of development in evolutionary multi-objective optimization (EMO) algorithms. The large quantity of existing algorithms introduces difficulty in selecting suitable algorithms for a given problem instance. In this paper, we perform instance space analysis on discrete multi-objective optimization problems (MOPs) for the first time under three different conditions. We create visualizations of the relationship between problem instances and algorithm performance for instance features previously identified using decision trees, as well an independent feature selection. The suitability of these features in discriminating between algorithm performance and understanding strengths and weaknesses is investigated. Furthermore, we explore the impact of various definitions of "good" performance. The visualization of the instance space provides an alternative method of algorithm discrimination by showing clusters of instances where algorithms perform well across the instance space. We validate the suitability of existing features and identify opportunities for future development.

An Instance Space Analysis of Constrained Multi-Objective Optimization Problems

“…We use a binary concept to define the 'goodness' of the measured performance with respect to others [7]. We consider the performance of an algorithm as a 'good' performance if the normalized HV is greater than zero and within 1% of the best algorithm on the same instance.…”

“…ISA has been employed successfully on related problem domains. For example, Yap et al, [7] performed an ISA of combinatorial multi-objective optimization problems (MOPs), discovering that MOEA/D is preferred, not only when the number of objectives increased, but also when the degree of conflict between objectives decreased. Similarly, Muñoz and Smith-Miles [8] analyzed the space of continuous singleobjective optimization problems, identifying that multi-modal instances with adequate global structure are hard to solve by most studied algorithms with exception to BIPOP-CMA-ES.…”

SELECTOR: Selecting a Representative Benchmark Suite for Reproducible Statistical Comparison

“…One approach to select the problem instances is the Instance Space Analysis (ISA) methodology [36], which uses visualization to assess the effect of instance characteristics on algorithm performance, by finding areas in the problem landscape space where some algorithms perform better than the others. The idea is to select instances that maximize the performance difference between algorithms to highlight their strengths and weaknesses [33,40]. Racing has also been explored for the selection of problems that highlight performance differences between algorithms [41].…”