On Benchmark Problems and Metrics for Decision Space Performance Analysis in Multi-Objective Optimization

Abstract

A number of benchmark problems exist for evaluating multi-objective evolutionary algorithms (MOEAs) in the objective space. However, the decision space performance analysis is a recent and relatively less explored topic in evolutionary multi-objective optimization research. Among other implications, such analysis can lead to designing more realistic test problems, gaining better understanding about optimal and robust design areas, and design and evaluation of knowledge-based optimization algorithms. This paper complements the existing research in this area and proposes a new method to generate multi-objective optimization test problems with clustered Pareto sets in hyper-rectangular defined areas of decision space. The test problem is parametrized to control number of decision variables, number and position of optimal areas in the decision space and modality of fitness landscape. Three leading MOEAs, including NSGA-II, NSGA-III, and MOEA/D, are evaluated on a number of problem instances with varying characteristics. A new metric is proposed that measures the performance of algorithms in terms of their coverage of the optimal areas in the decision space. The empirical analysis presented in this research shows that the decision space performance may not necessarily be reflective of the objective space performance and that all algorithms are sensitive to population size parameter for the new test problems.

A Framework to Handle Multimodal Multiobjective Optimization in Decomposition-Based Evolutionary Algorithms

“…This subsection shows results on the HPS and MMMOP test problems. Although the six HPS problem instances (HPS1, ..., HPS6) are proposed in [23], we use only HPS2. This is because all the other five problem instances are variants of HPS2 and also because the details of only HPS2 are provided in [23].…”

“…HPS [23] and MMMOP [24] have been recently proposed. However, HPS and MMMOP have so-called "distance-related" variables that affect only the distance between the objective vector and the Pareto front.…”

A Review of Evolutionary Multi-modal Multi-objective Optimization

“…Similar to SSUF1 and SSUF3, the MMF problems are derived from the idea of designing a problem that has multiple equivalent Pareto optimal solution subsets by mirroring the original one. A bottom-up framework for generating scalable test problems with any D is proposed in [57]. P equivalent Pareto optimal solution subsets are in P hyper-rectangular located in the solution space similar to the SYM-PART problems.…”

“…While the first k variables play the role of "position" parameters in the solution space, the other D − k variables represent "distance" parameters. The six HPS problem instances were constructed using this framework in [57].…”

“…CR is an alternative version of the maximum spread [38] to measure the spread of A. PSP is a combination of IGDX and CR. Performance indicators in the second category include the mean of the pairwise distance between two solutions [20], CS [16], SPS [16], the Solow-Polasky diversity measure [39] used in [22], [40], and PSV [57]. CS is the number of Pareto optimal solution subsets covered by at least one individual.…”