Learning Value Functions in Interactive Evolutionary Multiobjective Optimization

Abstract

This paper proposes an interactive multi-objective evolutionary algorithm (MOEA) that attempts to learn a value function capturing the users' true preferences. At regular intervals, the user is asked to rank a single pair of solutions. This information is used to update the algorithm's internal value function model, and the model is used in subsequent generations to rank solutions incomparable according to dominance. This speeds up evolution towards the region of the Pareto front most desirable to the user. We take into account the most general additive value function as a preference model, and we empirically compare different ways to identify the value function that seems to be most representative with respect to the given preference information, different types of user preferences, and different ways to use the learned value function in the MOEA. Results on a number of different scenarios suggest that the proposed algorithm works well over a range of benchmark problems and types of user preferences.

From evolutionary computation to the evolution of things

“…In tandem with algorithmic advances, this has spurred renewed interest in Interactive Evolutionary Algorithms, which have been successfully applied to elicit user preferences and knowledge in many areas from design to art 51,52 . Recent results suggest a useful synergy, with periodic user interaction to incorporate preferences helping to focus search down to a more manageable set of dimensions 80 . Importantly, this involves eliciting user preferences in response to what is discovered to be possible, rather than a priori.…”

Multi-Objective Optimization Under Uncertainty With Real-Time Integrated Decision Making Applied to Structural Engineering

“…[5], are in principle the most accurate, because the algorithm is able to calculate the most representative value function with respect to the user preferences. DM preferences are specified at each interaction.…”

“…Two approaches have been explored: direct integration of the DM in the optimization process [3,4], by giving them the possibility to insert some reference point, and real-time interaction with the optimization algorithm [5], by expressing their preference each time after a defined number of iterations. The target point method follows the first approach.…”

“…This can lead to a more efficient exploration of specific regions of the Pareto front and reduce the computational cost of finding desirable solutions. Interactive MCDM approaches have been recently given more attention in the multi-objective optimization community [3,4,5]. A validation of this approach on simple test-cases is shown as well as its application to the design of a simple building structure under uncertainties with seismic hazard and snow loads.…”

A novel multi-objective optimization method for the pressurized reservoir in hydraulic robotics

“…However, many current interactive methods still depend on the preference model, which is used to identify the region of interest (Chaudhuri and Deb, 2010;Sinha et al, 2014) or refine the approximation of the Pareto front (Klamroth and Miettinen, 2008). Other studies built designer preference interactively by query to the designer (Pedro and Takahashi, 2013) or comparison of pairwise solutions by the designer (Branke et al, 2015;2016); in these schemes the designer is provided with only fractional information, instead of a big picture of the optimization potential in the current situation. The visualization of Pareto-optimal solutions is also often studied in specialized topics so that the designer can make decisions based on that visual information (Kollat and Reed, 2007;Blasco et al, 2008).…”