Extreme Value Based Adaptive Operator Selection

Abstract

Credit Assignment is an important ingredient of several proposals that have been made for Adaptive Operator Selection. Instead of the average fitness improvement of newborn offspring, this paper proposes to use some empirical order statistics of those improvements, arguing that rare but highly beneficial jumps matter as much or more than frequent but small improvements. An extreme value based Credit Assignment is thus proposed, rewarding each operator with the best fitness improvement observed in a sliding window for this operator. This mechanism, combined with existing Adaptive Operator Selection rules, is investigated in an EC-like setting. First results show that the proposed method allows both the Adaptive Pursuit and the Dynamic Multi-Armed Bandit selection rules to actually track the best operators along evolution.

Dynamic Multi-Armed Bandits and Extreme Value-Based Rewards for Adaptive Operator Selection in Evolutionary Algorithms

“…This section reports on the empirical validation of the Extreme -Dynamic MultiArmed Bandit (Ex-DMAB) AOS, combining Extreme-Value-Based Credit Assignment and DMAB Operator Selection, first described in [5].…”

“…For the sake of self-containedness, this section summarizes both heuristics, referring the interested reader respectively to [4] and [5] for more details.…”

“…The main originality of the Credit Assignment proposed in [5] is to consider the extreme as opposed to the average instant reward. Let us compare an operator bringing frequent small improvements, and an operator bringing rare but large improvements.…”

“…This paper investigates the combination of an Operator Selection and Credit Assignment heuristics, first described in [4] and [5] respectively. The proposed Operator Selection rule ( [4], section 3.1), performs the dynamic operator selection by combining a Multi-Armed Bandit algorithm [6] with a statistical test for change point detection, the Page-Hinkley test [7], assuming an unbiased Credit Assignment mechanism.…”

“…The proposed Operator Selection rule ( [4], section 3.1), performs the dynamic operator selection by combining a Multi-Armed Bandit algorithm [6] with a statistical test for change point detection, the Page-Hinkley test [7], assuming an unbiased Credit Assignment mechanism. The proposed Credit Assignment ( [5], section 3.2) considers the extreme values of the fitness improvement due to an operator, claiming that rare but highly beneficial "jumps" matter as much or more than frequent but small improvements.…”