Swarm-Inspired Computing to Solve Binary Optimization Problems: A Backward Q-Learning Binarization Scheme Selector

Abstract

In recent years, continuous metaheuristics have been a trend in solving binary-based combinatorial problems due to their good results. However, to use this type of metaheuristics, it is necessary to adapt them to work in binary environments, and in general, this adaptation is not trivial. The method proposed in this work evaluates the use of reinforcement learning techniques in the binarization process. Specifically, the backward Q-learning technique is explored to choose binarization schemes intelligently. This allows any continuous metaheuristic to be adapted to binary environments. The illustrated results are competitive, thus providing a novel option to address different complex problems in the industry.

Binarization of Metaheuristics: Is the Transfer Function Really Important?

“…In the literature, there are several related works on binarization [ 30 , 42 ] that have laid the groundwork for investigations into this domain problem, as there are several practical applications where working in binary domains is necessary. Moreover, research has emerged on how the change of binarization schemes affects each iteration of the search process, such as time-varying binarization schemes [ 38 ] or binarization scheme selectors [ 2 , 32 , 60 ], where the influence of binarization schemes and their impact at both the problem level and each iteration of the search has been demonstrated.…”

“…As can be seen in Section 2.3 , metaheuristics perform the search process alternating between intensification (exploration) and diversification (exploitation). Previous proposals [ 2 , 31 , 40 , 60 , 61 ] used these phases as the states to use in Q-learning. In these papers, the authors determined the stage of the search process by calculating the population diversity.…”

Chaotic Binarization Schemes for Solving Combinatorial Optimization Problems Using Continuous Metaheuristics

“…Binary combinatorial problems, such as the Set Covering Problem [9,11,77,104,105], Knapsack Problem [109,110], or Cell Formation Problem [106], are increasingly common in the industry. Given the demand for good results in reasonable times, metaheuristics have begun to gain ground as resolution techniques.…”

“…In the literature [9,[104][105][106], it can be seen that the authors perform a static test to compare the experimental results to determine if there is any significant difference between each experiment. For this type of experimentation, a non-parametric statistical test must be applied.…”

“…The study of metaheuristics has grown in recent years, with hybridizations emerging as the current trend. There exist hybridizations between metaheuristics such as those proposed in [2][3][4][5], hyperheuristic approaches where a high-level metaheuristic guides another low-level one [6][7][8], approaches where machine learning techniques enhance metaheuristics [9][10][11], and other approaches in which chaos theory is utilized to modify the stochastic behavior of metaheuristics [12][13][14][15].…”