Reinforcement learning for combinatorial optimization: A survey

Mazyavkina, Nina; Sviridov, Sergey; Ivanov, Sergei Evgenievich; Burnaev, Evgeny

doi:10.1016/j.cor.2021.105400

Cited by 358 publications

(147 citation statements)

References 47 publications

Supporting

Mentioning

108

Contrasting

Unclassified

Order By: Relevance

“…These architectures were trained just via the supervision of the objective function shown in Equation (1). Several RL algorithms were applied so far to solve TSP (and CO problems in general [35]). The actor-critic algorithm was employed in [36].…”

Section: Literature Reviewmentioning

confidence: 99%

A New Constructive Heuristic Driven by Machine Learning for the Traveling Salesman Problem

2021

View full text Add to dashboard Cite

Recent systems applying Machine Learning (ML) to solve the Traveling Salesman Problem (TSP) exhibit issues when they try to scale up to real case scenarios with several hundred vertices. The use of Candidate Lists (CLs) has been brought up to cope with the issues. A CL is defined as a subset of all the edges linked to a given vertex such that it contains mainly edges that are believed to be found in the optimal tour. The initialization procedure that identifies a CL for each vertex in the TSP aids the solver by restricting the search space during solution creation. It results in a reduction of the computational burden as well, which is highly recommended when solving large TSPs. So far, ML was engaged to create CLs and values on the elements of these CLs by expressing ML preferences at solution insertion. Although promising, these systems do not restrict what the ML learns and does to create solutions, bringing with them some generalization issues. Therefore, motivated by exploratory and statistical studies of the CL behavior in multiple TSP solutions, in this work, we rethink the usage of ML by purposely employing this system just on a task that avoids well-known ML weaknesses, such as training in presence of frequent outliers and the detection of under-represented events. The task is to confirm inclusion in a solution just for edges that are most likely optimal. The CLs of the edge considered for inclusion are employed as an input of the neural network, and the ML is in charge of distinguishing when such edge is in the optimal solution from when it is not. The proposed approach enables a reasonable generalization and unveils an efficient balance between ML and optimization techniques. Our ML-Constructive heuristic is trained on small instances. Then, it is able to produce solutions—without losing quality—for large problems as well. We compare our method against classic constructive heuristics, showing that the new approach performs well for TSPLIB instances up to 1748 cities. Although ML-Constructive exhibits an expensive constant computation time due to training, we proved that the computational complexity in the worst-case scenario—for the solution construction after training—is O(n2logn2), n being the number of vertices in the TSP instance.

show abstract

Section: Literature Reviewmentioning

confidence: 99%

A New Constructive Heuristic Driven by Machine Learning for the Traveling Salesman Problem

2021

View full text Add to dashboard Cite

show abstract

“…Connected to the development of deep learning techniques, these approaches are able to solve small instances of constraint problems, but are not competitive with respect to the capabilities of state-of-the-art constraint solvers. A recent survey on the usage of reinforcement learning for combinatorial optimization can be found in [66].…”

Section: Learning To Solvementioning

confidence: 99%

Predictive Machine Learning of Objective Boundaries for Solving COPs

Spieker

Gotlieb

2021

View full text Add to dashboard Cite

Solving Constraint Optimization Problems (COPs) can be dramatically simplified by boundary estimation, that is providing tight boundaries of cost functions. By feeding a supervised Machine Learning (ML) model with data composed of the known boundaries and extracted features of COPs, it is possible to train the model to estimate the boundaries of a new COP instance. In this paper, we first give an overview of the existing body of knowledge on ML for Constraint Programming (CP), which learns from problem instances. Second, we introduce a boundary estimation framework that is applied as a tool to support a CP solver. Within this framework, different ML models are discussed and evaluated regarding their suitability for boundary estimation, and countermeasures to avoid unfeasible estimations that avoid the solver finding an optimal solution are shown. Third, we present an experimental study with distinct CP solvers on seven COPs. Our results show that near-optimal boundaries can be learned for these COPs with only little overhead. These estimated boundaries reduce the objective domain size by 60-88% and can help the solver find near-optimal solutions early during the search.

show abstract

“…It provides guidelines for perspective research directions at the intersection of these two disciplines based on identified present shortcomings and perceived future advantages. The work of Mazyavkina et al [6] has a more narrow focus as it explores reinforcement learning as a sole tool for solving combinatorial optimization problems.…”

Section: Related Workmentioning

confidence: 99%

“…In contrast, Bengio et al [5] focus on any N P-hard combinatorial optimization problem. Mazyavkina et al [6] investigate reinforcement learning as a sole tool for approximating combinatorial optimization problems of any kind (not specifically those defined on graphs), whereas we survey all machine learning methods developed or applied for solving combinatorial optimization problems with focus on those tasks formulated on graphs. We also differ in the audience, who for Bengio et al [5] and Mazyavkina et al [6] is primarily the machine learning, mathematical and operations research communities (as explicitly stated in [6] and implicitly in [5] through the specialized literature discussed therein).…”

Section: Related Workmentioning

confidence: 99%

Learning Combinatorial Optimization on Graphs: A Survey with Applications to Networking

Vesselinova,

Steinert,

Perez-Ramirez

et al. 2020

Preprint

View full text Add to dashboard Cite

Existing approaches to solving combinatorial optimization problems on graphs suffer from the need to engineer each problem algorithmically, with practical problems recurring in many instances. The practical side of theoretical computer science, such as computational complexity then needs to be addressed. Relevant developments in machine learning research on graphs is surveyed, for this purpose. We organize and compare the structures involved with learning to solve combinatorial optimization problems, with a special eye on the telecommunications domain and its continuous development of live and research networks.

show abstract

Reinforcement learning for combinatorial optimization: A survey

Cited by 358 publications

References 47 publications

A New Constructive Heuristic Driven by Machine Learning for the Traveling Salesman Problem

A New Constructive Heuristic Driven by Machine Learning for the Traveling Salesman Problem

Predictive Machine Learning of Objective Boundaries for Solving COPs

Learning Combinatorial Optimization on Graphs: A Survey with Applications to Networking

Contact Info

Product

Resources

About