Reinforcement Learning Based Routing in Networks: Review and Classification of Approaches

Abstract

Reinforcement learning (RL), which is a class of machine learning, provides a framework by which a system can learn from its previous interactions with its environment to efficiently select its actions in the future. RL has been used in a number of application fields, including game playing, robotics and control, networks, and telecommunications, for building autonomous systems that improve themselves with experience. It is commonly accepted that RL is suitable for solving optimization problems related to distributed systems in general and to routing in networks in particular. RL also has reasonable overheadin terms of control packets, memory and computation-compared to other optimization techniques used to solve the same problems. Since the mid-1990s, over 60 protocols have been proposed, with major or minor contributions in the field of optimal route selection to convey packets in different types of communication networks under various user QoS requirements. This paper provides a comprehensive review of the literature on the topic. The review is structured in a way that shows how network characteristics and requirements were gradually considered over time. Classification criteria are proposed to present and qualitatively compare existing RL-based routing protocols.INDEX TERMS Reinforcement learning, communication networks, routing protocols, path optimization, quality of service.

Latency and Lifetime Enhancements in Industrial Wireless Sensor Networks: A Q-Learning Approach for Graph Routing

“…The states, actions, rewards, and exploration approach for a specific problem are defined during design time and must consider the problem characteristics [19]. Further details of RL and the Q-Learning algorithm can be found in [18]- [21].…”

“…A survey on RL routing approaches for networks was recently presented in [21], but the centralized approaches described are not related to IWSN. A literature review in RL approaches for WSN networks is presented in [24] and a survey in [20] describes three main decentralized RL approaches for WSN, where each node has an agent to choose routes.…”

“…The number of iterations required depends on the complexity of the learning model, which is influenced by the number of states, actions, and the choice of the rewards function [19], [20], [26]. The agent should explore several times the available actions in each state, requiring many iterations to converge [21], [26]. The actions of the agent will also be associated with the schedule changes because the changes in G U will cause different reconfiguration patterns on the timeslots depending on the scheduling strategy.…”

Q-LBR: Q-Learning Based Load Balancing Routing for UAV-Assisted VANET

“…In recent years, artificial intelligence techniques, which include machine learning, have attracted a significant amount of interest from researchers of various fields [ 8 ]. Among such techniques, reinforcement learning (RL) is being investigated in wireless systems because it provides a solution to optimize the system parameters by learning the surrounding area in a dynamic and complicated wireless environment [ 10 , 11 , 12 ]. Q-learning is a representative RL, and studies on using this approach to allocate routing policies in a dynamically changing network environment have been conducted.…”

Towards A Machine Learning-Based Framework For Automated Design of Networking Protocols

“…RL is also used for designing routing protocols in WSNs. For a comprehensive survey on RL for routing, interested reader is referred to [44]. Authors in [28] propose QELAR a RL-based routing protocol for routing underwater sensor networks.…”