A Reinforcement Learning Approach to Adaptive Redundancy for Routing in Tactical Networks

Johnston, Matthew; Danilov, Claudiu; Larson, Kevin A.

doi:10.1109/milcom.2018.8599797

Cited by 12 publications

(20 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This study utilizes the AODV protocol for the route discovery process, and Q-learning is used to optimize the path discovery concerning QoS requirements. Johnston et al [19] have introduced an intelligent routing scheme for battel networks to meet the real-time requirements. In this scheme, an approach of Q-learning is utilized to generalize and learn the next-hops to perform successful transmission of unicast-packets to the end nodes.…”

Section: Related Workmentioning

confidence: 99%

Sparse Distributed Memory Approach for Reinforcement Learning Driven Efficient Routing in Mobile Wireless Network System

Vidyadhar¹,

Nagaraj²,

Sudha³

2021

IJACSA

View full text Add to dashboard Cite

In recent years, researchers have explored the applicability of Q-learning, a model-free reinforcement learning technology towards designing QoS-aware, resource-efficiency, and reliable routing techniquesin a dynamically changing network environment. However, Q-learning is based on tabular representation to characterize learned policies that frequently encounter a dimension disaster problem when introduced to the uncertain and dynamically changing network environment. In addition, the time required for agent learning in the training phase is too long, which makes it difficult for the agent to generalize the observation state efficiently. To this end, this paper attempts to overcome the overhead memory problems encountered in Q-learning-based routing techniques. In this paper, the study presents a novel memory-efficient intelligent routing mechanism based on adaptive Kanerva coding, which minimizes the storage cost required for storing large action and a state value. Unlike existing schemes, the proposed method optimizes memory requirements. Also, it enables better generalization by storing the learnable parameters of the function approximator present in the agent in a Kanerva-coding data structure. The Kanerva-coding is a sparse memory with distributed reading and writing mechanism which enables optimal compression and state abstractions for learning with fewer parameterized components making it highly memory efficient. The design and implementation of the proposed technique are done on the Anaconda tool. Simulation results demonstrate that the proposed technique can adaptively adjust the routing policy according to the varying network environment to meet the transmission requirements of different services with low memory requirements.

show abstract

Section: Related Workmentioning

confidence: 99%

Sparse Distributed Memory Approach for Reinforcement Learning Driven Efficient Routing in Mobile Wireless Network System

Vidyadhar¹,

Nagaraj²,

Sudha³

2021

IJACSA

View full text Add to dashboard Cite

show abstract

“…The SRR algorithm [8] uses the network parameters C and H 1 for the routing decisions primarily introduced in seminal CQ-routing [7]. Each node i has a H-factor, h(i, j, d) (i.e.…”

Section: A Srr (Cq+ Routing) Protocolmentioning

confidence: 99%

“…C-values) in their CQ routing protocol. To improve reliability and exploration speed of the CQ-routing, smart robust routing (SRR) algorithm [8] was proposed to add selective broadcasting actions. SRR utilizes heuristic rules on when to broadcast in order to improve robustness but keep the overall overhead under control.…”

Section: Introductionmentioning

confidence: 99%

DeepCQ+: Robust and Scalable Routing with Multi-Agent Deep Reinforcement Learning for Highly Dynamic Networks

Kaviani¹,

Ryu²,

Ahmed³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Highly dynamic mobile ad-hoc networks (MANETs) remain as one of the most challenging environments to develop and deploy robust, efficient, and scalable routing protocols. In this paper, we present DeepCQ+ routing protocol which, in a novel manner, integrates emerging multi-agent deep reinforcement learning (MADRL) techniques into existing Q-learning-based routing protocols and their variants, and achieves persistently higher performance across a wide range of topology and mobility configurations. While keeping the overall protocol structure of the Q-learning-based routing protocols, DeepCQ+ replaces statically configured parameterized thresholds and hand-written rules with carefully designed MADRL agents such that no configuration of such parameters is required a priori. Extensive simulation shows that DeepCQ+ yields significantly increased end-to-end throughput with lower overhead and no apparent degradation of end-to-end delays (hop counts) compared to its Q-learningbased counterparts. Qualitatively, and perhaps more significantly, DeepCQ+ maintains remarkably similar performance gains under many scenarios that it was not trained for in terms of network sizes, mobility conditions, and traffic dynamics. To the best of our knowledge, this is the first successful application of the MADRL framework for the MANET routing problem that demonstrates a high degree of scalability and robustness even under the environments that are outside the trained range of scenarios. This implies that our MARL-based DeepCQ+ design solution significantly improves the performance of Q-learningbased CQ+ baseline approach for comparison and increases its practicality and explainability because the real-world MANET environment will likely vary outside the trained range of MANET scenarios. Additional techniques to further increase the gains in performance and scalability are discussed.

show abstract

“…SRR (Smart Robust Routing)- [96] proposed an RL-based algorithm for routing in tactical networks (i.e., military battlefield networks) where applications require deterministic guarantees on network performance to meet mission requirements. Through RL, nodes learn stable paths and use them to forward unicast packets.…”

Section: R-crs (Natg) (Rl-based Cooperative Relay Selection) -mentioning

confidence: 99%

“…-High, when nodes periodically exchange link-state information (such as Q-values, energy consumption, locations, so on). The amount of control packet needed depends on the period of Hello packets as in [32], [35], [42], [47], [53], [54], [55], [63], [64], [65], [66], [72], [91], [93], [96]. Notice that routing protocols with high communication overhead may be inefficient under some network conditions.…”

Section: ) Communication Overheadmentioning

confidence: 99%

Reinforcement Learning Based Routing in Networks: Review and Classification of Approaches

Mammeri¹

2019

IEEE Access

174

View full text Add to dashboard Cite

Reinforcement learning (RL), which is a class of machine learning, provides a framework by which a system can learn from its previous interactions with its environment to efficiently select its actions in the future. RL has been used in a number of application fields, including game playing, robotics and control, networks, and telecommunications, for building autonomous systems that improve themselves with experience. It is commonly accepted that RL is suitable for solving optimization problems related to distributed systems in general and to routing in networks in particular. RL also has reasonable overheadin terms of control packets, memory and computation-compared to other optimization techniques used to solve the same problems. Since the mid-1990s, over 60 protocols have been proposed, with major or minor contributions in the field of optimal route selection to convey packets in different types of communication networks under various user QoS requirements. This paper provides a comprehensive review of the literature on the topic. The review is structured in a way that shows how network characteristics and requirements were gradually considered over time. Classification criteria are proposed to present and qualitatively compare existing RL-based routing protocols.INDEX TERMS Reinforcement learning, communication networks, routing protocols, path optimization, quality of service.

show abstract

A Reinforcement Learning Approach to Adaptive Redundancy for Routing in Tactical Networks

Cited by 12 publications

References 5 publications

Sparse Distributed Memory Approach for Reinforcement Learning Driven Efficient Routing in Mobile Wireless Network System

Sparse Distributed Memory Approach for Reinforcement Learning Driven Efficient Routing in Mobile Wireless Network System

DeepCQ+: Robust and Scalable Routing with Multi-Agent Deep Reinforcement Learning for Highly Dynamic Networks

Reinforcement Learning Based Routing in Networks: Review and Classification of Approaches

Contact Info

Product

Resources

About