Relational Deep Reinforcement Learning for Routing in Wireless Networks

Manfredi, Victoria; Wolfe, Alicia P.; Wang, Bing; Zhang, Xiaolan

doi:10.1109/wowmom51794.2021.00029

Cited by 8 publications

(15 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recent works on DNN-based routing can be divided into strategies that consider network topologies with limited link dynamics [11,14,30,31,43,[48][49][50][51][52], and the many fewer strategies that consider networks with mobile devices [23,27]. Those approaches that do consider device mobility have limitations: for instance, [27] focuses on choosing which relay to activate to establish network connectivity, rather than routing, while [23] focuses on scenarios with a few fixed flows and up to 50 devices.…”

Section: Deeprl For Routingmentioning

confidence: 99%

“…In comparison, our use of relational features, combined with our neighborhood features, allows us to define features independent of a specific network topology, which is critical for modeling mobility. No prior work on routing that we have seen uses relational features except for [30] which focuses on stationary networks rather than the mobile networks we consider here. The use of a DNN to represent the DeepRL agent's routing strategy, given our state and action features, additionally lets us take advantage of the generalization capabilities of DNNs.…”

Section: Dnns To Model Network Dynamicsmentioning

confidence: 99%

“…As a consequence, we do not have to model fairness between flows and can model each packet as an independent agent with only indirect effects on other packets. While [30] also models each packet as an independent agent, they assume that at each timestep only the packet at the front of the queue can be forwarded, rather than our assumption of all packets.…”

Section: Mobile Routing Mdp Formulationmentioning

confidence: 99%

“…Like in [30] for stationary devices, however, we make packets, rather than devices, the decision-making agents, to more clearly propagate reward back from a delivered packet to the intermediate devices that forwarded the packet and Q-value for taking action in state a s . .…”

Section: Mobile Routing Mdp Formulationmentioning

confidence: 99%

“…Recent works on designing DeepRL based routing strategies for wireless networks can be divided into two categories: those that consider network topologies with limited link dynamics [11,14,30,31,43,[48][49][50][51][52], and the many fewer works that consider networks with mobile devices [23,27]. Those works that do consider device mobility, however, have limitations: for instance, focusing on establishing network connectivity [27] or considering smaller networks with very few flows [23].…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Learning an Adaptive Forwarding Strategy for Mobile Wireless Networks: Resource Usage vs. Latency

Manfredi¹,

Wolfe²,

Zhang³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Designing effective routing strategies for mobile wireless networks is challenging due to the need to seamlessly adapt routing behavior to spatially diverse and temporally changing network conditions. In this work, we use deep reinforcement learning (DeepRL) to learn a scalable and generalizable single-copy routing strategy for such networks. We make the following contributions: i) we design a reward function that enables the DeepRL agent to explicitly trade-off competing network goals, such as minimizing delay vs. the number of transmissions per packet; ii) we propose a novel set of relational neighborhood, path, and context features to characterize mobile wireless networks and model device mobility independently of a specific network topology; and iii) we use a flexible training approach that allows us to combine data from all packets and devices into a single offline centralized training set to train a single DeepRL agent. To evaluate generalizeability and scalability, we train our DeepRL agent on one mobile network scenario and then test it on other mobile scenarios, varying the number of devices and transmission ranges. Our results show our learned single-copy routing strategy outperforms all other strategies in terms of delay except for the optimal strategy, even on scenarios on which the DeepRL agent was not trained. CCS CONCEPTS• Networks → Routing protocols; Network protocol design; Mobile ad hoc networks; • Computing methodologies → Sequential decision making; Neural networks.

show abstract

Section: Deeprl For Routingmentioning

confidence: 99%

Section: Dnns To Model Network Dynamicsmentioning

confidence: 99%

Section: Mobile Routing Mdp Formulationmentioning

confidence: 99%

Section: Mobile Routing Mdp Formulationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Learning an Adaptive Forwarding Strategy for Mobile Wireless Networks: Resource Usage vs. Latency

Manfredi¹,

Wolfe²,

Zhang³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

Transfer Learning in Deep Reinforcement Learning

Islam¹,

Abid²,

Rahman³

et al. 2022

Lecture Notes in Networks and Systems

View full text Add to dashboard Cite

Reinforcement learning has quickly risen in popularity because of its simple, intuitive nature and it's powerful results. In this paper, we study a number of reinforcement learning algorithms, ranging from asynchronous q-learning to deep reinforcement learning. We focus on the improvements they provide over standard reinforcement learning algorithms, as well as the impact of initial starting conditions on the performance of a reinforcement learning agent.

show abstract