Improved Route Selection Approaches using Q-learning framework for 2D NoCs

Gupta, Niyati; Kumar, Manoj; Sharma, Ashish; Gaur, Manoj Singh; Laxmi, Vijay; Daneshtalab, Masoud; Ebrahimi, Masoumeh

doi:10.1145/2768177.2768180

Cited by 9 publications

(5 citation statements)

References 14 publications

(23 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The Q-value represents latency to reach destination d with router y, and the algorithm has to select router y with the lowest Q-value(or latency). An improvement of QCA as Credencebased-Q-routing & Probabilistic-Credence-based-Q-routing is proposed by Gupta et al [49]. It uses credence as a c-value to measure Q-value and makes the learning rate vary with the time that improves the learning process.…”

Section: ) Table Based -Using Table Entriesmentioning

confidence: 99%

Review, Analysis, and Implementation of Path Selection Strategies for 2D NoCs

et al. 2022

View full text Add to dashboard Cite

Recent advances in very-large-scale integration (VLSI) technologies have offered the capability of integrating thousands of processing elements onto a single silicon microchip. Multiprocessor systems-on-chips (MPSoCs) are the latest creation of this technology evolution. Network-on-Chip (NoC) is a scalable and promising interconnection solution used by MPSoCs to achieve high performance. Routing algorithms provide a path to a packet toward the destination. For this, these algorithms should exhibit two characteristics. First, the route selection function should provide enough degree of adaptiveness to avoid network congestion. Second, it should not offer stale information on network congestion status to the neighboring routers. Many researchers have investigated network congestion and proposed techniques to control/avoid congestion. Such congestion avoidance-based algorithms significantly improve NoC performance. However, they may result in hardware overhead for side network implementation to collect congestion status. This paper reviews the selection strategies used to reduce congestion in NoC and classifies them on the technique adopted to handle and propagate congestion information. Additionally, this paper provides the implementation and analysis details of some state-of-art selection methods.

show abstract

Section: ) Table Based -Using Table Entriesmentioning

confidence: 99%

Review, Analysis, and Implementation of Path Selection Strategies for 2D NoCs

et al. 2022

View full text Add to dashboard Cite

show abstract

“…Two improved versions of Q-routing, namely Credencebased Q-routing (CrQ-Routing) and Probabilistic Credence-based Q-routing (PCrQ-Routing), are proposed in [23] to capture the traffic congestion dynamically and to improve the learning process to select less congested paths. CrQ-Routing uses variable learning rates based on the inferred confidence to make the Q-value updates more efficient.…”

Section: A Prior Workmentioning

confidence: 99%

Fully-Echoed Q-Routing With Simulated Annealing Inference for Flying Adhoc Networks

Rovira-Sugranes

Afghah

et al. 2021

IEEE Trans. Netw. Sci. Eng.

View full text Add to dashboard Cite

Current networking protocols deem inefficient in accommodating the two key challenges of Unmanned Aerial Vehicle (UAV) networks, namely the network connectivity loss and energy limitations. One approach to solve these issues is using learning-based routing protocols to make close-tooptimal local decisions by the network nodes, and Q-routing is a bold example of such protocols. However, the performance of the current implementations of Q-routing algorithms is not yet satisfactory, mainly due to the lack of adaptability to continued topology changes. In this paper, we propose a fullecho Q-routing algorithm with a self-adaptive learning rate that utilizes Simulated Annealing (SA) optimization to control the exploration rate of the algorithm through the temperature decline rate, which in turn is regulated by the experienced variation rate of the Q-values. Our results show that our method adapts to the network dynamicity without the need for manual re-initialization at transition points (abrupt network topology changes). Our method exhibits a reduction in the energy consumption ranging from 7% up to 82%, as well as a 2.6 fold gain in successful packet delivery rate, compared to the state of the art Q-routing protocols 1 .

show abstract

“…However, Q-Routing does not always guarantee finding the shortest path and does not explore multiple forwarding options in parallel. • Credence-based Q-Routing (CrQ-Routing) and Probabilistic Credence-based Q-Routing (PCrQ-Routing) [215]: These two methods dynamically capture the traffic congestion to improve the learning process to select less congested paths. Both methods adapt to the current network conditions much faster than the conventional Q-Routing.…”

Section: B Ai-enabled Routing Protocolsmentioning

confidence: 99%

A Review of AI-enabled Routing Protocols for UAV Networks: Trends, Challenges, and Future Outlook

Rovira-Sugranes¹,

Razi²,

Afghah³

et al. 2021

Preprint

View full text Add to dashboard Cite

Unmanned Aerial Vehicles (UAVs), as a recently emerging technology, enabled a new breed of unprecedented applications in different domains. This technology's ongoing trend is departing from large remotely-controlled drones to networks of small autonomous drones to collectively complete intricate tasks time and cost-effectively. An important challenge is developing efficient sensing, communication, and control algorithms that can accommodate the requirements of highly dynamic UAV networks with heterogeneous mobility levels. Recently, the use of Artificial Intelligence (AI) in learning-based networking has gained momentum to harness the learning power of cognizant nodes to make more intelligent networking decisions. An important example of this trend is developing learning-powered routing protocols, where machine learning methods are used to model and predict topology evolution, channel status, traffic mobility, and environmental factors for enhanced routing.This paper reviews AI-enabled routing protocols designed primarily for aerial networks, with an emphasis on accommodating highly-dynamic network topology. To this end, we review the basics of UAV technology, different approaches to swarm formation, and commonly-used mobility models, along with their impact on networking paradigms. We proceed with reviewing conventional and AI-enabled routing protocols, including topology-predictive and self-adaptive learning-based routing algorithms. We also discuss tools, simulation environments, remote experimentation platforms, and public datasets that can be used for developing and testing AI-enabled networking protocols for UAV networks. We conclude by presenting future trends, and the remaining challenges in AI-based UAV Networking, for different aspects of routing, connectivity, topology control, security and privacy, energy efficiency, and spectrum sharing.

show abstract

Improved Route Selection Approaches using Q-learning framework for 2D NoCs

Cited by 9 publications

References 14 publications

Review, Analysis, and Implementation of Path Selection Strategies for 2D NoCs

Review, Analysis, and Implementation of Path Selection Strategies for 2D NoCs

Fully-Echoed Q-Routing With Simulated Annealing Inference for Flying Adhoc Networks

A Review of AI-enabled Routing Protocols for UAV Networks: Trends, Challenges, and Future Outlook

Contact Info

Product

Resources

About