Q-LBR: Q-Learning Based Load Balancing Routing for UAV-Assisted VANET

Abstract

Although various unmanned aerial vehicle (UAV)-assisted routing protocols have been proposed for vehicular ad hoc networks, few studies have investigated load balancing algorithms to accommodate future traffic growth and deal with complex dynamic network environments simultaneously. In particular, owing to the extended coverage and clear line-of-sight relay link on a UAV relay node (URN), the possibility of a bottleneck link is high. To prevent problems caused by traffic congestion, we propose Q-learning based load balancing routing (Q-LBR) through a combination of three key techniques, namely, a low-overhead technique for estimating the network load through the queue status obtained from each ground vehicular node by the URN, a load balancing scheme based on Q-learning and a reward control function for rapid convergence of Q-learning. Through diverse simulations, we demonstrate that Q-LBR improves the packet delivery ratio, network utilization and latency by more than 8, 28 and 30%, respectively, compared to the existing protocol.

Unmanned Aerial Vehicle Communications for Civil Applications: A Review

“…Although few studies such as [228] [229] [230] [231] have considered load balancing routing protocols to address both complicated dynamic network environments as well as network traffic increase in future, it can be concluded from the comparison that majority of routing protocols do not take traffic load balancing into account [232]. Most of the routing protocols including multi-path routing have not been able to effectively balance the load of network as well as energy utilization.…”

Peer-to-peer overlay techniques for vehicular ad hoc networks: Survey and challenges

“…With the aim to improve routing performance and reduce the control overhead, these algorithms use the geographic positions (Global Positioning System (GPS)) of all the vehicles in routing processes, so the nodes in the network must be aware of each others geographic position. There are many examples like Geographic Landmark Routing (GLR) [93], Location-Aided Routing (LAR) [94], Distance Routing Effect Algorithm for Mobility (DREAM) [95], Analytical Hierarchical Process (AHP)-Based Multimetric Geographical-Routing Protocol (AMGRP) [96], Named Data VANET Protocol (NVP) [97], Multimetric Next-Hop Vehicle Selection for Geocasting in Vehicular Ad-Hoc Networks [98], VANET Routing Based on Real-Time Road-Vehicle Density [99], Adaptive Geographical Routing Based on Quality of Transmission for Urban Vehicular Networks (AGQOT) [100], Greedy Curvemetric Routing Protocol (GCRP) [101], Greedy Perimeter Stateless Routing (GPRS)-Modified (GPSR-M) [102], Maxduration-Minangle GPSR (MM-GPSR) [103], Connectivity-Aware Intersection-Based Routing (CAIR) Protocol [104], Distance and Signal Quality Aware Routing (DSQR) [105], Q-learning based Traffic-Aware Routing protocol (QTAR) [106], Position-Based Q-learning Routing (PBQR) [107], and Q-learning based Load Balancing Routing (Q-LBR) [108].…”

Primena Učenja Potkrepljivanjem U Protokolima Rutiranja Za Dinamičke Bežične Ad Hoc Mreže

“…[10] protok gubitak paketa, protok TnesorFlow, Phyton [11] da li je čvor prvi sused, broj skokova, isplativost akcije, kvalitet veze stepen uspešno isporučenih paketa, broj kolizija MAC okvira, kašnjenje, protok NS-2 [12] da li je poruka isporučena grid-u sa odredišnim čvorom stepen uspešno isporučenih paketa, broj skokova, kašnjenje, broj prosleđivanja, protok nije naglašeno [13] tip i destinacija kontrolnih paketa stepen uspešno isporučenih paketa, vreme od slanja zahteva do prijema odgovora, overhed NS-3 [14] reputacija i isplativost akcije odgovarajućeg čvora stepen uspešno isporučenih paketa, reputacija, korisnost nije naglašeno [15] direktna veza sa odredištem, odnosno broj skokova i proteklo vreme od poslednje konekcije stepen uspešno isporučenih paketa, kašnjenje ONE [16] broj skokova, pouzdanost veze, propusni opseg stepen uspešno isporučenih paketa, kašnjenje, prosečna dužina putanje, overhed NS-2 [17] razni neželjeni efekti / nisu vršene simulacije [18] opterećenje relejnog čvora i zagušenje zemaljske mreže stepen uspešno isporučenih paketa, iskorišćenost mreže, kašnjenje OPNET [19] da li je kontrolni paket stigao od čvora pošiljaoca stepen uspešno isporučenih paketa, kašnjenje, broj skokova, overhed QualNet [20] broj skokova, kvalitet veze protok, broj gejtvej čvorova nije naglašeno [21] uspešnost slanja paketa stepen uspešno isporučenih paketa, kašnjenje, overhed NS-2 [22] kvalitet veze, vreme isteka trajanja veze, kašnjenje stepen uspešno isporučenih paketa, kašnjenje QualNet [23] gubitak energije, brzina prenosa potrošnja energije, gubitak paketa, ukupno vreme prenosa, verovatnoća prekida komunikacije nije naglašeno Evaluacija performansi predloženih protokola vršena je u različitim simulacionim okruženjima, a neka od najčešće korišćenih su NS-3, NS-2, OPNET, Python, QualNet, MATLAB, itd. U simulacijama su posmatrane različite mrežne performanse, zavisno od toga šta je bio cilj optimizacije.…”