Intersection-based Traffic-Aware Routing with Fuzzy Q-learning for Urban VANETs

He, Yi‐Da; He, Rongxi

doi:10.1109/spac53836.2021.9539982

“…The next-hop node selection in the IFRMFD scheme is a fuzzy-based method considering vehicle density, distance, and relative speed of vehicles. Also, Cao et al [30] presented an intersection-based routing scheme with a fuzzy multi-factor called IRFMF. Density, number of lanes, and traffic flow were considered inputs of the fuzzy system in selecting the intersections.…”

Section: Traditional Intersection-based Routing In Vnsmentioning

confidence: 99%

“… Transmission of CP packets at intersections such as ITAR-FQ [31] requires updating periodically, which increases latency and overhead, especially at high densities or high traffic loads.  Routing decision-making based on the local knowledge or two-hop neighbor's information due to the restricted view of vehicular topology leads to nodes trapped into the local maxima problem resulting in degradation of data routing efficiency [25][26][27][28][29][30][31][32].  Regardless of road constraints and traffic light conditions, routing efficiency would be restricted.…”

Section: Research Gapmentioning

confidence: 99%

“…SDN controller computes the best path based on Eqs. ( 29), (30) The request message is transmitted to the SDN controller if the destination is outside the local controller's communication range (line 8). SDN computes the transmission path, including a sequence of intersections and relay nodes.…”

Section: 𝑑𝑖𝑠𝑡 𝑒𝑠𝑡𝑖𝑚𝑎𝑡𝑒 (𝑣 𝑖 𝑣 𝑗 )mentioning

confidence: 99%

See 1 more Smart Citation

Hierarchical traffic light-aware routing via fuzzy reinforcement learning in software-defined vehicular networks

Naderi

¹

,

Mahdaee

²

,

Rahmani

³

2023

Peer-to-Peer Netw. Appl.

6

0

View full text Add to dashboard Cite

Lack of a fully vehicular topology view and restricted vehicles' movement to streets with the time-varying traffic light conditions have caused drastic gaps in the traditional vehicular routing protocols. A hierarchical traffic light-aware routing scheme called HIFS is proposed in this paper using fuzzy reinforcement learning and software-defined network (SDN) to fill these gaps. At the first level of our HIFS scheme, a utility-based intersections selection policy is presented using fuzzy logic that jointly considers delay estimation, curve distance, and predicted of moving vehicles towards intersections. Then, a fuzzy logic-based path selection policy is proposed to choose the paths with highest flexibility against the intermittent connectivity and increased traffic loads. Residual bandwidth, Euclidean distance, angular orientation, and congestion are considered inputs of the fuzzy logic system. Meanwhile, traffic light states and nodes' information are used to tune the output fuzzy membership functions via reinforcement learning algorithm. The efficiency of our scheme in controlling ambiguity and uncertainty of the vehicular environment is confirmed through simulations in various vehicle densities and different traffic lights duration. Simulation results show our HIFS scheme's superiority over the state-of-the-art methods in terms of delivery ratio, average delay, path length, and routing overhead.

show abstract

“…Position-based routing algorithms use Q-learning as the basis for decision making: The ITAR-FQ [ 25 ] algorithm architecture consists of two main parts: the real-time traffic aware process and road evaluation (RTAP-RE) and the routing decision process with fuzzy Q-learning (RDP-FQ). The RTAP-RE designs a road evaluation method to process traffic information and estimate road quality (RQ), with reference to the number of vehicles moving in the same/a different direction, the length of the road, packet generation time, current time, etc.…”

Section: Related Workmentioning

confidence: 99%

A Software-Defined Directional Q-Learning Grid-Based Routing Platform and Its Two-Hop Trajectory-Based Routing Algorithm for Vehicular Ad Hoc Networks

Yang

¹

,

Yen

²

,

Chang

³

2022

Sensors

View full text Add to dashboard Cite

Dealing with the packet-routing problem is challenging in the V2X (Vehicle-to-Everything) network environment, where it suffers from the high mobility of vehicles and varied vehicle density at different times. Many related studies have been proposed to apply artificial intelligence models, such as Q-learning, which is a well-known reinforcement learning model, to analyze the historical trajectory data of vehicles and to further design an efficient packet-routing algorithm for V2X. In order to reduce the number of Q-tables generated by Q-learning, grid-based routing algorithms such as the QGrid have been proposed accordingly to divide the entire network environment into equal grids. This paper focuses on improving the defects of these grid-based routing algorithms, which only consider the vehicle density of each grid in Q-learning. Hence, we propose a Software-Defined Directional QGrid (SD-QGrid) routing platform in this paper. By deploying an SDN Control Node (CN) to perform centralized control for V2X, the SD-QGrid considers the directionality from the source to the destination, real-time positions and historical trajectory records between the adjacent grids of all vehicles. The SD-QGrid further proposes the flows of the offline Q-learning training process and the online routing decision process. The two-hop trajectory-based routing (THTR) algorithm, which depends on the source–destination directionality and the movement direction of the vehicle for the next two grids, is proposed as a vehicle node to forward its packets to the best next-hop neighbor node in real time. Finally, we use the real vehicle trajectory data of Taipei City to conduct extensive simulation experiments with respect to four transmission parameters. The simulation results prove that the SD-QGrid achieved an over 10% improvement in the average packet delivery ratio and an over 25% reduction in the average end-to-end delay at the cost of less than 2% in average overhead, compared with two well-known Q-learning grid-based routing algorithms.

show abstract

“… Transmission of CP packets at intersections such as ITAR-FQ [31] requires updating periodically, which increases latency and overhead, especially at high densities or high traffic loads.  Routing decision-making based on the local knowledge or two-hop neighbor's information due to the restricted view of vehicular topology leads to nodes trapped into the local maxima problem resulting in degradation of data routing efficiency [25][26][27][28][29][30][31][32].  Regardless of road constraints and traffic light conditions, routing efficiency would be restricted.…”

Section: Research Gapmentioning

confidence: 99%

Hierarchical traffic light-aware routing via fuzzy reinforcement learning in software-defined vehicular networks

Naderi

¹

,

Mahdaee

²

,

Rahmani

³

2022

Preprint

2

0

View full text Add to dashboard Cite

Lack of a fully vehicular topology view and restricted vehicles' movement to streets with the time-varying traffic light conditions have caused drastic gaps in the traditional vehicular routing protocols. A hierarchical traffic light-aware routing scheme called HIFS is proposed in this paper using fuzzy reinforcement learning and software-defined network (SDN) to fill these gaps. At the first level of our HIFS scheme, a utility-based intersections selection policy is presented using fuzzy logic that jointly considers delay estimation, curve distance, and predicted of moving vehicles towards intersections. Then, a fuzzy logic-based path selection policy is proposed to choose the paths with highest flexibility against the intermittent connectivity and increased traffic loads. Residual bandwidth, Euclidean distance, angular orientation, and congestion are considered inputs of the fuzzy logic system. Meanwhile, traffic light states and nodes' information are used to tune the output fuzzy membership functions via reinforcement learning algorithm. The efficiency of our scheme in controlling ambiguity and uncertainty of the vehicular environment is confirmed through simulations in various vehicle densities and different traffic lights duration. Simulation results show our HIFS scheme's superiority over the state-of-the-art methods in terms of delivery ratio, average delay, path length, and routing overhead.

show abstract

Intersection-based Traffic-Aware Routing with Fuzzy Q-learning for Urban VANETs

Cited by 5 publications

References 13 publications

Hierarchical traffic light-aware routing via fuzzy reinforcement learning in software-defined vehicular networks

Hierarchical traffic light-aware routing via fuzzy reinforcement learning in software-defined vehicular networks

A Software-Defined Directional Q-Learning Grid-Based Routing Platform and Its Two-Hop Trajectory-Based Routing Algorithm for Vehicular Ad Hoc Networks

Hierarchical traffic light-aware routing via fuzzy reinforcement learning in software-defined vehicular networks

Contact Info

Product

Resources

About