Flexible, Portable, and Practicable Solution for Routing in VANETs: A Fuzzy Constraint Q-Learning Approach

Abstract

Vehicular ad hoc networks (VANETs) have been attracting interest for their potential uses in driving assistance, traffic monitoring, and entertainment systems. However, due to vehicle movement, limited wireless resources, and the lossy characteristics of a wireless channel, providing a reliable multihop communication in VANETs is particularly challenging. In this paper, we propose PFQ-AODV, which is a portable VANET routing protocol that learns the optimal route by employing a fuzzy constraint Q-learning algorithm based on ad hoc on-demand distance vector (AODV) routing. The protocol uses fuzzy logic to evaluate whether a wireless link is good or not by considering multiple metrics, which are, specifically, the available bandwidth, link quality, and relative vehicle movement. Based on an evaluation of each wireless link, the proposed protocol learns the best route using the route request (RREQ) messages and hello messages. The protocol can infer vehicle movement based on neighbor information when position information is unavailable. PFQ-AODV is also independent of lower layers. Therefore, PFQ-AODV provides a flexible, portable, and practicable solution for routing in VANETs. We show the effectiveness of the proposed protocol by using both computer simulations and real-world experiments.Index Terms-Fuzzy logic, Q-learning, routing protocol, vehicular ad hoc networks (VANETs). 0018-9545 © 2013 IEEECelimuge Wu (M'10) received the M.E. degree from Beijing Institute of Technology,

Fuzzy-Based Channel Selection for Location Oriented Services in Multichannel VCPS Environments

“…(4), (5), (6), (7), (8), (10), (11),and (12) Calculate Steady state probablities by eq (13) Calculate freezing probability by eq. (14) Calculate updated transmission probability by eq.…”

“…The static quantification of channel quality is unable to incorporate the impact of vehicular environments considering the dynamic characteristics of the parameters directly affecting channel quality. Moreover, fuzzy-based techniques have been effectively applied for the quantification of parameters in dynamic environments in various domains [13].…”

Network Performance Optimization for Low-Voltage Power Line Communications

“…Q-learning is a recent form reinforcement learning that does not need a model of its environment and works by estimating the values of state-action pairs. It learns behavior through trial-and-error interactions with a dynamic environment, and has been employed for path selection [24]. The algorithm maintains a Q-value Q (s, a) in a table for every state-action pair.…”

Reinforcement Learning endowed with safe veto policies to learn the control of Linked-Multicomponent Robotic Systems

“…The success of Q-Learning has lead to many applications, such as path planning [25,31], energy management [30], routing in vehicular ad-hoc networks [42], management of water resources [27], and production planning [10].…”