Use of Q-learning approaches for practical medium access control in wireless sensor networks

Abstract

This paper studies the potential of a novel approach to ensure more efficient and intelligent assignment of capacity through medium access control (MAC) in practical wireless sensor networks. Q-Learning is employed as an intelligent transmission strategy. We review the existing MAC protocols in the context of Q-learning. A recently-proposed, ALOHA and Q-Learning based MAC scheme, ALOHA-Q, is considered which improves the channel performance significantly with a key benefit of simplicity. Practical implementation issues of ALOHA-Q are studied. We demonstrate the performance of the ALOHA-Q through extensive simulations and evaluations in various testbeds. A new exploration/exploitation method is proposed to strengthen the merits of the ALOHA-Q against dynamic the channel and environment conditions.

Reinforcement Learning Based Adaptive Duty Cycling in LR-WPANs

“…A study of Q-learning approaches for medium access in wireless sensor networks, named ALOHA-Q, is presented in [65]. The implementation of ALOHA-Q is extended to grid, linear chain, and random topologies.…”

“…However, a limitation of ALOHA-Q, is to trade-off exploration and exploitation by setting parameters accordingly. To solve this problem, ALOHA-Q-DEPS [65] is proposed. It is similar in working to ALOHA-Q but uses a decreasing ε -greedy policy in which a transmission slot with highest q-value is selected by a node with probability 1 − and random slot with probability .…”

Three-Dimensional Path-Following Control of a Robotic Airship with Reinforcement Learning

“…Up to now, reinforcement learning has been extensively studied in robot control, traffic dispatch, communication control, and game decision-making [15][16][17][18]. In the field of aircraft navigation and control, reinforcement learning also witnessed many successful applications.…”

Q-Learning Based Adaptive Channel Selection for Underwater Sensor Networks

“…A limitation is the possible unfairness and inefficiency of noncooperative equilibria. Recently, reinforcement learning (RL) [22], [23] has successfully provided adaptive MAC layers for radio wireless sensor networks [24]- [26]. In these works, Q-Learning is used to enable dynamic spectrum access in LTE networks [24] and to schedule TDMAbased MAC schemes [25], [26].…”

“…Recently, reinforcement learning (RL) [22], [23] has successfully provided adaptive MAC layers for radio wireless sensor networks [24]- [26]. In these works, Q-Learning is used to enable dynamic spectrum access in LTE networks [24] and to schedule TDMAbased MAC schemes [25], [26]. In the UWA community, [27], [28] use Q-Learning to adapt transmission parameters to the temporal variations of the channel in a single user context.…”