Q-Learning for energy balancing and avoiding the void hole routing protocol in underwater sensor networks

Javaid, Nadeem; Karim, Obaida Abdul; Sher, Arshad; Imran, Muhammad; Yasar, Ansar-Ul-Haque; Guizani, Mohsen

doi:10.1109/iwcmc.2018.8450289

Cited by 14 publications

(8 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, there have been few publications on such approaches for underwater networks. A few papers propose routing algorithms [25]- [29] and we have only found four studies recently published [30]- [32], [43] for fixed node underwater networks excluding our own prior work [7], [8].…”

Section: Previous Workmentioning

confidence: 99%

Reinforcement Learning Based MAC Protocol (UW-ALOHA-QM) for Mobile Underwater Acoustic Sensor Networks

2021

View full text Add to dashboard Cite

The demand for ocean exploration and exploitation is rapidly increasing and this has led to rapid growth in the market of mobile vehicles. Given the mobility, the key challenge is to design a highly adaptive solution with minimal signalling (and the associated delays) which current techniques have not fully addressed. Therefore, the mobility and associated challenges in the underwater channel necessitates the design of a new approach to Medium Access Control (MAC) which provides the capability to adapt to rapidly changing conditions with no reliance on signalling which causes delays. This paper proposes the UW-ALOHA-QM protocol, which uses reinforcement learning to allow nodes to adapt to the time varying environment through trial-and-error interaction and thereby improve network resilience and adaptability. Simulations are carried out in four distinct scenarios in which node mobility patterns are significantly different. Simulation results demonstrate that UW-ALOHA-QM provides up to 300% improvement in channel utilisation with respect to existing protocols designed for mobile networks.INDEX TERMS Medium access control, mobile sensor networks, reinforcement learning, Q-learning, underwater acoustic networks.

show abstract

Section: Previous Workmentioning

confidence: 99%

Reinforcement Learning Based MAC Protocol (UW-ALOHA-QM) for Mobile Underwater Acoustic Sensor Networks

2021

View full text Add to dashboard Cite

show abstract

“…While reinforcement learning based MAC protocols have been researched extensively for terrestrial networks, there has, however, been very little research into underwater reinforcement learning based protocols. Most of these are for routing [20]- [24] and only one protocol has been found for the MAC layer [25] which uses a reinforcement learning approach to extend the lifetime of underwater acoustic wireless sensor networks. The study was proposed in 2013 and the aim of the proposed protocol is to extend the lifetime of a network.…”

Section: Previous Workmentioning

confidence: 99%

Reinforcement Learning Based MAC Protocol (UW-ALOHA-Q) for Underwater Acoustic Sensor Networks

2019

View full text Add to dashboard Cite

The demand for regular monitoring of the marine environment and ocean exploration is rapidly increasing, yet the limited bandwidth and slow propagation speed of acoustic signals leads to low data throughput for underwater networks used for such purposes. This study describes a novel approach to medium access control that engenders efficient use of an acoustic channel. ALOHA-Q is a medium access protocol designed for terrestrial radio sensor networks and reinforcement learning is incorporated into the protocol to provide efficient channel access. In principle, it potentially offers opportunities for underwater network design, due to its adaptive capability and its responsiveness to environmental changes. However, preliminary work has shown that the achievable channel utilisation is much lower in underwater environments compared with the terrestrial environment. Three improvements are proposed in this paper to address key limitations and establish a new protocol (UW-ALOHA-Q). The new protocol includes asynchronous operation to eliminate the challenges associated with time synchronisation under water, offer an increase in channel utilisation through a reduction in the number of slots per frame, and achieve collision free scheduling by incorporating a new random back-off scheme. Simulations demonstrate that UW-ALOHA-Q provides considerable benefits in terms of achievable channel utilisation, particularly when used in large scale distributed networks. INDEX TERMS MAC protocol, medium Access control, reinforcement learning, underwater acoustic networks.

show abstract

“…Hence, energy consumption may occur by other aspects of issue.  Current protocols used single AUV for data collection, which increases end-to-end delay of each sensor and thus energy consumption rate is increased [16], [17]. In UWSN, presence of routing void holes leads to higher packet loss which makes the data unreliable.…”

Section: Introductionmentioning

confidence: 99%

REVOHPR: Relay-based Void Hole Prevention and Repair by Virtual Routing in Clustered Multi-AUV Underwater Wireless Sensor Network

Chaaf¹,

Msa²,

Muthanna³

et al. 2021

Preprint

View full text Add to dashboard Cite

Underwater Wireless Sensor Networks (UWSN) enables various oceanic applications which require effective packet transmission. In this case, sparse node distribution, dynamic network topology and inappropriate selection of relay nodes cause void holes. Addressing this problem, we present a Relay based Void Hole Prevention and Repair protocol (ReVOHPR) by multiple Autonomous Underwater Vehicles (AUV) for UWSN. ReVOHPR efficiently identifies and avoids void holes and trap relay nodes to avoid it. ReVOHPR adopts the following operations as Ocean Depth (levels) based Equal Cluster Formation, Dynamic Sleep Scheduling, Virtual Graph based Routing, and Relay Assisted Void Hole Repair. For energy efficient cluster forming, Entropy based Eligibility Ranking (E2R) is presented which elects stable cluster heads (CHs). Then, dynamic sleep scheduling is implemented Dynamic Kernel Kalman Filter (DK2F) algorithm in which Sleep and Active modes based on the nodes current status. Inter Cluster Routing is performed by maximum matching nodes which selects by Dual criteria and also data transmitted to AUV. Finally, void holes are detected and repair by Bi-Criteria Mayfly Optimization (BiCMO) algorithm. The BiCMO focuses on reducing the number of holes, data packet loss and maximizes Quality of Service (QoS) and energy efficiency of the networks. This protocol is timely deal with node failures in packet transmission via multi-hop routing. Simulation is implemented by NS3 (AquaSim module) simulator that evaluates the performance in network simulation for following metrics as average energy consumption, delay, packet delivery rate and throughput.

show abstract

Q-Learning for energy balancing and avoiding the void hole routing protocol in underwater sensor networks

Cited by 14 publications

References 11 publications

Reinforcement Learning Based MAC Protocol (UW-ALOHA-QM) for Mobile Underwater Acoustic Sensor Networks

Reinforcement Learning Based MAC Protocol (UW-ALOHA-QM) for Mobile Underwater Acoustic Sensor Networks

Reinforcement Learning Based MAC Protocol (UW-ALOHA-Q) for Underwater Acoustic Sensor Networks

REVOHPR: Relay-based Void Hole Prevention and Repair by Virtual Routing in Clustered Multi-AUV Underwater Wireless Sensor Network

Contact Info

Product

Resources

About