MAC Protocol for Underwater Acoustic Networks Based on Deep Reinforcement Learning

Abstract

This paper introduces a Medium Access Control (MAC) protocol for Underwater Acoustic Network (UAN) where one of the transmitter nodes equipped with a Deep Reinforcement Learning (DRL) agent learns the communication environment and adapts its transmission policy to maximize the network throughput. In contrast to a radio frequency (RF) wireless network where the propagation delay is ignored, the UAN experiences significant propagation delays in both transmission from source to sink and feedback from sink to source. When co-existing with a Time Division Multiple Access (TDMA) node or a slotted Aloha node, the DRL node learns to take advantage of the difference in propagation delays and occupy the spare time slots of the network to achieve minimum collision in the UAN environment. Computer simulation results demonstrate the performance gain of using the DRL agent in both synchronous and asynchronous timing models. CCS CONCEPTS• Networks → Network protocol design; Link-layer protocols.

Reinforcement Learning Based MAC Protocol (UW-ALOHA-QM) for Mobile Underwater Acoustic Sensor Networks

“…Two conference papers [31], [32] were published in 2019. They are inspired by a journal paper [24] which discusses Deep Reinforcement Learning (DRL) for heterogeneous wireless networks.…”

“…One of the conference papers [31] suggests a time synchronised mode and two non-synchronised modes to setup different simulation scenarios and the synchronised network shows the best performance in terms of the average channel throughput. This paper ignores the propagation delay of ACK packets from the sink nodes, which is not a practical assumption.…”

Distributed Learning for Dynamic Channel Access in Underwater Sensor Networks

“…By collecting the local sensing results from the neighboring transmitters, the receivers could assign vacant spectrum resources and optimal transmit powers. Recently, with the development of deep reinforcement learning (DRL) algorithms, dynamic DRL-based resource management problems were investigated in UASNs [ 53 , 54 ]. In [ 53 ], an agent node which uses DRL-based MAC protocol learns underwater environment and occupy the spare time slots to achieve minimum collision when coexisting with a time division multiple access based node and a slotted ALOHA-based node.…”

“…Recently, with the development of deep reinforcement learning (DRL) algorithms, dynamic DRL-based resource management problems were investigated in UASNs [ 53 , 54 ]. In [ 53 ], an agent node which uses DRL-based MAC protocol learns underwater environment and occupy the spare time slots to achieve minimum collision when coexisting with a time division multiple access based node and a slotted ALOHA-based node. The authors of [ 54 ] proposed a DRL-based multiple access protocol which maximizes the occupation of available time slots caused by long propagation delay or not used by other nodes.…”

“…The authors of [ 54 ] proposed a DRL-based multiple access protocol which maximizes the occupation of available time slots caused by long propagation delay or not used by other nodes. Despite the extensive amount of existing studies, including the works in [ 46 , 47 , 48 , 49 , 50 , 51 , 52 , 53 , 54 ] on dynamic channel access for UASNs, there is no existing work that considers autonomous channel access with no coordination between underwater sensors and utilizing the prediction information of link qualities of available acoustic channels at the same time.…”