Effective utilization of mobile ad hoc underwater distributed networks is challenging due to high system costs and the harsh environment characterized by low bandwidth, large latency, high energy consumption, and node mobility. This work addresses the routing issue, which is critical in successfully establishing and utilizing an underwater network. In particular, it focuses on reinforcement learning (RL)-based routing algorithms, which possess the ability to explore the network environment and adapt routing decisions to the constantly changing topology of the network due to node mobility and energy usage. This paper presents a routing algorithm based on Q-learning, one of the RL approaches, with additional Kinematic and Sweeping features, therefore referred to as QKS. These two additional features are introduced to address the potential slow convergence associated with pure RL algorithms. The results of a detailed packet-level simulation have been obtained using the NS-2 open-source network simulator with underwater modeling additions. The energy efficiency, convergence, and delivery performance of QKS are compared with two other routing protocols for underwater networks, a basic flooding approach (ICRP (Liang, 2007)) and a basic Q-learning implementation (QELAR (Hu, 2010)), using simulations of networks with both fixed and mobile nodes.
Unlike conventional Pulsed Active Sonar (PAS), which listens for target echoes in between short-burst transmissions, High Duty Cycle (HDC) sonar attempts to detect echoes amidst the continual interference from source(s) transmitting with nearly 100% duty cycle. HDC sonar presents an additional processing parameter, not available with PAS, which is the processing interval. The processing interval is a selectable subset of time within a CAS repetition cycle used for coherent processing. Hence, the choice of processing interval may be used to tune the performance of the sonar to local environmental conditions and to the operational scenario. Theoretically, increasing the processing interval increases target detectability, but in practice other factors should also be considered. In real acoustic environments, sound propagation is subject to temporal and spectral spreading effects, and these may limit the processing gains to lower levels than expected. Target Doppler can also become a more significant issue with longer processing intervals. Shorter processing intervals provide an increased number and rate of detection opportunities, which can be a significant advantage, leading to improved target holding, localization, tracking, and classification. This paper describes the various expected effects of the processing interval on performance for continuous-time LFM signals. It presents an analysis conducted on the TREX'13 sea trial dataset, and shows various results achieved as a function of processing interval. The results are explained and compared with theoretical expectations, and show the complicating effects of a real acoustic environment. In particular, we see the limitation on performance gains with increasing the processing interval due to acoustic environmental spreading effects, the target's physical extent and Doppler effects. Comparisons are shown between echoes from three different targets: mobile compact, mobile extended, and fixed. The evaluation describes performance using the quantities of Received Level, SNR, echo time-extent, and delay bias.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.