Jamming Strategy Optimization through Dual Q-Learning Model against Adaptive Radar

Abstract

Modern adaptive radars can switch work modes to perform various missions and simultaneously use pulse parameter agility in each mode to improve survivability, which leads to a multiplicative increase in the decision-making complexity and declining performance of the existing jamming methods. In this paper, a two-level jamming decision-making framework is developed, based on which a dual Q-learning (DQL) model is proposed to optimize the jamming strategy and a dynamic method for jamming effectiveness evaluation is designed to update the model. Specifically, the jamming procedure is modeled as a finite Markov decision process. On this basis, the high-dimensional jamming action space is disassembled into two low-dimensional subspaces containing jamming mode and pulse parameters respectively, then two specialized Q-learning models with interaction are built to obtain the optimal solution. Moreover, the jamming effectiveness is evaluated through indicator vector distance measuring to acquire the feedback for the DQL model, where indicators are dynamically weighted to adapt to the environment. The experiments demonstrate the advantage of the proposed method in learning radar joint strategy of mode switching and parameter agility, shown as improving the average jamming-to-signal radio (JSR) by 4.05% while reducing the convergence time by 34.94% compared with the normal Q-learning method.

Multifunctional Radar Cognitive Jamming Decision Based on Dueling Double Deep Q-Network

“…At present, DQN application in radar jamming decisions has achieved remarkable results [18]- [20], [24]. Nevertheless, analyzing the network structure and algorithm principle, the following aspects still deserve to be explored in depth.…”

“…The D3QN-based multifunctional radar decision-making method introduces the DDQN [24] to improve the decision accuracy. Furthermore, we improve the sample utilization and shorten the decision time through the prioritized experience replay mechanism [20].…”

“…Pan et al [23] applied the improved chaotic genetic algorithm to allocate jamming strategies and evaluated the jamming effect with the radar detection probability as the index. Liu et al [24] solved the jamming strategy allocation problem by comparing the differences between the Q-learning algorithm and Double Deep Q-Network (DDQN) algorithm. The above radar jamming decision methods have achieved the desired results to a certain extent.…”

“…The above radar jamming decision methods have achieved the desired results to a certain extent. However, there are still problems of slow decision speed and low accuracy due to the increased number of radar modes [18]- [20], [24]. This paper proposes a multifunctional radar jamming decision method based on Dueling Double Deep Q-Network (D3QN) to solve the above problems.…”

Signal Processing and Machine Learning for Smart Sensing Applications

“…Liu et al [ 9 ] developed a two-level framework for jamming decision-making against the adaptive radar and proposed a dual Q-learning model to optimize the jamming strategy. The jamming mode and pulse parameters were determined hierarchically, greatly reducing the dimensionality of the search space and improving the learning efficiency of the model.…”