Target tracking strategy using deep deterministic policy gradient

You, Shixun; Diao, Ming; Gao, Lipeng; Zhang, Fulong; Wang, Huan

doi:10.1016/j.asoc.2020.106490

Cited by 33 publications

(15 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The dynamic interaction process can be defined by the Markov decision process (MDP) and divided into finite discrete time steps in the RL control framework 32 . At each time step

k

, the agent observes state

s_{k}

, takes action

a_{k}

, receives scalar reward

r_{k}

, and then progresses to the next state

s_{k + 1}

, as indicated by the red dashed line

τ ({}, s_{0}, a_{0}, s_{1}, a_{1}, \dots, s_{N}, a_{N})

shown in Figure 2b 33…”

Section: Controller Design For Rths Based On Rlmentioning

confidence: 99%

See 1 more Smart Citation

Reinforcement learning control method for real‐time hybrid simulation based on deep deterministic policy gradient algorithm

Tang

et al. 2022

Structural Contr & Hlth

View full text Add to dashboard Cite

The tracking performance of an actuation transfer system in a real-time hybrid simulation (RTHS) frequently faces accuracy and robustness challenges under constraints and complicated environments with uncertainties. This study proposes a novel control approach based on the deep deterministic policy gradient algorithm in reinforcement learning (RL) combined with feedforward (FF) compensation, which emphasizes the implementation of shaking table control and substructure RTHS. The proposed method first describes the control plant within the RL environment. Then, the agent is trained offline to develop optimized control policies for interaction with the environment. A series of validation tests were conducted to assess the performance of the proposed method, starting with the dynamic testing of underwater shaking table control and then a virtual RTHS benchmark problem. For complex systems, such as controlling the underwater shaking table, the proposed algorithm, FF, and adaptive time series (ATS) compensation methods are compared under various water depths and motions. The results show better performance and wider broadband frequency applicability under different shaking table dynamic-coupling effects. Next, a controller based on the proposed method was designed by extending the virtual RTHS via the configuration of the control plant and substructure division, as provided in the RTHS benchmark problem. The proposed RL controller also improved the tracking accuracy and robustness of conventional FF compensators against unmodeled dynamics and perturbation uncertainties. This controller can be extended to further advanced control strategies as a component of model-based control methods. K E Y W O R D Sdeep deterministic policy gradient algorithm, hybrid control, real-time hybrid simulation, reinforcement learning, underwater shaking table Ning Li and Jichuan Tang have equal contributions and share first authorship.

show abstract

“…The dynamic interaction process can be defined by the Markov decision process (MDP) and divided into finite discrete time steps in the RL control framework 32 . At each time step

k

, the agent observes state

s_{k}

, takes action

a_{k}

, receives scalar reward

r_{k}

, and then progresses to the next state

s_{k + 1}

, as indicated by the red dashed line

τ ({}, s_{0}, a_{0}, s_{1}, a_{1}, \dots, s_{N}, a_{N})

shown in Figure 2b 33…”

Section: Controller Design For Rths Based On Rlmentioning

confidence: 99%

“…

{(|, \nabla_{θ^{μ}} J^{θ^{μ}} ≐ \frac{1}{N} true \sum_{i = k}^{N} \nabla_{a} Q ((), s_{k}, μ ((), s_{k}) false| θ^{Q}))}_{a = μ^{θ}} \nabla_{θ^{μ}} μ ((), s_{k} false| θ^{μ}),

where

\nabla

represents the gradient operation.

\nabla_{θ^{μ}} J^{θ^{μ}}

represents the sampled policy gradient, which is typically optimized using various optimizers based on gradient descent, such as the Adam method 32,39 …”

Section: Controller Design For Rths Based On Rlmentioning

confidence: 99%

Reinforcement learning control method for real‐time hybrid simulation based on deep deterministic policy gradient algorithm

Tang

et al. 2022

Structural Contr & Hlth

View full text Add to dashboard Cite

show abstract

“…At present, deep reinforcement learning has been widely applied in unmanned vehicle control, [ 20 ] robot path planning and control, [ 21 ] pursuit and avoidance of targets, [ 22 ] unmanned driving [ 23 , 24 ], and real-time strategy games [ 25 , 26 ]. However, most of the reinforcement learning algorithms used in air combat maneuvering decision making are discrete action space algorithms, which inevitably face the problems of rough flight paths and limited reachable domains.…”

Section: Introductionmentioning

confidence: 99%

Research on UCAV Maneuvering Decision Method Based on Heuristic Reinforcement Learning

Wang

Zhang

Rong

et al. 2022

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

With the rapid development of unmanned combat aerial vehicle (UCAV)-related technologies, UCAVs are playing an increasingly important role in military operations. It has become an inevitable trend in the development of future air combat battlefields that UCAVs complete air combat tasks independently to acquire air superiority. In this paper, the UCAV maneuver decision problem in continuous action space is studied based on the deep reinforcement learning strategy optimization method. The UCAV platform model of continuous action space was established. Focusing on the problem of insufficient exploration ability of Ornstein–Uhlenbeck (OU) exploration strategy in the deep deterministic policy gradient (DDPG) algorithm, a heuristic DDPG algorithm was proposed by introducing heuristic exploration strategy, and then a UCAV air combat maneuver decision method based on a heuristic DDPG algorithm is proposed. The superior performance of the algorithm is verified by comparison with different algorithms in the test environment, and the effectiveness of the decision method is verified by simulation of air combat tasks with different difficulty and attack modes.

show abstract

“…The image contains a variety of information, so the most basic link to restore the real 3D scene from the image taken by UAV is image matching. At present, image matching is currently used extensively in target tracking [1][2][3][4], 3D reconstruction [5][6][7][8], visual SLAM [9][10][11], UAV obstacle avoidance navigation [12,13], and land surveying and mapping [14]. However, one of the common situations in real life is in a low-light environment photographed by UAV.…”

Section: Introductionmentioning

confidence: 99%

Fast Matching Method of UAV Aerial Photography Enhanced Low Illumination Image

Liu

et al. 2022

International Journal of Aerospace Engineering

View full text Add to dashboard Cite

Aiming at the problems of insufficient image contrast in three-dimensional reconstruction of UAV in low illumination environment and the unstable iteration times of the RANSAC algorithm in the feature matching process, real-time matching method of UAV aerial image is proposed. First, a new image enhancement algorithm is applied to the image to enhance its quality and visibility. Second, the enhanced fast algorithm in ORB extracts the feature points from the preprocessed image, and cross-matching performs rough matching. Finally, the PROSAC algorithm solves the homography matrix by selecting the highest quality interior points from the extracted feature points. To improve the matching accuracy, some exterior points that do not conform to the geometric characteristics of the image are removed based on the homography matrix and the set mismatch threshold. The results show that the improved ORB algorithm is applied to the low illumination environment of UAV aerial photography, the image matching accuracy in 3D reconstruction is improved, and the correct matching rate tends to 97.24~99.39%. The relevant research findings and conclusions provide a fast and effective method for UAV image matching in different low illumination environments.

show abstract

Target tracking strategy using deep deterministic policy gradient

Cited by 33 publications

References 24 publications

Reinforcement learning control method for real‐time hybrid simulation based on deep deterministic policy gradient algorithm

Reinforcement learning control method for real‐time hybrid simulation based on deep deterministic policy gradient algorithm

Research on UCAV Maneuvering Decision Method Based on Heuristic Reinforcement Learning

Fast Matching Method of UAV Aerial Photography Enhanced Low Illumination Image

Contact Info

Product

Resources

About