Policy Rollout Action Selection in Continuous Domains for Sensor Path Planning

Abstract

Policy rollout is a method for the online computation of future costs in approximate dynamic programming, and has been utilized for various problems including sensor management. In previous work, it has predominately been applied to the selection of actions from discrete sets. In this paper we present methods for action selection from continuous sets and analyze their trade-offs. The methods are evaluated on the problem of sensor path planning, with the intent of minimizing the time to localize an emitter using bearing measurements.

Multi-UAV Collaborative Trajectory Optimization for Asynchronous 3-D Passive Multitarget Tracking

“…Since the arrival time of measurements from different targets w.r.t. each sensor is random, existing researches [7], [8], [9], [10], [11], [12], [13], [14], [15], [16], [17], [18], [19], [20], [21], [22], [23], [24], [25], [26] cannot be applied to the 3-D CTO problem in the passive MTT context. We introduce the fusion time interval for time alignment and implement CTO in each fusion time instant.…”

“…To tackle this challenge, several TO models and corresponding solution algorithms are proposed for target localization/tracking in [7], [8], [9], [10], [11], [12], [13], [14], [15], [16], [17], [18], [19], [20], [21], [22], [23], [24], [25], [26]. Tzoreff and Weiss [7] discuss online and offline TO problems for a single UAV in the presence of time of arrival (TOA) measurements, subject to speed and no-fly zone constraints.…”

“…Tzoreff and Weiss [7] discuss online and offline TO problems for a single UAV in the presence of time of arrival (TOA) measurements, subject to speed and no-fly zone constraints. The proximal policy method and the policy rollout algorithm are applied to TO problems to speed up the localization of an emitter with angle of arrival (AOA) measurements in [8] and [9], respectively. These TO schemes for single-UAV [7], [8], [9] do not require consideration of collision avoidance and communication distance constraints, whereas these constraints should be considered in multi-UAV collaborative TO (CTO) problems [10], [11], [12], [13], [14].…”

“…The proximal policy method and the policy rollout algorithm are applied to TO problems to speed up the localization of an emitter with angle of arrival (AOA) measurements in [8] and [9], respectively. These TO schemes for single-UAV [7], [8], [9] do not require consideration of collision avoidance and communication distance constraints, whereas these constraints should be considered in multi-UAV collaborative TO (CTO) problems [10], [11], [12], [13], [14]. A receiver TO problem subjected to the minimal distance allowed to the emitter is considered in [10] to improve the localization performance, which is solved by the projected gradient (PG) method.…”

“…Most of the previous works [7], [8], [9], [10], [11], [12], [13], [14], [15], [16], [17], [18], [19], [20], [21], [22], [23], [24], [25] focus on 2-D TO. On the one hand, the 2-D TO ignores the possibility of exploiting the altitude dimension to improve tracking performance.…”