Approximate dynamic programming with state aggregation applied to UAV perimeter patrol

Abstract

SUMMARYOne encounters the curse of dimensionality in the application of dynamic programming to determine optimal policies for large-scale controlled Markov chains. In this paper, we provide a reward-based aggregation method to construct suboptimal policies for a perimeter surveillance control problem which gives rise to a large scale Markov chain. The novelty of this approach lies in circumventing the need for value iteration over the entire state space. Instead, the state space is partitioned and the value function is approximated by a constant over each partition. We associate a meta-state with each partition, where the transition probabilities between these meta-states are known. The state aggregation approach results in a significant reduction in the computational burden and lends itself to value iteration over the aggregated state-space. We provide bounds to assess the quality of the approximation and give numerical results that support the proposed methodology. Published 2011.

Optimal perimeter patrol via max-plus probability

“…Note that the alarm state is zeroed out upon a loiter by one of the UAVs. In addition the model of [4] keeps up with the length of time each alert waits to be serviced. We denote by T i (t) the number of time steps an alert at station i has been waiting to be serviced by a UAV.…”

“…To illustrate, we consider the example computed and illustrated in [4]'s paper, with 1 UAV, 4 UGSs, and a perimeter discretized with N=15 locations. For the original information gain/waiting time penalty formulation, we take =0.9, =0.0013.…”

“…The optimization posed in [4] takes a quality of service approach. The cost function implemented therein balances information gained by loitering against the waiting times of alerts.…”

The Application Research of Oil Pipeline Patrol by UAV in PCOC

“…The distance between the single well in the changqing oil field areas is very long, and the environment for esert and animal husbandry is very complex [4,5,6], and the complex terrain of inspection line, which cause the way of manual inspection is inefficient by walking. Labor patrol also has the risk as personnel injury [7].…”

The Research of Oil Pipeline Patrol by UAV in the First Sub-Factory of PCOC

“…The first sub-factory of PCOC is located in the Yishan Slope of the Ordos Basin in Yan'an, all of the region is hilly gully region [7,8,9], most of the oil pipeline is laying along the topography, which is difficult for labor patrol. So you would like to explore drones in the pipeline patrolling, alleviate patrol staff workload.…”