Optimal Threshold-Based Distributed Control Policies for Persistent Monitoring on Graphs

Abstract

We consider the optimal multi-agent persistent monitoring problem defined by a team of cooperating agents visiting a set of nodes (targets) on a graph with the objective of minimizing a measure of overall node state uncertainty. The solution to this problem involves agent trajectories defined both by the sequence of nodes to be visited by each agent and the amount of time spent at each node. Since such optimal trajectories are generally intractable, we propose a class of distributed threshold-based parametric controllers through which agent transitions from one node to the next are controlled by threshold parameters on the node uncertainty states. The resulting behavior of the agent-target system can be described by a hybrid dynamic system. This enables the use of Infinitesimal Perturbation Analysis (IPA) to determine on line (locally) optimal threshold parameters through gradient descent methods and thus obtain optimal controllers within this family of threshold-based policies. We further show that in a single-agent case the IPA gradient is monotonic, which implies a simple structure whereby an agent visiting a node should reduce the uncertainty state to zero before moving to the next node. Simulation examples are included to illustrate our results and compare them to optimal solutions derived through dynamic programming when this is possible.

Event-Driven Receding Horizon Control of Energy-Aware Dynamic Agents for Distributed Persistent Monitoring

“…This problem setup is widely known as the persistent monitoring problem and it encompasses applications such as environmental sensing [1], surveillance [2], traffic monitoring [3], data collection [4], event detection [5] and energy management [6]. In order to suit different application scenarios, this persistent monitoring problem has been studied in the literature under different objective functions [7], agent dynamic models [8], [9] and target state dynamic models [10], [11].…”

“…A common way to categorize persistent monitoring problem setups is based on whether the shapes of trajectory segments (available for the agents to travel between targets) are predefined [4], [10] or not [11], [12]. In the latter case, the main challenge is to search for the optimal agent trajectory shapes.…”

“…segments are predefined, the challenge is to search for: 1) the optimal target visiting schedules of agents and 2) the optimal control laws to govern agents on corresponding trajectory segments -assuming an agent has to remain stationary on a target to monitor it. As introduced in [10] and illustrated in Fig. 1, this can be seen as a Persistent Monitoring on a Network (PMN) problem where targets and trajectory segments are modeled as nodes and edges of a network, respectively.…”

“…1, this can be seen as a Persistent Monitoring on a Network (PMN) problem where targets and trajectory segments are modeled as nodes and edges of a network, respectively. Such PMN problems are significantly more complicated than the NP-hard traveling salesman problems [13] and thus have inspired many different solution approaches [8], [10], [14]. The work in [14] proposes a centralized off-line greedy algorithm to determine the optimal target visiting schedules of agents (i.e., each agent's sequence of targets to visit and respective dwell-times to be spent at visited targets) in PMN problems.…”

“…The work in [14] proposes a centralized off-line greedy algorithm to determine the optimal target visiting schedules of agents (i.e., each agent's sequence of targets to visit and respective dwell-times to be spent at visited targets) in PMN problems. In contrast, for the same task, [10] proposes a gradient-based distributed on-line approach -which, however, requires a brief centralized off-line initialization stage to address non-convexities. An alternative approach is taken in the recent work [8] which exploits the event-driven nature of PMN systems to develop a distributed on-line solution based on event-driven Receding Horizon Control (RHC) [15].…”