Distributed adaptive-neighborhood control for stochastic reachability in multi-agent systems

Lukina, Anna; Tiwari, Ashish; Smolka, Scott A.; Grosu, Radu

doi:10.1145/3297280.3297370

Cited by 4 publications

(8 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We next present DAMPC [LTSG19], a distributed, adaptive-horizon and adaptiveneighborhood algorithm for solving the stochastic reachability problem in multi-agent systems; specifically the flocking problem modeled as an MDP. In DAMPC, at each time step, every agent first calls a centralized, adaptive-horizon model-predictive control (AMPC) algorithm to obtain an optimal solution for its local neighborhood.…”

Section: Introductionmentioning

confidence: 99%

V-Formation via Model Predictive Control

Grosu,

Lukina,

Smolka

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

We present recent results that demonstrate the power of viewing the problem of V-formation in a flock of birds as one of Model Predictive Control (MPC). The Vformation-MPC marriage can be understood in terms of the problem of synthesizing an optimal plan for a continuous-space and continuous-time Markov decision process (MDP), where the goal is to reach a target state that minimizes a given cost function.The first result we consider is ARES, an efficient approximation algorithm for generating optimal plans (action sequences) that take an initial state of an MDP to a state whose cost is below a specified (convergence) threshold. ARES uses Particle Swarm Optimization, with adaptive sizing for both the receding horizon and the particle swarm. Inspired by Importance Splitting, the length of the horizon and the number of particles are chosen such that at least one particle reaches a next-level state, i.e., a state where the cost decreases by a required delta from the previous-level state. The level relation on states and the plans constructed by ARES implicitly define a Lyapunov function and an optimal policy, respectively, both of which could be explicitly generated by applying ARES to all states of the MDP, up to some topological equivalence relation. We assess the effectiveness of ARES by statistically evaluating its rate of success in generating optimal plans for V-formation.ARES can alternatively be viewed as a model-predictive control (MPC) algorithm that utilizes an adaptive receding horizon, a technique we refer to as Adaptive MPC (AMPC). We next present Distributed AMPC (DAMPC), a distributed version of AMPC that works with local neighborhoods. We introduce adaptive neighborhood resizing, whereby the neighborhood size is determined by the cost-based Lyapunov function evaluated over a global system state. Our approach applies to reachability problems for any collection of entities that seek convergence from an arbitrary initial state to a desired goal state, where a notion of distance to the goal state(s) can be suitably defined. Our experimental evaluation shows that DAMPC can perform almost as well as centralized AMPC, while using only local information and a form of distributed consensus in each time step.Finally, inspired by security attacks on cyber-physical systems (CPS), we introduce controller-attacker games, where two players, a controller and an attacker, have antagonistic objectives. To highlight the power of adaptation, we formulate a special case of controllerattacker games called V-formation games, where the attacker's goal is to prevent the controller from attaining V-formation. We demonstrate how adaptation in the design of the controller helps in overcoming certain attacks.

show abstract

Section: Introductionmentioning

confidence: 99%

V-Formation via Model Predictive Control

Grosu,

Lukina,

Smolka

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Section VI discusses various approaches that have been proposed to solve this problem. In particular, there exist centralized [2] and (partially) distributed [3] solutions for achieving V-formation using MPC. None of these approaches, however, lead to a truly distributed solution for V-formation, i.e., without any form of consensus or information-sharing among non-neighbors.…”

Section: Introductionmentioning

confidence: 99%

“…Specifically, the distributed solutions in prior work have three shortcomings. First, the distributed controller [3] uses a consensus round at the beginning of every time step, so that all agents agree on a consistent set of actions. This augmented controller performs tasks similar to leader election in the process.…”

Section: Introductionmentioning

confidence: 99%

“…In [3], a distributed version of MPC is used to solve the V-formation problem, albeit with a reliance on a distributed consensus algorithm. It deploys adaptive neighborhood resizing and an adaptive-horizon version of MPC to determine the optimal action (acceleration) for every agent at every time-step.…”

Section: Introductionmentioning

confidence: 99%

“…It deploys adaptive neighborhood resizing and an adaptive-horizon version of MPC to determine the optimal action (acceleration) for every agent at every time-step. Our comparative performance evaluation considers a modified version of this controller: Distributed Adaptive-Horizon Model Predictive Control (DAMPC), which uses the adaptive-horizon feature of [3], but eschews any form of consensus. This is to ensure a fair comparison with our neural controller, which is also "consensus free".…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Learning Distributed Controllers for V-Formation

Roy¹,

Mehmood²,

Grosu³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

We show how a high-performing, fully distributed and symmetric neural V-formation controller can be synthesized from a Centralized MPC (Model Predictive Control) controller using Deep Learning. This result is significant as we also establish that under very reasonable conditions, it is impossible to achieve V-formation using a deterministic, distributed, and symmetric controller. The learning process we use for the neural V-formation controller is significantly enhanced by CEGkR, a Counterexample-Guided k-fold Retraining technique we introduce, which extends prior work in this direction in important ways. Our experimental results show that our neural V-formation controller generalizes to a significantly larger number of agents than for which it was trained (from 7 to 15), and exhibits substantial speedup over the MPC-based controller. We use a form of statistical model checking to compute confidence intervals for our neural Vformation controller's convergence rate and time to convergence.

show abstract