Probabilistic Swarm Guidance Subject to Graph Temporal Logic Specifications

Djeumou, Franck; Xu, Zhe; Topcu, Ufuk

doi:10.15607/rss.2020.xvi.058

Cited by 17 publications

(12 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Control synthesis with temporal logic specifications. There are three main categories of approaches to designing controllers that meet temporal logic specifications [21][22][23][24][25][26][27][28][29][30][31]. The first category of approaches abstract the system as a transition system and transform the control syntheses problem into a series of constrained reachability problems [32][33][34].…”

Section: Related Workmentioning

confidence: 99%

Control strategies for COVID-19 epidemic with vaccination, shield immunity and quarantine: A metric temporal logic approach

Topcu

2021

PLoS ONE

Self Cite

View full text Add to dashboard Cite

Ever since the outbreak of the COVID-19 epidemic, various public health control strategies have been proposed and tested against the coronavirus SARS-CoV-2. We study three specific COVID-19 epidemic control models: the susceptible, exposed, infectious, recovered (SEIR) model with vaccination control; the SEIR model with shield immunity control; and the susceptible, un-quarantined infected, quarantined infected, confirmed infected (SUQC) model with quarantine control. We express the control requirement in metric temporal logic (MTL) formulas (a type of formal specification languages) which can specify the expected control outcomes such as “the deaths from the infection should never exceed one thousand per day within the next three months” or “the population immune from the disease should eventually exceed 200 thousand within the next 100 to 120 days”. We then develop methods for synthesizing control strategies with MTL specifications. To the best of our knowledge, this is the first paper to systematically synthesize control strategies based on the COVID-19 epidemic models with formal specifications. We provide simulation results in three different case studies: vaccination control for the COVID-19 epidemic with model parameters estimated from data in Lombardy, Italy; shield immunity control for the COVID-19 epidemic with model parameters estimated from data in Lombardy, Italy; and quarantine control for the COVID-19 epidemic with model parameters estimated from data in Wuhan, China. The results show that the proposed synthesis approach can generate control inputs such that the time-varying numbers of individuals in each category (e.g., infectious, immune) satisfy the MTL specifications. The results also show that early intervention is essential in mitigating the spread of COVID-19, and more control effort is needed for more stringent MTL specifications. For example, based on the model in Lombardy, Italy, achieving less than 100 deaths per day and 10000 total deaths within 100 days requires 441.7% more vaccination control effort than achieving less than 1000 deaths per day and 50000 total deaths within 100 days.

show abstract

Section: Related Workmentioning

confidence: 99%

Control strategies for COVID-19 epidemic with vaccination, shield immunity and quarantine: A metric temporal logic approach

Topcu

2021

PLoS ONE

Self Cite

View full text Add to dashboard Cite

show abstract

“…In this case, λ i,t (θ) = 0 and does not contribute to the sum in (10). Therefore, we may only focus on the case where q i,t ∈ O i (θ, θ * ) and thus λ i,t (θ) = 0.…”

Section: Synchronous Distributed Hypothesis Algorithmmentioning

confidence: 99%

“…This paper studies a problem in distributed teams of cooperating agents to perform tasks beyond the capability of an individual agent. Similar problems have attracted recent interest, see, e.g., [1], [2], [3], [4], [5], [6], [7], [8], [9], [10], [11]. As a running example, consider a team of mobile agents performing persistent surveillance tasks as shown in Fig.…”

Section: Introductionmentioning

confidence: 99%

Byzantine-Resilient Distributed Hypothesis Testing With Time-Varying Network Topology

Carr

Bharadwaj

et al. 2022

IEEE Trans. Automat. Contr.

Self Cite

View full text Add to dashboard Cite

show abstract

“…These works examine the conditions in which group tasks described by automata may be broken into sub-tasks executable by individuals. [8,6] provide methods to synthesize control policies for large-scale multi-agent systems with temporal logic specifications. However, all of these works assume a known model of the environment, differing from the learning setting of this paper.…”

Section: Related Workmentioning

confidence: 99%

Reward Machines for Cooperative Multi-Agent Reinforcement Learning

Neary,

Xu,

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

In cooperative multi-agent reinforcement learning, a collection of agents learns to interact in a shared environment to achieve a common goal. We propose the use of reward machines (RM) -Mealy machines used as structured representations of reward functions -to encode the team's task. The proposed novel interpretation of RMs in the multi-agent setting explicitly encodes required teammate interdependencies and independencies, allowing the team-level task to be decomposed into sub-tasks for individual agents. We define such a notion of RM decomposition and present algorithmically verifiable conditions guaranteeing that distributed completion of the sub-tasks leads to team behavior accomplishing the original task. This framework for task decomposition provides a natural approach to decentralized learning: agents may learn to accomplish their sub-tasks while observing only their local state and abstracted representations of their teammates. We accordingly propose a decentralized q-learning algorithm. Furthermore, in the case of undiscounted rewards, we use local value functions to derive lower and upper bounds for the global value function corresponding to the team task. Experimental results in three discrete settings exemplify the effectiveness of the proposed RM decomposition approach, which converges to a successful team policy two orders of magnitude faster than a centralized learner and significantly outperforms hierarchical and independent q-learning approaches.

show abstract

Probabilistic Swarm Guidance Subject to Graph Temporal Logic Specifications

Cited by 17 publications

References 0 publications

Control strategies for COVID-19 epidemic with vaccination, shield immunity and quarantine: A metric temporal logic approach

Control strategies for COVID-19 epidemic with vaccination, shield immunity and quarantine: A metric temporal logic approach

Byzantine-Resilient Distributed Hypothesis Testing With Time-Varying Network Topology

Reward Machines for Cooperative Multi-Agent Reinforcement Learning

Contact Info

Product

Resources

About