Reactive Control Improvisation

Abstract

Reactive synthesis is a paradigm for automatically building correctby-construction systems that interact with an unknown or adversarial environment. We study how to do reactive synthesis when part of the specification of the system is that its behavior should be random. Randomness can be useful, for example, in a network protocol fuzz tester whose output should be varied, or a planner for a surveillance robot whose route should be unpredictable. However, existing reactive synthesis techniques do not provide a way to ensure random behavior while maintaining functional correctness. Towards this end, we generalize the recently-proposed framework of control improvisation (CI) to add reactivity. The resulting framework of reactive control improvisation provides a natural way to integrate a randomness requirement with the usual functional specifications of reactive synthesis over a finite window. We theoretically characterize when such problems are realizable, and give a general method for solving them. For specifications given by reachability or safety games or by deterministic finite automata, our method yields a polynomial-time synthesis algorithm. For various other types of specifications including temporal logic formulas, we obtain a polynomial-space algorithm and prove matching PSPACE-hardness results. We show that all of these randomized variants of reactive synthesis are no harder in a complexity-theoretic sense than their non-randomized counterparts. arXiv:1804.05037v2 [cs.LO] 19 Apr 2018This is perhaps disappointing, but is an inevitable consequence of LTL subsuming Boolean formulas. On the other hand, our general polynomial-space scheme applies to LTL and its much more expressive generalization LDL:Theorem 7.2. RCI (LDL, LDL) has a polynomial-space improvisation scheme.Proof. This follows from Theorem 5.2, since satisfaction of an LDL formula by a finite word can be checked in polynomial time (e.g. by combining dynamic programming on subformulas with a regular expression parser).Thus for temporal logics polynomial-time algorithms are unlikely, but adding randomization to reactive synthesis does not increase its complexity.The same is true for NFA and CFG specifications, where it is again PSPACE-hard to find even a single winning strategy: Theorem 7.3. Finite-window reactive synthesis for NFAs is PSPACE-hard.Proof (sketch). Reduce from QBF as in Theorem 7.1, constructing an NFA accepting the satisfying assignments of χ (as done in [12]).

Control Improvisation with Probabilistic Temporal Specifications

“…From streams of time-stamped system events, we learn models that are assumed to vary as a function of an underlying state space governed by events with durations. We leverage the recently-proposed technique of control improvisation [5], [21] to generate randomized control sequences, which are similar to an observed set of behaviors, and moreover, always satisfy some desired hard constraints and mostly satisfy soft constraints, while exhibiting variability. We presented an implementation of the end-to-end control improvisation workflow using the PRISM tool to enforce soft constraints on the improviser.…”

“…In this paper, we show how a recently-proposed formalism termed control improvisation [5] can be suitably adapted to address the problem of randomized control for IoT systems. We consider the specific setting of a system whose components can be controlled either by humans or automatically.…”

“…So for any ρ ∈ (0, 1], we can satisfy the randomness requirement by generating sufficiently long strings. We can efficiently detect when the EDHMM is not ergodic using standard graph 1 The definition of the CI problem given in [5] requires that I be described by a finite automaton. It is straightforward to build a nondeterministic finite automaton that accepts precisely those strings assigned nonzero probability by the EDHMM, but we will not describe the construction here since it is not needed for the technique used in this paper.…”

Risk-aware motion planning for automated vehicle among human-driven cars

“…Therefore, instead of using deterministic human models that follow optimal policies under given cost functions, we may consider probabilistic models that are able to regenerate diverse human behaviors and reflect human preferences through underlying probability distributions. To achieve this, we choose the control improvisation technique [7]- [9] that allows us to explicitly formulate and validate such probabilistic specifications.…”

Security-Aware Synthesis Using Delayed-Action Games

“…Stochastic multiplayer games (SMGs) are used to model reactive systems where nondeterministic decisions are made by multiple players [4,13,23]. SMGs extend probabilistic automata by assigning a player to each choice to be made in the game.…”