Compositional abstractions of interconnected discrete-time stochastic control systems

Proceedings of the 21st International Conference on Hybrid Systems: Computation and Control (Part of CPS Week)

Zamani

2018

Self Cite

This paper is concerned with a compositional approach for constructing finite Markov decision processes of interconnected discrete-time stochastic control systems. The proposed approach leverages the interconnection topology and a notion of so-called stochastic storage functions describing joint dissipativitytype properties of subsystems and their abstractions. In the first part of the paper, we derive dissipativitytype compositional conditions for quantifying the error between the interconnection of stochastic control subsystems and that of their abstractions. In the second part of the paper, we propose an approach to construct finite Markov decision processes together with their corresponding stochastic storage functions for classes of discrete-time control systems satisfying some incremental passivablity property. Under this property, one can construct finite Markov decision processes by a suitable discretization of the input and state sets. Moreover, we show that for linear stochastic control systems, the aforementioned property can be readily checked by some matrix inequality. We apply our proposed results to the temperature regulation in a circular building by constructing compositionally a finite Markov decision process of a network containing 200 rooms in which the compositionality condition does not require any constraint on the number or gains of the subsystems. We employ the constructed finite Markov decision process as a substitute to synthesize policies regulating the temperature in each room for a bounded time horizon.

Section: Introductionmentioning

confidence: 99%

From Dissipativity Theory to Compositional Construction of Finite Markov Decision Processes

Proceedings of the 21st International Conference on Hybrid Systems: Computation and Control (Part of CPS Week)

Zamani

2018

Self Cite

“…The next theorem shows how SSF can be employed to compare output trajectories of two interconnected dt-SCS (without internal signals) in a probabilistic sense. This theorem is borrowed from [LSMZ17, Theorem 3.3], and holds for the setting here since the max form of SSF here implies the additive form used in [LSMZ17].…”

Section: Stochastic (Pseudo-)simulation Functionsmentioning

confidence: 99%

“…(2) in [LSMZ17]). In this case, function V becomes a nonnegative supermartingle if ρ ext (·) is also equal to zero.…”

Section: Stochastic (Pseudo-)simulation Functionsmentioning

confidence: 99%

“…Recently, compositional construction of finite abstractions is presented in [SAM15] and [LSZ18b] using dynamic Bayesian networks and small-gain type conditions, respectively. Compositional construction of infinite abstractions (reduced-order models) is proposed in [LSMZ17] and [LSZ18a] using small-gain type conditions and dissipativity-type properties of subsystems and their abstractions, respectively, both for discrete-time stochastic control systems. To the best of our knowledge, this is the first work that consolidate a compositional approach for constructing both infinite and finite abstractions of interconnected stochastic systems.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Compositional (In)Finite Abstractions for Large-Scale Interconnected Stochastic Systems

IEEE Trans. Automat. Contr.

2020

Self Cite

This paper is concerned with a compositional approach for constructing both infinite (reducedorder models) and finite abstractions (a.k.a. finite Markov decision processes) of large-scale interconnected discrete-time stochastic control systems. The proposed framework is based on the notion of stochastic simulation functions enabling us to use an abstract system as a substitution of the original one in the controller design process with guaranteed error bounds. In the first part of the paper, we derive sufficient small-gain type conditions for the compositional quantification of the probabilistic distance between the interconnection of stochastic control subsystems and that of their infinite abstractions. We then construct infinite abstractions together with their corresponding stochastic simulation functions for a class of discrete-time nonlinear stochastic control systems. In the second part of the paper, we leverage small-gain type conditions for the compositional construction of finite abstractions. We propose an approach to construct finite Markov decision processes (MDPs) of the concrete models (or their reduced-order versions) satisfying an incremental input-tostate stability property. We also show that for a particular class of nonlinear stochastic control systems, the aforementioned property can be readily checked by matrix inequalities. We demonstrate the effectiveness of the proposed results by applying our approaches to the temperature regulation in a circular building and constructing compositionally a finite abstraction of a network containing 1000 rooms. We employ the constructed finite abstractions as substitutes to compositionally synthesize policies regulating the temperature in each room for a bounded time horizon. We also apply our proposed techniques to a fully connected network of 20 nonlinear subsystems (totally 100 dimensions) and construct finite MDPs from their reduced-order versions (together 20 dimensions) with guaranteed error bounds on their output trajectories.

“…Recently, compositional construction of finite abstractions is presented in [SAM15,LSZ18b] using dynamic Bayesian networks and small-gain type conditions, respectively. Compositional construction of infinite abstractions (reduced-order models) is presented in [LSMZ17,LSZ18a] using small-gain type conditions and dissipativity-type properties of subsystems and their abstractions, respectively, both for discrete-time stochastic control systems. Although [LSMZ17,LSZ18a] deal only with infinite abstractions (reduced-order models), our proposed approach here considers finite Markov decision processes as abstractions which are the main tools for automated synthesis of controllers for complex logical properties.…”

Section: Introductionmentioning

confidence: 99%

Compositional abstraction of large-scale stochastic systems: A relaxed dissipativity approach

Nonlinear Analysis: Hybrid Systems

2020

Self Cite

This paper is concerned with a compositional approach for the construction of finite abstractions (a.k.a. finite Markov decision processes) for networks of discrete-time stochastic control systems that are not necessarily stabilizable. The proposed approach leverages the interconnection topology and finite-step stochastic storage functions, that describe joint dissipativity-type properties of subsystems and their abstractions, in order to establish a finite-step stochastic simulation function between the network and its abstraction. In comparison with the existing notions of simulation functions, a finite-step stochastic simulation function needs to decay only after some finite numbers of steps instead of at each time step. In the first part of the paper, we develop a new type of compositional conditions, which is less conservative than the existing ones, for quantifying the probabilistic error between the interconnection of stochastic control subsystems and that of their abstractions. In particular, using this relaxation via a finite-step stochastic simulation function, it is possible to construct finite abstractions such that stabilizability of each subsystem is not required. In the second part of the paper, we propose an approach to construct finite Markov decision processes (MDPs) together with their corresponding finite-step storage functions for general discrete-time stochastic control systems satisfying an incremental passivablity property. We show that for a particular class of stochastic control systems, the aforementioned property can be readily checked by matrix inequalities. We also construct finite MDPs together with their storage functions for a particular class of nonlinear stochastic control systems. To demonstrate the effectiveness of our proposed results, we first apply our approach to an interconnected system composed of 4 subsystems such that 2 of them are not stabilizable. We then consider a road traffic network in a circular cascade ring composed of 50 cells, each of which has the length of 500 meters with 1 entry and 1 way out, and construct compositionally a finite MDP of the network. We employ the constructed finite abstractions as substitutes to compositionally synthesize policies keeping the density of traffic lower than 20 vehicles per cell. Finally, we apply our proposed technique to a fully connected network of 500 nonlinear subsystems and construct their finite MDPs with guaranteed error bounds on their probabilistic output trajectories.