Safe Markov Chains for ON/OFF Density Control With Observed Transitions

Abstract

This paper presents a convex optimization approach to control the density distribution of autonomous mobile agents with two control modes: ON and OFF. The main new characteristic distinguishing this model from standard Markov decision models is the existence of the ON control mode and its observed actions. When an agent is in the ON mode, it can measure the instantaneous outcome of one of the actions corresponding to the ON mode and decides whether it should take this action or not based on this new observation. If it does not take this action, it switches to the OFF mode where it transitions to the next state based on a predetermined set of transitional probabilities, without making any additional observations. In this decisionmaking model, each agent acts autonomously according to an ON/OFF decision policy, and the discrete probability distribution for the agent's state evolves according to a discrete-time Markov chain that is a linear function of the stochastic environment (i.e., transition probabilities) and the ON/OFF decision policy. The relevant policy synthesis is formulated as a convex optimization problem where hard safety and convergence constraints are imposed on the resulting Markov matrix. We first consider the case where the ON mode has a single action and the OFF mode has deterministic transitions (rather than stochastic) to demonstrate the model and the ideas behind our approach, which is then generalized to the case where ON mode has multiple actions and OFF mode has stochastic transitions.

Graph Temporal Logic Inference for Classification and Identification

“…The edge label y represents the Euclidean distance between the centroids of the subregions. The probabilistic densities of the robots in the subregions are governed by a time-varying Markov chain [21]. Figure 6: The swarm of 72 robots in the 9 sub-regions.…”

“…The edge label y represents the Euclidean distance between the centroids of the subregions. The probabilistic densities of the robots in the subregions are governed by a time-varying Markov chain [21]. We randomly generate graph-temporal trajectories and randomly choose 10 from them that satisfy the following constraint: whenever the probabilistic density of a subregion reaches above 1/8, then for the next 2 time units there always exists at least one neighbor subregion within distance of 1 with probabilistic density below 1/9.…”

Computing Probabilistic Controlled Invariant Sets

“…MDPs are widely used in applications such as motion planning [18]. In [19], [20], constraints are imposed on the state probability density function of an MDP under control. Probabilistic invariance, as developed in this paper, can be used for such control systems to characterize the invariant region of the state space.…”

Review of Multi-Agent Algorithms for Collective Behavior: a Structural Taxonomy

“…Applications: Both homogeneous [277]- [284] and inhomogeneous MC algorithms [276], [285]- [287] algorithms can be used for pattern formation, coverage, area exploration, and goal searching. Additional applications include multi-agent surveillance [288], coverage [289], [290], and task allocation [291], [292].…”