Transfer Entropy in MDPs with Temporal Logic Specifications

Abstract

Emerging applications in autonomy require control techniques that take into account uncertain environments, communication and sensing constraints, while satisfying highlevel mission specifications. Motivated by this need, we consider a class of Markov decision processes (MDPs), along with a transfer entropy cost function. In this context, we study highlevel mission specifications as co-safe linear temporal logic (LTL) formulae. We provide a method to synthesize a policy that minimizes the weighted sum of the transfer entropy and the probability of failure to satisfy the specification. We derive a set of coupled non-linear equations that an optimal policy must satisfy. We then use a modified Arimoto-Blahut algorithm to solve the non-linear equations. Finally, we demonstrated the proposed method on a navigation and path planning scenario of a Mars rover.

Collaborative Rover-copter Path Planning and Exploration with Temporal Logic Specifications Based on Bayesian Update Under Uncertain Environments

“…Several motion planning/exploration techniques employing the rover-copter collaboration have been provided, see, e.g., [4][5][6]. In particular, our approach is closely related to [4], in the sense that the overall synthesis problem is decomposed into two sub-problems, i.e., the problem of synthesizing a copter's exploration policy in the uncertain environment, and the problem of synthesizing the optimal policy for the rover so as to satisfy the scLTL formula.…”

“…Set 𝑘 ← 𝑘 + 𝑇 𝑐 and the copter transmits B to the rover; (2) Mission execution phase (see Section 4.3): The rover computes the optimal policy such that (5) is maximized and executes it for a given time period 𝑇 𝑟 . Moreover, update the environmental beliefs of the atomic propositions B as well as the mapping 𝑏 max , i.e., {𝑥 𝑟 , 𝑏 max , B} ← MissionExecution(𝑥 𝑟 , B,𝑇 𝑟 ).…”

“…If the trajectory of M 𝑆 reaches the accepting state in 𝑆 𝑓 in finite time, it means that the corresponding trajectory of A 𝜙 reaches the accepting state in 𝑄 𝑓 in finite time (i.e., it satisfies 𝜙). Hence, the problem of maximizing the belief for the satisfaction of 𝜙 defined by (5), can be reduced to the problem of maximizing the belief that the trajectory of M 𝑆 reaches 𝑆 𝑓 in finite time, i.e.,…”

“…In particular, to investigate Martian geology and habitability, NASA has decided to send copters to Mars in order to help the rover discover target samples in an efficient way [3]. Motivated by this fact, several motion planning techniques employing the rover-copter collaboration for the Mars exploration have been investigated in recent years, see, e.g., [4][5][6].…”