Recurrent Neural Network Controllers for Signal Temporal Logic Specifications Subject to Safety Constraints

Abstract

In this paper, we consider the problem of learning a neural network controller for a system required to satisfy a Signal Temporal Logic (STL) specification. We exploit STL quantitative semantics to define a notion of robust satisfaction. Guaranteeing the correctness of a neural network controller, i.e., ensuring the satisfaction of the specification by the controlled system, is a difficult problem that received a lot of attention recently. We provide a general procedure to construct a set of trainable High Order Control Barrier Functions (HOCBFs) enforcing the satisfaction of formulas in a fragment of STL. We use the BarrierNet, implemented by a differentiable Quadratic Program (dQP) with HOCBF constraints, as the last layer of the neural network controller, to guarantee the satisfaction of the STL formulas. We train the HOCBFs together with other neural network parameters to further improve the robustness of the controller. Simulation results demonstrate that our approach ensures satisfaction and outperforms existing algorithms.

Safe Model-based Control from Signal Temporal Logic Specifications Using Recurrent Neural Networks

“…Similarly to generating the initial dataset, each time after the control policy is improved, we sample N initial states, starting from which the safe control inputs in (12) are applied until arriving at the time horizon T . We add all the system transition data (totally N T data pairs) to the dataset D. Then the FNN is retrained on the new dataset D to minimize the loss function C in (7) (see Alg.…”

“…In this section, we evaluate our approach on two case studies. We compare the results with the approach in [12], where an RNN controller is trained via imitation learning, i.e., a dataset containing success trajectories is first generated and the controller is trained on that dataset. The system dynamics in [12] are assumed to be known.…”

“…The RNN was implemented using Pytorch [32]. We used a Mac with a 2.6GHz Core i7 CPU and 16GB of RAM for both the method developed in this paper and for the one from [12].…”

“…For instance, if a specification requires an agent to visit region A and then region B, the agent needs to know whether it has already visited region A in order to decide which region it should go at the current time. A Recurrent Neural Network (RNN) controller, which has memory, was proposed in [12] to satisfy STL specifications. The RNN was trained on a dataset containing trajectories generated from optimization methods, which is again computationally expensive.…”

Non-Parametric Neuro-Adaptive Control Subject to Task Specifications

“…Other learning-based related works include modeling with Gaussian processes [6,23,24,25], or use neural networks [26,27,28,29,30,31,32,33,30,34,35,36] to accomplish reachability, verification or temporal logic specifications. Nevertheless, the aforementioned works either use partial information on the underlying robot dynamics, or do not consider them at all.…”