Training Agents to Satisfy Timed and Untimed Signal Temporal Logic Specifications with Reinforcement Learning

Hamilton, Nathaniel; Robinette, Preston K.; Johnson, Taylor T.

doi:10.1007/978-3-031-17108-6_12

Cited by 4 publications

(1 citation statement)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For example, neural network-based methods are investigated in Liu et al (2022); Leung and Pavone (2022);Hashimoto et al (2022). By generating rewards in proper ways from STL specifications, reinforcement learning-based approaches have been proposed in Venkataraman et al (2020); Kapoor et al (2020); Singh and Saha (2023); Hamilton et al (2022).…”

Section: 224mentioning

confidence: 99%

Online control synthesis for uncertain systems under signal temporal logic specifications

Yu,

Gao,

Jiang

et al. 2023

The International Journal of Robotics Research

View full text Add to dashboard Cite

Signal temporal logic (STL) formulas have been widely used as a formal language to express complex robotic specifications, thanks to their rich expressiveness and explicit time semantics. Existing approaches for STL control synthesis suffer from limited scalability with respect to the task complexity and lack of robustness against the uncertainty, for example, external disturbances. In this paper, we study the online control synthesis problem for uncertain discrete-time systems subject to STL specifications. Different from existing techniques, we propose an approach based on STL, reachability analysis, and temporal logic trees. First, based on a real-time version of STL semantics, we develop the notion of tube-based temporal logic tree (tTLT) and its recursive (offline) construction algorithm. We show that the tTLT is an under-approximation of the STL formula, in the sense that a trajectory satisfying a tTLT also satisfies the corresponding STL formula. Then, an online control synthesis algorithm is designed using the constructed tTLT. It is shown that when the STL formula is robustly satisfiable and the initial state of the system belongs to the initial root node of the tTLT, it is guaranteed that the trajectory generated by the control synthesis algorithm satisfies the STL formula. We validate the effectiveness of the proposed approach by several simulation examples and further demonstrate its practical usability on a hardware experiment. These results show that our approach is able to handle complex STL formulas with long horizons and ensure the robustness against the disturbances, which is beyond the scope of the state-of-the-art STL control synthesis approaches.

show abstract