Little Tricky Logic: Misconceptions in the Understanding of LTL

Abstract

Context Linear Temporal Logic () has been used widely in verification. Its importance and popularity have only grown with the revival of temporal logic synthesis, and with new uses of in robotics and planning activities. All these uses demand that the user have a clear understanding of what an specification means.Inquiry Despite the growing use of , no studies have investigated the misconceptions users actually have in understanding formulas. This paper addresses the gap with a first study of misconceptions.Approach We study researchers' and learners' understanding of in four rounds (three written surveys, one talk-aloud) spread across a two-year timeframe. Concretely, we decompose "understanding " into three questions. A person reading a spec needs to understand what it is saying, so we study the mapping from to English. A person writing a spec needs to go in the other direction, so we study English to . However, misconceptions could arise from two sources: a misunderstanding of 's syntax or of its underlying semantics. Therefore, we also study the relationship between formulas and specific traces.Knowledge We find several misconceptions that have consequences for learners, tool builders, and designers of new property languages. These findings are already resulting in changes to the Alloy modeling language. We also find that the English to direction was the most common source of errors; unfortunately, this is the critical "authoring" direction in which a subtle mistake can lead to a faulty system. We contribute study instruments that are useful for training learners (whether academic or industrial) who are getting acquainted with , and we provide a code book to assist in the analysis of responses to similar-style questions.Grounding Our findings are grounded in the responses to our survey rounds. Round used Quizius to identify misconceptions among learners in a way that reduces the threat of expert blind spots. Rounds and confirm that both additional learners and researchers (who work in formal methods, robotics, and related fields) make similar errors. Round adds deep support for our misconceptions via talk-aloud surveys.Importance This work provides useful answers to two critical but unexplored questions: in what ways is tricky and what can be done about it? Our survey instruments can serve as a starting point for other studies.

Anti-Patterns (Smells) in Temporal Specifications

“…There are few works on the difficulties in understanding temporal specifications and on misconceptions involved in using them. Greenman et al [19] identified a set of misconceptions of LTL users. Other works [10], [11] empirically checked the comprehension of temporal specifications, in particular w.r.t.…”