Analysing sanity of requirements for avionics systems

Abstract

In the last decade it became a common practice to formalise software requirements to improve the clarity of users' expectations. In this work we build on the fact that functional requirements can be expressed in temporal logic and we propose new sanity checking techniques that automatically detect flaws and suggest improvements of given requirements. Specifically, we describe and experimentally evaluate approaches to consistency and redundancy checking that identify all inconsistencies and pinpoint their exact source (the smallest inconsistent set). We further report on the experience obtained from employing the consistency and redundancy checking in an industrial environment. To complete the sanity checking we also describe a semi-automatic completeness evaluation that can assess the coverage of user requirements and suggest missing properties the user might have wanted to formulate. The usefulness of our completeness evaluation is demonstrated in a case study of an aeroplane control system. arXiv:1510.02669v1 [cs.SE] 9 Oct 2015 the system itself can be checked using model checking [4,12] or theorem proving [7] tools. If there are some requirements the system does not meet, the cause has to be found and the development reverted. The longer it takes to discover an error in the development, the more expensive the error is to mend. Consequently, errors made during requirements specification are among the most expensive ones in the whole development.Model checking is particularly efficient in finding bugs in the design, however, it exhibits some shortcomings when it is applied to requirement analysis. In particular, if the system satisfies the formula then the model checking procedure only provides an affirmative answer and does not elaborate for the reason of the satisfaction. It could be the case that, e.g. the specified formula is a tautology, hence, it is satisfied for any system. Tautologies can be obvious, such as p ∨ ¬p or subtle, such as p U ¬(r ∧ q),

Specification: The Biggest Bottleneck in Formal Methods and Autonomy

“…Coverage checking for LTL can be integrated into model checking [11]. Algorithms for automatically checking LTL coverage and completeness have been successfully used in industry for sanity checking, e.g., the requirements for an airplane control system [6].…”

TORUS: Tracing Complex Requirements for Large Cyber-Physical Systems

“…Traceability also helps to establish the reason why a particular software algorithm should exist in the product. However, traces alone do not provide ways of ascertaining completeness; the degree to which the requirements have been implemented in the system so far (Barnat et al, 2015). Estimating completeness and other metrics can be complex and time-consuming, yet they are crucial in order to successfully deliver a product that meets the expectations of all of its stakeholders (M. J.…”

Evaluation of Domain Agnostic Approaches for Enumeration of Minimal Unsatisfiable Subsets

“…Checking for satisfiability (also called consistency) means checking whether all the requirements can be implemented at once. If the set of requirements is unsatisfiable, the extraction of MUSes helps to identify and fix the conflicts among the requirements [5,9].…”

“…Further improvements to Hou's algorithm were presented later by Han and Lee [24] and by de la Banda et al [17]. A similar solution based on step-by-step traversal of the power set was proposed by Barnat et al [5]. However, the explicit exploration of the power set is the bottleneck for all of these algorithms since the power set is exponentially large w.r.t.…”

“…We selected the three constraint domains because several applications of MUSes arised in these domains recently [26,37,17,4,36,28,35,5,9,1] (below, we provide two example use cases). Also, the three domains vary in the complexity of the satisfiability problem going from relatively effectively solvable NP-complete problem in the SAT domain to PSPACE-complete problem in the LTL domain.…”