Linking unit tests and properties

Gerdes, Alex; Hughes, John; Smallbone, Nick; Wang, Meng

doi:10.1145/2804295.2804298

Cited by 5 publications

(4 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As such, they can be impenetrable for stakeholders such as Volvo Cars. It can even be difficult to tell whether or not a particular test case can be generated (because API operations can have preconditions that might not be satisfiable) [9]. How can we know, then, whether the tests generated from a formal specification are testing the right thing?…”

Section: The Questionmentioning

confidence: 99%

Understanding formal specifications through good examples

Gerdes

Hughes

Smallbone

et al. 2018

Proceedings of the 17th ACM SIGPLAN International Workshop on Erlang

Self Cite

View full text Add to dashboard Cite

Formal specifications of software applications are hard to understand, even for domain experts. Because a formal specification is abstract, reading it does not immediately convey the expected behaviour of the software. Carefully chosen examples of the software's behaviour, on the other hand, are concrete and easy to understand-but poorly-chosen examples are more confusing than helpful. In order to understand formal specifications, software developers need good examples.We have created a method that automatically derives a suite of good examples from a formal specification. Each example is judged by our method to illustrate one feature of the specification. The generated examples give users a good understanding of the behaviour of the software. We evaluated our method by measuring how well students understood an API when given different sets of examples; the students given our examples showed significantly better understanding.CCS Concepts • Software and its engineering → Software verification and validation; Software defect analysis; Programming by example;

show abstract

Section: The Questionmentioning

confidence: 99%

Understanding formal specifications through good examples

Gerdes

Hughes

Smallbone

et al. 2018

Proceedings of the 17th ACM SIGPLAN International Workshop on Erlang

Self Cite

View full text Add to dashboard Cite

show abstract

“…We were asked "does your model cover this test case? ", so we developed tools to answer such questions-and when the AUTOSAR consortium released six "official" test cases for the CAN bus stack, we used them to show that three of these test cases were not covered by the model, because they were not consistent with the standard (Gerdes, Hughes, Smallbone, and Wang, 2015). We were asked: "which requirements have you tested?…”

Section: In Conclusionmentioning

confidence: 99%

Experiences with QuickCheck: Testing the Hard Stuff and Staying Sane

Hughes

2016

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

Abstract. This is not a typical scientific paper. It does not present a new method, with careful experiments to evaluate it, and detailed references to related work. Rather, it recounts some of my experiences over the last 15 years, working with QuickCheck, and its purpose is as much to entertain as to inform. QuickCheck is a random testing tool that Koen Claessen and I invented, which has since become the testing tool of choice in the Haskell community. In 2006 I co-founded Quviq, to develop and market an Erlang version, which we have since applied for a wide variety of customers, encountering many fascinating testing problems as a result. This paper introduces Quviq QuickCheck, and in particular the extensions made for testing stateful code, via a toy example in C. It goes on to describe the largest QuickCheck project to date, which developed acceptance tests for AUTOSAR C code on behalf of Volvo Cars. Finally it explains a race detection method that nailed a notorious bug plaguing Klarna, northern Europe's market leader in invoicing systems for e-commerce. Together, these examples give a reasonable overview of the way QuickCheck has been used in industrial practice.

show abstract

“…QuickCheck started as a testing framework for testing pure functional programs [7]. However, recent development in the area of property-based testing [9,13] incorporates the state-fulness of systems. That allowed for the testing of state-ful systems and even test programs written in imperative languages such as C. Hughes assert that testing state-ful systems is challenging.…”

Section: Property-based Testingmentioning

confidence: 99%