A historical perspective on runtime assertion checking in software development

Abstract

This report presents initial results in the area of software testing and analysis produced as part of the Software Engineering Impact Project. The report describes the historical development of runtime assertion checking, including a description of the origins of and significant features associated with assertion checking mechanisms, and initial findings about current industrial use. A future report will provide a more comprehensive assessment of development practice, for which we invite readers of this report to contribute information. IntroductionThe Software Engineering Impact Project is documenting the impact that software engineering research has had on computer science research and on software development practice. The authors of this paper are responsible for documenting the impact of research in software testing and analysis for the Impact Project. One aspect of testing and analysis that has clearly had an impact is the widespread use of assertions, particularly for use in automated runtime detection of faults. This report documents the results of our initial assessment of assertions and narrates the history of software engineering research as it relates to the evolution and maturation of runtime assertion checking capabilities in programming languages and software development support tools.Despite decades of research into powerful software engineering technologies, and despite the continual discovery of tenets of good software engineering practice, software development remains an exceedingly complex endeavor. No matter how thoroughly a software system's requirements are documented, and no matter how carefully and elegantly the system's design has been constructed, inevitably latent faults, or incorrect program statements, are introduced in the system's implementation. These faults may be revealed during various levels of testing, or they may make a more inopportune appearance during field use by endusers. In such situations the faults are typically manifested externally as program failures, such as unexpected outputs, or other undesirable outcomes such as a program crash. Such failures provide developers with precious little information for initiating the task of correlating the simple external evidence of failure with the complexity of searching numerous possible locations for the faults that caused them.Assertions are one of the most useful automated techniques available for detecting faults and providing information about their locations, even for faults that are traversed during execution but do not lead to failures.As described in this report, assertions have a long and distinguished history in the annals of software engineering and programming language design. Initially developed as a means of stating expected or desired program properties as a necessary step in constructing formal, deductive proofs of program correctness, assertions have found many other applications in software engineering over the years, albeit primarily in the later stages of development (particularly in the developmen...

Run-Time Assertion Checking of Data- and Protocol-Oriented Properties of Java Programs: An Industrial Case Study

“…Run-time assertion checking is one of the most useful techniques for detecting faults, and can be applied during any program execution context, including debugging, testing, and production [7]. Compared to program logics, run-time assertion checking emphasizes executable specifications.…”

Run-Time Verification of Black-Box Components Using Behavioral Specifications: An Experience Report on Tool Development

“…Run-time assertion checking is one of the most useful techniques for detecting faults, and can be applied during any program execution context, including debugging, testing, and production [3]. Compared to program logics, assertion checking emphasizes executable specifications.…”

“…In general, assertions expressed in languages supporting design by contract (like the Java Modeling Language (JML) [1]) cannot be used to specify the interaction protocol between objects or components, in contrast to other formalisms such as message sequence charts and UML sequence diagrams. Nor can state-based assertions be used to specify component interfaces since such interfaces do not have a state 3 . This paper reports on an integrated tool environment which provides a smooth integration of the specification and run-time checking of both data-and protocol-oriented properties of component interfaces.…”

Run-Time Verification of Coboxes

“…Run-time assertion checking is one of the most useful techniques for detecting faults, and can be applied during any program execution context, including debugging, testing, and production [4]. We provide a new compositional semantics for a specific class of coboxes.…”