Design tests: An approach to programmatically check your code against design rules

Abstract

Assuring that a program conforms to its specification is a key concern in software quality assurance. Although there is substantial tool support to check whether an implementation complies to its functional requirements, checking whether it conforms to its design remains as an almost completely manual activity. In this paper, we present the concept of design tests, which are test-like programs that automatically check whether an implementation conforms to a specific design rule. Design rules are implemented directly in the target programming language in the form of tests. As a proof of concept, we present DesignWizard, an API developed to support design tests for Java programs as JUnit test cases. We applied design tests in two case studies and observed that our approach is suitable to check conformance automatically. Moreover, we observed that designers and programmers appreciate design tests as an executable documentation that can be easily kept up to date. Overview and MotivationChecking conformance between implementation and design rules is an important activity to guarantee quality on source code. One process widely used by software companies is Design Review [9], which consists of a mechanism to find errors in the design and its representation by splitting the team into groups and making them analyze each others' code. The problem with Design Review, being a manual process, that can lead the team to errors during the analysis. Moreover, this process does not scale, since analyzing a number of classes may take several hours.There have been several attempts to check conformance between code and design rules automatically. ArchJava [1], for example, proposes an extension of Java programming language that aims at ensuring conformance between implementation and architectural constraints specified using the concepts of ports and components introduced by the authors.Fairbanks et al. [4] use the term design fragment to refer to patterns that describe how a program interacts with frameworks. Design fragments describe what programmers should build to accomplish some goals and which are the relevant parts of the framework that the programmer's code will interact. The authors have created a catalog of design fragments and also mechanisms to implement (using XML) design fragments to check conformance between application and those design fragments.Another related work [8] describes a software reflexion model technique to help engineers to perform various software engineering tasks by exploiting the drift between design and implementation. This approach is interactive and the user has to specify the expected model, extract the real model from the source code using static analysis and then, describe a mapping between the extracted source model and the stated high-level structural model.Despite the existence of several academic approaches to check conformance between code and design rules, the gap between the state of the art and the state of the practice has become most apparent. We believe one of the factors responsible f...

Unveiling Exception Handling Bug Hazards in Android Based on GitHub and Google Code Issues

“…Error, checked Exception or RuntimeException). The module responsible for this analysis uses the Design Wizard framework [16] to inspect the bytecode of the exceptions reported on stack traces. It walks up the type hierarchy of a Java exception until it reaches a base exception type.…”

Structural conformance checking with design tests: An evaluation of usability and calability

“…In a previous work [10], we introduced a preliminary idea to check conformance through tests -the concept of design tests. Design tests are an application of the concept of tests to design conformance checking.…”

DCL 2.0: modular and reusable specification of architectural constraints

“…Design test is a technique to verify the architectural conformance by using automated tests similar to the ones done by unit testing [24]. This technique also suffers from a low abstraction level, which hinders the architectural understanding.…”

“…Design tests [24] employ the concept of unit testing applied to architectural compliance. Since it is based on unit testing, this technique is easy to learn for users who are familiar with testing frameworks.…”