TAOS: Testing with Analysis and Oracle Support

Abstract

Few would question that software testing is a necessary activity for assuring software quality, yet the typical testing process is a human intensive activity and as such, it is unproductive, error-prone, and often inadequately done. Moreover, testing is seldom given a prominent place in software development or maintenance processes, nor is it an integral part of them. Major productivity and quality enhancements can be achieved by automating the testing process through tool development and use and effectively incorporating it with development and maintenance processes. The TAOS toolkit, Testing with Analysis and Oracle Support, provides support for the testing process. It includes tools that automate many tasks in the testing process, including management and persistence of test artifacts and the relationships between those artifacts, test development, test execution, and test measurement. A unique aspect of TAOS is its support for test oracles and their use to verify behavioral correctness of test executions, TAOS also supports structural/dependence coverage, by measuring the adequacy of test criteria coverage, and regression testing, by identifying tests associated or dependent upon modified software artifacts. This is accomplished by integrating the ProDAG toolset, Program Dependence Analysis Graph, with TAOS, which supports the use of program dependence analysis in testing, debugging, and maintenance. This paper describes the TAOS toolkit and its capabfities as well as testing, debugging and maintenance processes based on program dependence analysis.We also describe our experience with the toolkit and discuss our future plans.

Designing and comparing automated test oracles for GUI-based software applications

“…Notable is the work by Richardson in TAOS (Testing with Analysis and Oracle Support) [43] who proposes several levels of test oracle support. One level of test oracle support is given by the Range-checker which checks for ranges of values of variables during test-case execution.…”

“…Methods to specify oracles: As noted by several other researchers, software systems rarely have an automated oracle [9,40,43,44]. In most cases, the expected behavior of the software is assumed to be provided by the test designer.…”

“…In most cases, the expected behavior of the software is assumed to be provided by the test designer. This behavior may be specified in several ways: (1) as predicates represented in a tabular form to express functional specifications of the software [40], (2) as temporal constraints that specify conditions that must not be violated during software execution [8,9,43,44], (3) as logical expressions to be satisfied by the software [11], and (4) as formal specifications [1,4,6,10,14,15,41], e.g., algebraic specifications to represent abstract data types (ADTs) [4,10,15]. This expected behavior is then used by a verifier that either performs a table lookup [40], creates a state-machine model [9,20], or evaluates a boolean expression [11] to determine the correctness of the actual output.…”

Research in Advanced Environments

“…It accomplishes these goals by providing developers and testers with formal, automated support for: test artifact development and maintenance; parallel, monitored test execution; formal behavior verification via automated test oracles; test adequacy criteria definition and measurement; and interactive and programmatic test process management. TAOS is further described in [Ric93].…”

Automatic Generation of Object-Oriented Type Checkers

“…However, type systems have also been applied in multiple different scenarios, such as reverse engineering [8], testing [9], web metadata [10], and even security [11].…”