Construction of Reliable Software in Resource‐Constrained Environments

Abstract

Current software development practices are often "business-driven" and therefore tend to encourage testing approaches that reduce verification and validation activities to shorten schedules. In this paper, we focus on efficiency of testing and resulting software quality (reliability) in such an environment.We use a resource constrained testing model to analyze data from several projects. Our results indicate that in a practical resource constrained environment there is little room for improving testing during the actual process. Furthermore, there is likely to be considerable variability in the quality of testing, within a testing phase, that may stem from a variety of sources such as the human element, software development practices and the structure of the software itself. However, it is interesting to note that there are experimental resultsshowing that testing efficiency can be improved over longer periods of time through process improvement, and that software quality does appear to be positively correlated with an increase in the average testing efficiency that stems from process improvements. metrics and models can then be used to guide the process (Musa, et al (1987)). When software development and testing is resource and schedule constrained, some traditional quality monitoring metrics and models may become unusable. For example, during non-operational profile based testing, failure intensity decay may be an impractical guiding and decision tool (Rivers (1998), Rivers and Vouk (1999)).End-product reliability can be assessed in several ways. The aspect of reliability that we are concerned with is efficient reduction of as many defects as possible in the software being shipped. In the next section we describe and discuss the notion of resource constrained software development and testing.In Section 3, we present a software reliability model that formalizes resource constrained development and testing with respect to its outcome. Empirical results, from several sources, that illustrate different constrained software development issues are presented in Section 4. CONSTRAINED DEVELOPMENTSoftware development always goes through some basic groups of tasks related to specification, design, implementation, and testing. Of course, there are always additional activities, such as deployment and maintenance phase activities, as well as general management, and verification and validation activities that occur in parallel with the above activities. How much emphasis is put into each phase, and how the phases overlap, is a strong function of the software process adopted (or not) by the software manufacturer, and of how resource constraints may impact its modification and implementation (Pressman (2001), Beck (2000, Potok and Vouk (1997)). A typical resource constraint is development time. However, "time" (or exposure of the product to usage, in its many forms and shapes) is only one of the resources that may be in short supply when it comes to software development and testing. Other resources may include available staff,...

Requirements based test prioritization using risk factors: An industrial study

“…Definity and other industrial projects, many argue that creating and maintaining the operational profile is too resource intensive and seek non-operational profile driven techniques [18].…”

Towards the prioritization of system test cases

“…This paper uses a similar approach in test planning and considers customer‐assigned priority in the prioritization scheme where the priority of requirements are determined and applied to the TCP scheme. Despite the positive business results from applying SRET, some consider the creation and maintenance of the operational profile to be resource intensive and propose non‐operational techniques. PORT is a non‐operational profile system TCP scheme.…”

Ground Axioms to Achieve Movables: Methodology

“…But personnel resources are not unlimited, there is also some part of the software that may go untested, or may be verified to a lesser degree. The basic, and most difficult, aspect of system verification is to decide what must be tested, and what can be left untested, or partially tested [5]. The main part of agile methodology is that, it does not work on certain rules but it works on philosophies and principles.…”