On the correlation between code coverage and software reliability

Frate, F. Del; Garg, Pankaj; Mathur, Aditya P.; Pasquini, Alberto

doi:10.1109/issre.1995.497650

Cited by 42 publications

(30 citation statements)

References 4 publications

(2 reference statements)

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The ATC Simulation improves a typical safety analysis, which has to deal with the completeness problem [6]. The simulation makes possible to analyse dynamic states and then offer to a safety analysis group a powerful tool to determine the reliability of a computer based railway system.…”

Section: Resultsmentioning

confidence: 99%

Automatic train controller safety simulation

Gimenes¹,

Almeida²,

Nogueira³

2006

WIT Transactions on the Built Environment, Vol 88

View full text Add to dashboard Cite

Nowadays, we are observing an increased demand for better and safer mass transport systems. The supervision and control of these systems is made through an architecture known as Automatic Train Controller (ATC). The use of processors in ATC provides new challenges in a safety analysis. A typical railway system has a Speed-Distance Profile Generator that determines the maximum allowed speed in each track circuit. Dangerous situations are verified through the relative positioning and speeds between trains, switching machine positioning and other restrictions from operational commands generated by the ATC. Independent Safety Auditors should consider the failure modes of hardware and software in use. The safety analysis should consider alternative techniques to complete the complex task of evaluating how safe is the use of the processor and its associated software. Therefore, the use of simulation can improve and increase safety analysis, searching for fault states that could not be found in a static analysis. The main goal of this paper is to describe the development of a tool that simulates the behaviour of trains' movement in a subway system, with boolean expressions. The set of boolean expressions coordinates all the movements in a subway line and the simulation provides the possibility to find out lack of safety, considering different combinations in those boolean expressions. Another important goal is to simulate equipment faults in order to investigate problems not visible in a static analysis or even in a practical field test. Preliminary results have shown that the use of a simulator to execute boolean expressions offers a great variety of tests, allowing the detection of unsafe situations, complementing software tests validation in a final release. Through simulation, it is possible to observe the behaviour of simulated objects in specific internal points which improves the completeness in safety analysis.

show abstract

Section: Resultsmentioning

confidence: 99%

Automatic train controller safety simulation

Gimenes¹,

Almeida²,

Nogueira³

2006

WIT Transactions on the Built Environment, Vol 88

View full text Add to dashboard Cite

show abstract

“…The assumption behind the use of coverage criteria as a adequacy measurement is that a test suite can reveal a fault only if it executes the portion of code that contains the fault, and thus an higher level of coverage (which indicates that a bigger portion of the code is exercised by the test suite) should correlate with a higher number of revealed faults, although other factors may influence the actual outcome [16]- [19].…”

Section: Related Workmentioning

confidence: 99%

AURORA: AUtomatic RObustness coveRage Analysis Tool

Gargantini

Guarnieri

Magri

2013

2013 IEEE Sixth International Conference on Software Testing, Verification and Validation

View full text Add to dashboard Cite

Abstract-Code coverage is usually used as a measurement of testing quality and as adequacy criterion. Unfortunately, code coverage is very sensitive to modifications of the code structure, and, therefore, we can achieve the same degree of coverage with different testing effort by writing the same program in syntactically different ways. For this reason, code coverage can provide the tester with misleading information.In order to understand how a testing criterion is affected by code structure modifications, we have introduced a way to measure the sensitivity of coverage to code changes by means of code-to-code transformations. However the manual execution of the robustness analysis is tedious, time consuming and error prone. In order to solve these issues we present AURORA, a tool that automates the robustness analysis process and leverages the capabilities offered from several existing tools. AURORA has an extendible architecture that concretely supports the tester in the execution of the robustness analysis. Due to this extendible architecture, each user can personalize the robustness analysis to his/her needs. AURORA allows the user to add new transformations by using TXL, which is a programming language specifically designed to support source transformation tasks. It performs the coverage evaluation by using existing code coverage tools and is based on the use of the JUnit framework.

show abstract

“…Several studies have been conducted to investigate the relationship between code coverage and software reliability (Adams 1980;Ramsey and Basili 1985;Garg 1994;Veevers and Marshall 1994;Malaiya et al 1994;Jalote and Muralidhara 1994;Varadan 1995;Horgan et al 1995;Malaiya et al 1996;Chen et al 1996;Gokhale et al 1996;Krishnamurthy and Mathur 1996;Chen et al 1997;Chen et al 2001;Ye and Malaiya 2002;Malaiya et al 2002;Grottke 2002;Pham and Zhang 2003;Li et al 2004). An early experiment using control flow and data flow based testing was conducted by Frate et al (1995a). Their results provide evidence of a relationship between software reliability and coverage of elements required by the testing criteria.…”

Section: Motivationmentioning

confidence: 99%

A binomial software reliability model based on coverage of structural testing criteria

et al. 2007

View full text Add to dashboard Cite

A new approach to software reliability modeling is discussed where variables indirectly related with software reliability are used to provide additional information for the modeling process. Previous studies, empirical and theoretical evidences, and results from experiments indicate that there is a strong relationship between software reliability and coverage of program elements required to be exercised by structural testing criteria. This paper develops a binomial type coverage-based software reliability model through the definition of a coverage-based failure rate function. The Binomial software reliability Model Based on Coverage-BMBC-is proposed and discussed. In the BMBC test data between failures is used instead of time as independent variable; the model was assessed with test data from a real application, making use of the following structural testing criteria: all-nodes, all-edges, and potential-uses-a data-flow based family of testing criteria. The results from our experiments have shown that our modeling approach has some advantages over some traditional reliability models and points to a very promising research direction in software reliability.

show abstract

On the correlation between code coverage and software reliability

Cited by 42 publications

References 4 publications

Automatic train controller safety simulation

Automatic train controller safety simulation

AURORA: AUtomatic RObustness coveRage Analysis Tool

A binomial software reliability model based on coverage of structural testing criteria

Contact Info

Product

Resources

About