Test‐Case Design

Myers, Glenford J.; Badgett, Tom; Sandler, Corey

doi:10.1002/9781119202486.ch4

Cited by 3 publications

(12 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…2.1.2 Black or White Box Testing. The most important aspect of testing is the creation and design of effective test cases [51]. Testing a system 'completely' is impossible given a limited budget and limited amount of time.…”

Section: Testing Software Systemsmentioning

confidence: 99%

“…Therefore, a good testing strategy can help uncover the existence of bugs, but not prove that a program is entirely bug-free [50]. The question then becomes: łWhat subset of all possible test cases has the highest probability of detecting the most errors?ž [51] We can differentiate between two different perspectives when designing tests: black box and white box testing. Either test perspective can be used at any of the levels of abstraction discussed previously.…”

Section: Testing Software Systemsmentioning

confidence: 99%

“…Several strategies exist to determine how to achieve the highest possible confidence in the system with the least amount of test cases. A first important strategy to determine how many tests are required is 'equivalence class partitioning' [51]. In equivalence class partitioning, the idea is that a tester uses their knowledge of what input values can be considered realistic, and then divides the input space into classes of 'equivalent' values, i.e., groups of input values for which it is reasonable to assume the system will behave in the same way.…”

Section: Testing Software Systemsmentioning

confidence: 99%

“…However, we will show that this is not guaranteed. In software engineering, significant effort is spent designing test cases to validate that code is indeed reliable and free of errors [51]. Designing such test cases is considered a software engineering best practice, and is even included in the ACM/IEEE-CS Software Engineering Code of Ethics [24].…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

A Framework and Toolkit for Testing the Correctness of Recommendation Algorithms

Michiels¹,

Verachtert

Ferraro

et al. 2024

ACM Trans. Recomm. Syst.

View full text Add to dashboard Cite

Evaluating recommender systems adequately and thoroughly is an important task. Significant efforts are dedicated to proposing metrics, methods and protocols for doing so. However, there has been little discussion in the recommender systems’ literature on the topic of testing. In this work, we adopt and adapt concepts from the software testing domain, e.g., code coverage, metamorphic testing, or property-based testing, to help researchers to detect and correct faults in recommendation algorithms. We propose a test suite that can be used to validate the correctness of a recommendation algorithm, and thus identify and correct issues that can affect the performance and behavior of these algorithms. Our test suite contains both black box and white box tests at every level of abstraction, i.e., system, integration and unit. To facilitate adoption, we release RecPack Tests , an open-source Python package containing template test implementations. We use it to test four popular Python packages for recommender systems: RecPack , PyLensKit , Surprise and Cornac . Despite the high test coverage of each of these packages, we find that we are still able to uncover undocumented functional requirements and even some bugs. This validates our thesis that testing the correctness of recommendation algorithms can complement traditional methods for evaluating recommendation algorithms.

show abstract

Section: Testing Software Systemsmentioning

confidence: 99%

Section: Testing Software Systemsmentioning

confidence: 99%

Section: Testing Software Systemsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

A Framework and Toolkit for Testing the Correctness of Recommendation Algorithms

Michiels¹,

Verachtert

Ferraro

et al. 2024

ACM Trans. Recomm. Syst.

View full text Add to dashboard Cite

show abstract

“…However, as the functional criteria are based on the specification, they cannot ensure that critical and essential parts of the code have been covered. [3] Functional testing can also take into account non-functional requirements. Non-functional tests are the field of functional testing focused on evaluating non-functional aspects of the application, such as performance, security, stability, and usability.…”

Section: Introductionmentioning

confidence: 99%

Software testing with performance profiles clustering using unsupervised learning

Montes

Braga

Durães

et al. 2022

Preprint

View full text Add to dashboard Cite

The functional testing technique has been widely applied to reveal unknown faults in the software caused by programmer mistakes. Nevertheless, in autonomous data processing systems with highly variable inputs and outputs, such as embedded applications, data streams, and machine learning algorithms, the non-functional testing helps to reveal unknown-nontrivial faults in the deployment of software products. This paper addresses the detection of unknown code-faults by employing performance analysis as a nonfunctional requirement without predefined test cases, test oracles, or source-code analyses. The premise is that codefaults change demands for hardware resources during software execution, and a novel testing methodology can automatically detect them. This paper proposes the Tricorder testing methodology for automating workload characterization and detecting potential performance anomalies, caused by code-faults in autonomous data processing systems. Tricorder evaluates the performance profiles of hardware regarding the detection of the source code-faults. DAMICORE, a non-parametric multipurpose clustering methodology, enables Tricorder to group performance profiles of the software under testing and identify performance anomalies using non-parametric data in unsupervised learning based on Normalized Compression Distance (NCD). Tricorder reveals unknown source code faults, with no specialist to determine standards for input and output data, previous models inherent to architecture, test case creation, or workload characterization. We evaluate the capability of Tricorder in revealing faults through experiments based on three benchmarks: cryptography system, machine learning algorithm, and data stream processing server. Tricorder detects faults even under various workloads for different applications in our experiments. Tricorder helps the maintenance phase of the software development life cycle, providing additional information regarding the proper functioning of the application release before and after the updating process. This work contributes to the cost reduction of regression testing during the maintenance phase of autonomous data processing applications and can be used as a complementary testing technique.

show abstract

Using Mutation Analysis to Evolve Subdomains for Random Testing

Patrick

Alexander

Manuel

et al. 2013

2013 IEEE Sixth International Conference on Software Testing, Verification and Validation Workshops

View full text Add to dashboard Cite

Abstract-Random testing is inexpensive, but it can also be inefficient. We apply mutation analysis to evolve efficient subdomains for the input parameters of eight benchmark programs that are frequently used in testing research. The evolved subdomains can be used for program analysis and regression testing. Test suites generated from the optimised subdomains outperform those generated from random subdomains with 10, 100 and 1000 test cases for uniform, Gaussian and exponential sampling. Our subdomains kill a large proportion of mutants for most of the programs we tested with just 10 test cases.

show abstract

Test‐Case Design

Cited by 3 publications

References 0 publications

A Framework and Toolkit for Testing the Correctness of Recommendation Algorithms

A Framework and Toolkit for Testing the Correctness of Recommendation Algorithms

Software testing with performance profiles clustering using unsupervised learning

Using Mutation Analysis to Evolve Subdomains for Random Testing

Contact Info

Product

Resources

About