Enhanced Adaptive Random Test Case Prioritization for Model-based Test Suites

Pospíšil, Tomáš; Sobotka, Jan; Novák, Jiří

doi:10.12700/aph.17.7.2020.7.7

Cited by 4 publications

(3 citation statements)

References 31 publications

(38 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Pospisil [32] conducted the TCP process by employing an enhanced variant of the Adaptive Random Prioritisation (ARP) method. In contrast to the traditional ARP, they substituted the original distance function with a multi-criteria decision-making approach.…”

Section: Test Case Prioritisationmentioning

confidence: 99%

OCL-Based Test Case Optimisation with Modified APFD Metric

Jin,

Lano

2024

Preprint

View full text Add to dashboard Cite

Testing is one of the most time-consuming and unpredictable processes within the software development life cycle. As a result, many Test Case Optimisation (TCO) techniques have been proposed to make this process more scalable. Object Constraint Language (OCL) was initially introduced as a constraint language to provide additional details to UML models. However, as OCL continues to evolve, an increasing number of systems are being expressed by this language. Despite this growth, a noticeable research gap exists for the testing of systems whose specifications are expressed in OCL. In our previous work, we verified the effectiveness and efficiency of performing the Test Case Prioritisation (TCP) process for these systems. In this study, we extend our previous work by integrating the Test Case Minimisation (TCM) process to determine whether TCM can also benefit the testing process under the context of OCL. The evaluation of TCO approaches often relies on well-established metrics such as the Average Percentage of Fault Detection (APFD). However, the suitability of APFD for Model-Based Testing (MBT) is not ideal. This paper addresses this limitation by proposing a modification to the APFD metric to enhance its viability for MBT scenarios. We conducted four case studies to evaluate the feasibility of integrating the TCM and TCP processes in our proposed approach. In these studies, we applied the multi-objective optimisation algorithm NSGA-II and the genetic algorithm independently to the TCM and TCP processes. The objective was to assess the effectiveness and efficiency of combining TCM and TCP in enhancing the testing phase. Through experimental analysis, the results highlight the benefits of integrating TCM and TCP in the context of OCL-based testing, providing valuable insights for practitioners and researchers aiming to optimise their testing efforts. Specifically, the main contributions of this work include: 1). We introduce the integration of the TCM process into the TCO process for systems expressed by OCL. This integration benefits the testing process further by reducing redundant test cases while ensuring sufficient coverage. 2). We comprehensively analyse the limitations associated with the commonly used metric, APFD, and then a modified version of the APFD metric has been proposed to overcome these weaknesses. 3). We systematically evaluate the effectiveness and efficiency of OCL-based TCO processes on four real-world case studies with different complexities.

show abstract

Section: Test Case Prioritisationmentioning

confidence: 99%

OCL-Based Test Case Optimisation with Modified APFD Metric

Jin,

Lano

2024

Preprint

View full text Add to dashboard Cite

show abstract

“…Fault detection has been improved in existing studies via the use of new and enhanced hybrid techniques for string distance equations. Recent work by Pospisil et al [26] aimed to enhance adaptive random TCP for model-based test suites using original technique for Jaccard, Manhattan distance and similarity functions. All of the examined systems achieved improved fault detection performance as a result of the proposed improvement.…”

Section: Related Workmentioning

confidence: 99%

“…Results of the study show that this approach was able to reduce test suite size. Compared to the works by Pospisil et al [26] and Kumar et al [9], the current study concentrated more on using a hybrid string distance method to determine the degree of dissimilarity and then locate the distance with the greatest similarity reading.…”

Section: Related Workmentioning

confidence: 99%

The Hybrid Jaro-Winkler and Manhattan Distance using Dissimilarity Measure for Test Case Prioritization Approach

Shareef,

Sulaiman,

Basari

2023

IJACSA

View full text Add to dashboard Cite

Software product line (SPL) is a concept that has revolutionized the software development industry. It refers to a set of related software products that are developed from a common set of core assets but can be customized to meet specific customer requirements. Integrating SPL techniques into test case prioritization (TCP) can greatly enhance its effectiveness. By considering variability across different products within an SPL, it becomes possible to prioritize test cases based on their relevance to specific product configurations. However, the concept itself still has certain issues, such as in finding the highest rate of early failure detection. Various solutions have been proposed to mitigate this problem, among them is to improve the calculation of string distance using hybrid technique to achieve a high degree for similarity. Dissimilarity-based Technique (DBP) is the basis for our ranking method. The objective is to identify further weaknesses in the product lines as well as the differences between the experiment and real-world applications. Our focus is to enhance hybrid techniques that produce the highest rate of early failure detection. In this paper, early fault detection is selected as the performance goal. In order to choose the optimal methods for DBP for TCP, a comparison between several string distance measures was conducted. This study proposed hybrid techniques that combined Jaro-Winkler and Manhattan string distance namely New Enhanced Hybrid Technique 1 (NEHT1), New Enhanced Hybrid Technique 2 (NEHT2) and New Enhanced Hybrid Technique 3 (NEHT3). The case study was generated using the PLEDGE tool based on a Feature Model (FM). Six test cases were used in the experiment. Result shows the effectiveness of the combination where it achieved higher degree of similarity for T1 vs. T4, T2 vs. T3, T2 vs. T6, and T3 vs. T6, as well as perfect degree of similarity for NEHT1 (100.00%). The result proves that the combination of both techniques improve SPL testing effectiveness compared to existing techniques.

show abstract

Object Constraint Language based test case optimization with modified Average Percentage of Fault Detection metric

Jin,

Lano

2024

J Software Evolu Process

View full text Add to dashboard Cite

Testing is one of the most time‐consuming and unpredictable processes within the software development life cycle. As a result, many test case optimization (TCO) techniques have been proposed to make this process more scalable. Object Constraint Language (OCL) was initially introduced as a constraint language to provide additional details to Unified Modeling Language models. However, as OCL continues to evolve, an increasing number of systems are being expressed by this language. Despite this growth, a noticeable research gap exists for the testing of systems whose specifications are expressed in OCL. In our previous work, we verified the effectiveness and efficiency of performing the test case prioritization (TCP) process for these systems. In this study, we extend our previous work by integrating the test case minimization (TCM) process to determine whether TCM can also benefit the testing process under the context of OCL. The evaluation of TCO approaches often relies on well‐established metrics such as the average percentage of fault detection (APFD). However, the suitability of APFD for model‐based testing (MBT) is not ideal. This paper addresses this limitation by proposing a modification to the APFD metric to enhance its viability for MBT scenarios. We conducted four case studies to evaluate the feasibility of integrating the TCM and TCP processes into our proposed approach. In these studies, we applied the multi‐objective optimization algorithm NSGA‐II and the genetic algorithm independently to the TCM and TCP processes. The objective was to assess the effectiveness and efficiency of combining TCM and TCP in enhancing the testing phase. Through experimental analysis, the results highlight the benefits of integrating TCM and TCP in the context of OCL‐based testing, providing valuable insights for practitioners and researchers aiming to optimize their testing efforts. Specifically, the main contributions of this work include the following: (1) we introduce the integration of the TCM process into the TCO process for systems expressed by OCL. This integration benefits the testing process further by reducing redundant test cases while ensuring sufficient coverage. (2) We comprehensively analyze the limitations associated with the commonly used metric, APFD, and then, a modified version of the APFD metric has been proposed to overcome these weaknesses. (3). We systematically evaluate the effectiveness and efficiency of OCL‐based TCO processes on four real‐world case studies with different complexities.

show abstract

Enhanced Adaptive Random Test Case Prioritization for Model-based Test Suites

Cited by 4 publications

References 31 publications

OCL-Based Test Case Optimisation with Modified APFD Metric

OCL-Based Test Case Optimisation with Modified APFD Metric

The Hybrid Jaro-Winkler and Manhattan Distance using Dissimilarity Measure for Test Case Prioritization Approach

Object Constraint Language based test case optimization with modified Average Percentage of Fault Detection metric

Contact Info

Product

Resources

About