Evaluation of Software Product Line Test Case Prioritization Technique

Abstract

Software product line (SPL) engineering paradigm is commonly used to handle commonalities and variabilities of business applications to satisfy the specific needs or goal of a particular market. However, due to time and space complexities, testing all products is not feasible, and SPL testing is proven to be difficult due to a combinatorial explosion of the number of products to be considered. Combinatorial interaction testing (CIT) is suggested to reduce the size of test suites to overcome budget limitations and deadlines. CIT is conducted to fulfill certain quality attributes. This method can be further improvised through the prioritization of list configuration generated from CIT to gain better results in terms of efficiency and scalability, However, to the best of our knowledge, not much research has been done to evaluate existing Test Case Prioritization (TCP) techniques in SPL. This paper provides a survey of existing works on test case prioritization technique. This study provides classification and compares the best technique, trends, gaps and proposed frameworks based on the literature. The evaluation and discussion are using Normative Information Model-based Systems Analysis and Design (NIMSAD) on aspects that include context, content, and validation. The discussion highlights the lack of technique for scalability issue in SPL with most of the work is on academia setting but not on industrial practices.

Test Case Prioritization for Software Product Line: A Systematic Mapping Study

“…A survey explicitly done on SPL-TCP has been done by Sahak et al [11], Sanchez et al [12], and Kumar et al [13]. Sahak et al [11] evaluate the test case prioritization techniques by distributing and comparing the approach, trends, differences, and suggested frameworks based on the research article.…”

“…A survey explicitly done on SPL-TCP has been done by Sahak et al [11], Sanchez et al [12], and Kumar et al [13]. Sahak et al [11] evaluate the test case prioritization techniques by distributing and comparing the approach, trends, differences, and suggested frameworks based on the research article. Sanchez et al [12] explore the TCP technique by comparing proposed prioritization criteria with established common metrics of the feature model.…”

Target-based test path prioritization for UML activity diagram using weight assignment methods

“…Test case prioritization aims to cover any given part of the program by: detecting faults earlier, finding high priority errors, focusing on testing of the last modified part, and reducing the time and cost of testing. Several studies have examined methods to prioritize test paths to increase the efficiency and effectiveness of testing [1,26].…”

Test Case Minimization Applying Firefly Algorithm

“…Therefore, numerous studies have been conducted to minimize the test sequences or test cases [27]. These methods are also used in test case prioritization techniques in software product lines [25] and also in other fields such as in monitoring water status of plants [26].…”