Constraint handling in combinatorial test generation using forbidden tuples

Yu, Linbin; Duan, Feng; Lei, Yu; Kacker, Raghu N.; Kuhn, D. Richard

doi:10.1109/icstw.2015.7107441

Cited by 37 publications

(23 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This is explained by the fact that our base algorithm always has to compute the best parameters and values to fix, although this effort often results in a smaller test suite. Hence, we could only observe in some industry examples that the minimum forbidden tuple computation of Yu et al [38] may become a significant overhead. We leave it for future work to implement our constraint handling approach in the IPOG algorithm to fairly compare our constraint handling and the minimum forbidden tuple approach.…”

Section: Resultsmentioning

confidence: 95%

“…There is substantial work on combinatorial testing taking constraints and forbidden tuples into account, including meta-heuristic approaches [10,17,20,27], SAT-based approaches [29,36], and greedy approaches, which is further categorized into one-test-at-a-time (OTAT) approaches [9,11,10] and in-parameter-order generalized (IPOG) approaches [37,38].…”

Section: Mac Does Not Support Amd Cpusmentioning

confidence: 99%

“…In more recent work, Yu et al [38] significantly improved the efficiency of the approach, by developing a dedicated analysis of minimal forbidden tuples.…”

Section: Mac Does Not Support Amd Cpusmentioning

confidence: 99%

“…Yu et al [38] improved the constraint handling of the IPOG algorithm using the notion of minimum forbidden tuples (MFTs), i. e., the forbidden tuples whose proper subsets are not forbidden. Their first algorithm precomputes all MFTs, and uses this information during the test case generation.…”

Section: Actsmentioning

confidence: 99%

“…Their first algorithm precomputes all MFTs, and uses this information during the test case generation. Moreover, to relax the cost of computing all MFTs, which is significant if the constraint is complex [38], they further introduced an algorithm using necessary forbidden tuples (NFTs), that computes forbidden tuples when it becomes necessary.…”

Section: Actsmentioning

confidence: 99%

See 4 more Smart Citations

Greedy combinatorial test case generation using unsatisfiable cores

Yamada

Biere

Artho

et al. 2016

Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering

View full text Add to dashboard Cite

Combinatorial testing aims at covering the interactions of parameters in a system under test, while some combinations may be forbidden by given constraints (forbidden tuples).In this paper, we illustrate that such forbidden tuples correspond to unsatisfiable cores, a widely understood notion in the SAT solving community. Based on this observation, we propose a technique to detect forbidden tuples lazily during a greedy test case generation, which significantly reduces the number of required SAT solving calls. We further reduce the amount of time spent in SAT solving by essentially ignoring constraints while constructing each test case, but then "amending" it to obtain a test case that satisfies the constraints, again using unsatisfiable cores. Finally, to complement a disturbance due to ignoring constraints, we implement an efficient approximative SAT checking function in the SAT solver Lingeling.Through experiments we verify that our approach significantly improves the efficiency of constraint handling in our greedy combinatorial testing algorithm.

show abstract

Section: Resultsmentioning

confidence: 95%

Section: Mac Does Not Support Amd Cpusmentioning

confidence: 99%

“…In more recent work, Yu et al [38] significantly improved the efficiency of the approach, by developing a dedicated analysis of minimal forbidden tuples.…”

Section: Mac Does Not Support Amd Cpusmentioning

confidence: 99%

Section: Actsmentioning

confidence: 99%

Section: Actsmentioning

confidence: 99%

See 3 more Smart Citations

Greedy combinatorial test case generation using unsatisfiable cores

Yamada

Biere

Artho

et al. 2016

Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering

View full text Add to dashboard Cite

show abstract

An end‐user‐centric test generation methodology for performance evaluation of mobile networked applications

Al‐tekreeti

Abdrabou

Naik

2019

Software Testing Verif & Rel

View full text Add to dashboard Cite

We propose a model-based test generation methodology to evaluate the impact of the interaction of the wireless network and application configurations on the performance of mobile networked applications. We consider waiting time delay to model wireless network quality. We classify mobile applications into two groups. Group I represents applications where end-user experience is mainly affected by waiting time delay during service consumption, while group II represents applications where end-user experience is affected by waiting time delay before service consumption. Test generation is formulated as an inversion problem. However, for group I applications, solving the inversion problem is expensive. Therefore, we utilize metamorphic testing to mitigate the cost of test oracles. We formulate metamorphic test generation as maximization of the distance between seed and follow-up test cases. Two test coverage criteria are proposed: user experience and user-experience-and-input interaction. Network models are developed for a mobile device that has network access through a WiFi hot spot and uses either transmission control protocol or user datagram protocol. Two mobile applications are used to demonstrate the methodology: multimedia streaming and web browsing. Application of the methodology when user actions are taken into consideration is also addressed. The effectiveness of the methodology is evaluated using two metrics: the incurred time cost and redundancy in the generated test suite. The obtained results show the advantage of casting test generation as an inversion problem, compared with random testing. For apps with intensive performance models, combining metamorphic testing with the methodology has tremendously reduced the cost of test oracles. 15 of 38 state y. Therefore, the SUT desired behaviour introduces the following two constraints (Procedure 3.S1), where B, M , and L are defined as an integer number of multimedia frames:M Ä B and L Ä M 1:(4) 4.1.2. The wireless network model. The end user is streaming via a WiFi hot spot and using the UDP. We assume all the fluctuations in the wireless link and the queuing effects of the different routers along the path from the server to the client device manifest as a time delay. That is, packet loss is negligible. A reusable network model for the UDP scenario was developed in our previous work [15]. We have developed two analytical expressions for the mean and variance of packet interarrival time delay. The probability distribution was matched with the hyper-Erlang distribution. The network impact is captured by three operating parameters (NOPs): the data rate (D), the mean rate of frame arrivals into the access point ( ) per user, and the number of end users (N ) connected to the AP. The data rate according to the IEEE 802.11 a/g standard can be in one of the following values: 6, 9, 12, 18, 24, 32, 48, or 54 Mbps. It mainly relates to the quality of the wireless connection between the AP and the end user. Regarding the number of users N , the network model is validated...

show abstract