Test Case Prioritization for Compilers: A Text-Vector Based Approach

Chen, Junjie; Bai, Yanwei; Hao, Dan; Xiong, Yingfei; Zhang, Hongyu; Zhang, Lu; Xie, Bing

doi:10.1109/icst.2016.19

Cited by 43 publications

(26 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…To accelerate compiler testing, some approaches have been proposed recently [5], [6], including TB−G and LET.…”

Section: Compiler Testing Accelerationmentioning

confidence: 99%

“…TB−G, the first compiler testing acceleration approach, is a text-vector based test-program prioritization approach [5]. It regards each test program as text, and extracts the tokens reflecting bug-relevant characteristics in order to transform each test program into a text-vector.…”

Section: Compiler Testing Accelerationmentioning

confidence: 99%

“…To accelerate compiler testing, some approaches have been proposed in recent years [5], [6], [7]. These approaches utilize some criteria to prioritize test programs, so that test programs that are more likely to trigger compiler bugs are executed earlier.…”

Section: Introductionmentioning

confidence: 99%

“…To evaluate the acceleration effectiveness of COP, we use two widely-used C compilers, i.e., GCC and LLVM, as subjects following the existing work on compiler testing [1], [3], [4], [5], [6]. In particular, We use the constructed dataset in the existing work [6], which is convenient to compare with the state-of-the-art approach LET.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Coverage Prediction for Accelerating Compiler Testing

Chen

Wang

Hao

et al. 2021

IIEEE Trans. Software Eng.

Self Cite

View full text Add to dashboard Cite

Compilers are one of the most fundamental software systems. Compiler testing is important for assuring the quality of compilers. Due to the crucial role of compilers, they have to be well tested. Therefore, automated compiler testing techniques (those based on randomly generated programs) tend to run a large number of test programs (which are test inputs of compilers). The cost for compilation and execution for these test programs is significant. These techniques can take a long period of testing time to detect a relatively small number of compiler bugs. That may cause many practical problems, e.g., bringing a lot of costs including time costs and financial costs, and delaying the development/release cycle. Recently, some approaches have been proposed to accelerate compiler testing by executing test programs that are more likely to trigger compiler bugs earlier according to some criteria. However, these approaches ignore an important aspect in compiler testing: different test programs may have similar test capabilities (i.e., testing similar functionalities of a compiler, even detecting the same compiler bug), which may largely discount their acceleration effectiveness if the test programs with similar test capabilities are executed all the time. Test coverage is a proper approximation to help distinguish them, but collecting coverage dynamically is infeasible in compiler testing since most test programs are generated on the fly by automatic test-generation tools like Csmith. In this paper, we propose the first method to predict test coverage statically for compilers, and then propose to prioritize test programs by clustering them according to the predicted coverage information. The novel approach to accelerating compiler testing through coverage prediction is called COP (short for COverage Prediction). Our evaluation on GCC and LLVM demonstrates that COP significantly accelerates compiler testing, achieving an average of 51.01% speedup in test execution time on an existing dataset including three old release versions of the compilers and achieving an average of 68.74% speedup on a new dataset including 12 latest release versions. Moreover, COP outperforms the state-of-the-art acceleration approach significantly by improving 17.16%∼82.51% speedups in different settings on average.

show abstract

“…To accelerate compiler testing, some approaches have been proposed recently [5], [6], including TB−G and LET.…”

Section: Compiler Testing Accelerationmentioning

confidence: 99%

Section: Compiler Testing Accelerationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Coverage Prediction for Accelerating Compiler Testing

Chen

Wang

Hao

et al. 2021

IIEEE Trans. Software Eng.

Self Cite

View full text Add to dashboard Cite

show abstract

“…13 shows. The Junjie Chen community aims to prioritize test cases for compilers to accelerate the process of compiler testing [66,67]. The A. Kalinov community focuses on test case generation for mpC compiler [68], and the Heung Seok Chae community is interested in test case reduction for retargeted compilers [62,73].…”

Section: Compiler Testing: a Systematic Literature Analysismentioning

confidence: 99%

Compiler testing: a systematic literature analysis

Tang

Zhang

Kong

et al. 2019

Front. Comput. Sci.

View full text Add to dashboard Cite

Compilers are widely-used infrastructures in accelerating the software development, and expected to be trustworthy. In the literature, various testing technologies have been proposed to guarantee the quality of compilers. However, there remains an obstacle to comprehensively characterize and understand compiler testing. To overcome this obstacle, we propose a literature analysis framework to gain insights into the compiler testing area. First, we perform an extensive search to construct a dataset related to compiler testing papers. Then, we conduct a bibliometric analysis to analyze the productive authors, the influential papers, and the frequently tested compilers based on our dataset. Finally, we utilize association rules and collaboration networks to mine the authorships and the communities of interests among researchers and keywords. Some valuable results are reported. We find that the USA is the leading country that contains the most influential researchers and institutions. The most active keyword is "random testing". We also find that most researchers have broad interests within small-scale collaborators in the compiler testing area.

show abstract

Dissimilarity‐based test case prioritization through data fusion

Huang

Towey

et al. 2022

Softw Pract Exp

View full text Add to dashboard Cite

Test case prioritization (TCP) aims at scheduling test case execution so that more important test cases are executed as early as possible. Many TCP techniques have been proposed, according to different concepts and principles, with dissimilarity-based TCP (DTCP) prioritizing tests based on the concept of test case dissimilarity: DTCP chooses the next test case from a set of candidates such that the chosen test case is farther away from previously selected test cases than the other candidates. DTCP techniques typically only use one aspect/granularity of the information or features from test cases to support the prioritization process. In this article, we adopt the concept of data fusion to propose a new family of DTCP techniques, data-fusion-driven DTCP (DDTCP), which attempts to use different information granularities for prioritizing test cases by dissimilarity. We performed an empirical study involving 30 versions of five subject programs, investigating the testing effectiveness and efficiency by comparing DDTCP against DTCP techniques that use a dissimilarity granularity. The experimental results show that not only does DDTCP have better fault-detection rates than single-granularity DTCP techniques, but it also appears to only incur similar prioritization costs. The results also show that DDTCP remains robust over multiple system releases.

show abstract

Test Case Prioritization for Compilers: A Text-Vector Based Approach

Cited by 43 publications

References 47 publications

Coverage Prediction for Accelerating Compiler Testing

Coverage Prediction for Accelerating Compiler Testing

Compiler testing: a systematic literature analysis

Dissimilarity‐based test case prioritization through data fusion

Contact Info

Product

Resources

About