Automatic Test Case Reduction for OpenCL

Pflanzer, Moritz; Donaldson, Alastair F.; Lascu, Andrei

doi:10.1145/2909437.2909439

Cited by 12 publications

(7 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Compiler testing is a difficult and essential task due to the complexity and importance of compilers, and thus it has attracted extensive attentions from researchers [37], [38], [39], [40], [41]. Research on compiler testing can be divided into three main aspects, i.e., test-program generation, testoracle construction, and test-execution optimization.…”

Section: Related Work 81 Compiler Testingmentioning

confidence: 99%

Coverage Prediction for Accelerating Compiler Testing

Chen

Wang

Hao

et al. 2021

IIEEE Trans. Software Eng.

View full text Add to dashboard Cite

Compilers are one of the most fundamental software systems. Compiler testing is important for assuring the quality of compilers. Due to the crucial role of compilers, they have to be well tested. Therefore, automated compiler testing techniques (those based on randomly generated programs) tend to run a large number of test programs (which are test inputs of compilers). The cost for compilation and execution for these test programs is significant. These techniques can take a long period of testing time to detect a relatively small number of compiler bugs. That may cause many practical problems, e.g., bringing a lot of costs including time costs and financial costs, and delaying the development/release cycle. Recently, some approaches have been proposed to accelerate compiler testing by executing test programs that are more likely to trigger compiler bugs earlier according to some criteria. However, these approaches ignore an important aspect in compiler testing: different test programs may have similar test capabilities (i.e., testing similar functionalities of a compiler, even detecting the same compiler bug), which may largely discount their acceleration effectiveness if the test programs with similar test capabilities are executed all the time. Test coverage is a proper approximation to help distinguish them, but collecting coverage dynamically is infeasible in compiler testing since most test programs are generated on the fly by automatic test-generation tools like Csmith. In this paper, we propose the first method to predict test coverage statically for compilers, and then propose to prioritize test programs by clustering them according to the predicted coverage information. The novel approach to accelerating compiler testing through coverage prediction is called COP (short for COverage Prediction). Our evaluation on GCC and LLVM demonstrates that COP significantly accelerates compiler testing, achieving an average of 51.01% speedup in test execution time on an existing dataset including three old release versions of the compilers and achieving an average of 68.74% speedup on a new dataset including 12 latest release versions. Moreover, COP outperforms the state-of-the-art acceleration approach significantly by improving 17.16%∼82.51% speedups in different settings on average.

show abstract

Section: Related Work 81 Compiler Testingmentioning

confidence: 99%

Coverage Prediction for Accelerating Compiler Testing

Chen

Wang

Hao

et al. 2021

IIEEE Trans. Software Eng.

View full text Add to dashboard Cite

show abstract

“…CLSmith test cases require reduction in order to expose the underlying bug. An automated approach to OpenCL test case reduction is presented in [25], though it requires on average 100 minutes for each test case using a parallelized implementation (and over 6 hours if this parallelization is not available); the authors also suggest a final manual pass after automated reduction. In contrast, DeepSmith learned to program from humans, and humans do not typically write such large kernel functions.…”

Section: Comparison To State-of-the-artmentioning

confidence: 99%

Compiler fuzzing through deep learning

Cummins

Petoumenos

Murray

et al. 2018

Proceedings of the 27th ACM SIGSOFT International Symposium on Software Testing and Analysis

111

View full text Add to dashboard Cite

Random program generation -fuzzing -is an effective technique for discovering bugs in compilers but successful fuzzers require extensive development effort for every language supported by the compiler, and often leave parts of the language space untested.We introduce DeepSmith, a novel machine learning approach to accelerating compiler validation through the inference of generative models for compiler inputs. Our approach infers a learned model of the structure of real world code based on a large corpus of open source code. Then, it uses the model to automatically generate tens of thousands of realistic programs. Finally, we apply established differential testing methodologies on them to expose bugs in compilers. We apply our approach to the OpenCL programming language, automatically exposing bugs with little effort on our side. In 1,000 hours of automated testing of commercial and open source compilers, we discover bugs in all of them, submitting 67 bug reports. Our test cases are on average two orders of magnitude smaller than the state-of-the-art, require 3.03× less time to generate and evaluate, and expose bugs which the state-of-the-art cannot. Our random program generator, comprising only 500 lines of code, took 12 hours to train for OpenCL versus the state-of-the-art taking 9 man months to port from a generator for C and 50,000 lines of code. With 18 lines of code we extended our program generator to a second language, uncovering crashes in Solidity compilers in 12 hours of automated testing. CCS CONCEPTS• Software and its engineering → Software testing and debugging;

show abstract

“…比如, Regehr 等 [12] 提出 CReduce 工具, 将触发 C 语言编译器缺陷的测试程序进行化简, 即将该程序化简为最小的程序但仍能触发该编译器缺陷. Pflanzer 等 [13] 进一步将 CReduce 拓展到化简 OpenCL 编译器的测试程序. 上述两个方法都是特定于某种语言的编译器的程序化简工具.…”

Section: 编译器调试与测试unclassified

Static duplicate bug-report identification for compilers

Chen

Hao

et al. 2019

Sci. Sin.-Inf.

View full text Add to dashboard Cite

Automatic Test Case Reduction for OpenCL

Cited by 12 publications

References 10 publications

Coverage Prediction for Accelerating Compiler Testing

Coverage Prediction for Accelerating Compiler Testing

Compiler fuzzing through deep learning

Static duplicate bug-report identification for compilers

Contact Info

Product

Resources

About