Concolic testing for functional languages

Abstract

Concolic testing is a software testing technique combining concrete execution of a program (given specific input, along specific paths) with symbolic execution (generating new test inputs that give better path coverage than random test case generation). Concolic testing has so far been applied, mainly at the level of bytecode or assembly code, to programs written in imperative languages that manipulate primitive data types such as integers and arrays. In this paper, we demonstrate its application to a functional programming language core, a subset of the core language of Erlang, that supports pattern matching, structured recursive data types such as lists, recursion and higher-order functions. Moreover, we present CutEr, a tool implementing this testing technique. We describe CutEr's architecture, the challenges that need to be addressed by such a tool, its current limitations, and report some experiences from its use.

Sound and Complete Concolic Testing for Higher-order Functions

“…These programs ultimately come from other papers; see figure 18. The second source is CutEr [18], the Erlang concolic tester. We collected all of the test cases in CutEr's test suite that use higher-order functions and translated them to our prototype's language.…”

“…Second, they enable concretization, permitting symbolic evaluation to seamlessly switch to concrete evaluation and back, thus facilitating interoperability with external libraries. Testament to the success of concolic testing is adaptations to a gamut of linguistic, platform and application settings [3,6,7,12,14,15,17,21,22,23,25,29,30,35,37,38,39,41,43].…”

“…Alas, SMT solvers largely deal with first-order formulas that cannot capture higher-order properties of inputs. As a result, existing concolic testers struggle with JavaScript, Python or Racket components whose inputs are often higher-order functions and fall back to incomplete approximations [17,28,31,36].…”

Lazy counterfactual symbolic execution

“…However, SmtEn does not symbolically execute general purpose Haskell code, which makes G2 more usable as an offthe-shelf debugging aid. Symbolic Execution for Functional Languages CutEr [10,11] is a symbolic execution engine for Erlang programs. SCV [24,35] is a static contract verifier for Racket based on symbolic execution.…”

Concolic Testing in CLP

“…Although concolic testing is quite popular in imperative and object-oriented languages, only a few works can be found in the context of functional and logic programming languages. Some notable exceptions are those of Giantsios et al (2015) and Palacios and Vidal (2015) for a functional language, and those of Vidal (2014) and Mesnard et al (2015a) for a logic language. In the context of logic programming, concolic execution becomes particularly challenging because computing the alternatives of a predicate call is not as straightforward as in imperative programming, where negating a condition suffices.…”