Adversarial Examples for Models of Code

Abstract

Neural models of code have shown impressive performance for tasks such as predicting method names and identifying certain kinds of bugs. In this paper, we show that these models are vulnerable to adversarial examples, and introduce a novel approach for attacking trained models of code with adversarial examples. The main idea is to force a given trained model to make an incorrect prediction as specified by the adversary by introducing small perturbations that do not change the program's semantics. To find such perturbations, we present a new technique for Discrete Adversarial Manipulation of Programs (DAMP). DAMP works by deriving the desired prediction with respect to the model's inputs while holding the model weights constant, and following the gradients to slightly modify the code.To defend a model against such attacks, we propose placing a defensive model (Anti-DAMP) in front of it. Anti-DAMP detects unlikely mutations and masks them before feeding the input to the downstream model.We show that our DAMP attack is effective across three neural architectures: code2vec, GGNN, and GNN-FiLM, in both Java and C#. We show that DAMP has up to 89% success rate in changing a prediction to the adversary's choice ("targeted attack"), and a success rate of up to 94% in changing a given prediction to any incorrect prediction ("non-targeted attack"). By using Anti-DAMP, the success rate of the attack drops drastically for both targeted and non-targeted attacks, with a minor penalty of 2% relative degradation in accuracy while not performing under attack.

CoCoFuzzing: Testing Neural Code Models with Coverage-Guided Fuzzing

“…Given the non-trivial search space constructed by combinations of mutation operators, CoCoFuzzing employs a neuron coverage-guided (short for NC-guided) approach (line 5-25), to search for certain types of mutated programs. At a high level, CoCoFuzzing searches for program mutations that activate new neurons (line [13][14][15][16][17] while controlling the maximum mutations on one seed program (line 5 and a threshold MAX), for both naturalness and the feasibility of utilizing metamorphic testing (i.e., mutated programs can use the same test oracle as the seed program).…”

“…Algorithm 1 details how CoCoFuzzing uses neuron coverage in searching for new test data (line 5-25). In each iteration (line [10][11][12][13][14][15][16][17][18], CoCoFuzzing tries all the mutation operators and identifies one mutated program that activates the most number of new neurons. This mutated program is produced as one test data (line 24) and also kept for the next mutation iteration.…”

“…Table I summarizes the ten mutation operators and how each mutation operator can be applied to transform a program. Note that, among the ten operators, Op1 and Op10 are proposed and experimented in prior studies [15], [16] to generate adversarial examples for neural code models. The other eight mutation operators are first used in this work.…”

“…• (Op1) Dead store [16]: inserts an unused variable declaration with one primitive type (e.g., string, int, double, and long, etc.) to a randomly selected basic block in the program.…”

“…Recently, some adversarial example generation approaches for source code were proposed, e.g., [15] performed variable name replacements to perturb the programs and [16] proposed to use both variables renaming and dead code (i.e., unused variable declaration) insertion to generate semantically equivalent adversarial examples. However, the types of perturbations introduced by the two transformations are limited, as we have witnessed many other types of noise/perturbation in real-world software programs that sharing the same semantics [17].…”