Neural Transfer Learning for Repairing Security Vulnerabilities in C Code

Abstract

In this paper, we address the problem of automatic repair of software vulnerabilities with deep learning. The major problem with data-driven vulnerability repair is that the few existing datasets of known confirmed vulnerabilities consist of only a few thousand examples. However, training a deep learning model often requires hundreds of thousands of examples. In this work, we leverage the intuition that the bug fixing task and the vulnerability fixing task are related and that the knowledge learned from bug fixes can be transferred to fixing vulnerabilities. In the machine learning community, this technique is called transfer learning. In this paper, we propose an approach for repairing security vulnerabilities named VRepair which is based on transfer learning. VRepair is first trained on a large bug fix corpus and is then tuned on a vulnerability fix dataset, which is an order of magnitude smaller. In our experiments, we show that a model trained only on a bug fix corpus can already fix some vulnerabilities. Then, we demonstrate that transfer learning improves the ability to repair vulnerable C functions. We also show that the transfer learning model performs better than a model trained with a denoising task and fine-tuned on the vulnerability fixing task. To sum up, this paper shows that transfer learning works well for repairing security vulnerabilities in C compared to learning on a small dataset.

A Survey of Learning-based Automated Program Repair

“…Besides, researchers use long short-term memory (LSTM) architecture to capture the long-distance dependencies among code sequences [20,107]. Recently, as a variant of the Seq2Seq model, Transformer [150] has been considered the state-of-the-art NMT repair architecture due to the self-attention mechanism [25,26,40].…”

“…• In the data pre-processing phase, a given software buggy code snippet (e.g., buggy statement) is taken as the input and the processed code tokens are returned. According to existing learning-based APR studies [25,26], there generally exist three potential ways to pre-process the buggy code: code context, abstraction, and tokenization. First, code context information refers to other correlated non-buggy lines within the buggy program.…”

“…In the data pre-processing phase, a given software buggy code snippet (e.g., a buggy function) is taken as the input and the processed code tokens are returned. According to existing learning-based repair studies [25,26], the data pre-processing phase generally consists of three parts: code abstraction, code context and code tokenization.…”

“…Instead of renaming rare identifiers through a custom abstraction process, SequenceR [24] utilizes the copy mechanism to generate candidate patches with a large set of tokens. Chen et al [25] adopt the raw source code as they think abstracted code may hide valuable information about the variable that can be learned by word embedding. A strategy similar to Chen et al [25] is also implemented in other learning-based APR techniques, such as in CODIT [20], CIRCLE [159] and TFix [13].…”

“…Chen et al [25] adopt the raw source code as they think abstracted code may hide valuable information about the variable that can be learned by word embedding. A strategy similar to Chen et al [25] is also implemented in other learning-based APR techniques, such as in CODIT [20], CIRCLE [159] and TFix [13].…”