Identifying the characteristics of vulnerable code changes: an empirical study

Abstract

To focus the efforts of security experts, the goals of this empirical study are to analyze which security vulnerabilities can be discovered by code review, identify characteristics of vulnerable code changes, and identify characteristics of developers likely to introduce vulnerabilities. Using a three-stage manual and automated process, we analyzed 267,046 code review requests from 10 open source projects and identified 413 Vulnerable Code Changes (VCC). Some key results include: (1) code review can identify common types of vulnerabilities;(2) while more experienced contributors authored the majority of the VCCs, the less experienced contributors' changes were 1.8 to 24 times more likely to be vulnerable; (3) the likelihood of a vulnerability increases with the number of lines changed, and (4) modified files are more likely to contain vulnerabilities than new files. Knowing which code changes are more prone to contain vulnerabilities may allow a security expert to concentrate on a smaller subset of submitted code changes. Moreover, we recommend that projects should: (a) create or adapt secure coding guidelines, (b) create a dedicated security review team, (c) ensure detailed comments during review to help knowledge dissemination, and (d) encourage developers to make small, incremental changes rather than large changes.

The current practices of changing secure software

“…Bosu et al [7] and Bosu [6] performed a study on the characteristics of vulnerable code changes. The study found that careful review should be done to modified files and that code changes are made often by less-experienced developers.…”

“…There are extensive work on techniques, tools, and processes to develop a secure development process, including the development of secure software using the agile approach [5] and identification of characteristics of vulnerable code change [7]. This study enumerates the security aspects that the developers address when they modify their software.…”

Identification of the Impacts of Code Changes on the Security of Software

“…al. analyzed the security vulnerabilities that could be discovered by code review, identified the characteristics of vulnerable code, and the developers that wrote it [18]. They found that the common types of identified vulnerabilities are buffer overflow and cross-site scripting.…”

Profiling Android Vulnerabilities

“…Additionally, our focus is specific to the Android system for which there is scarce work in the literature of vulnerability prediction. Perhaps the most relevant work to ours is by Bosu et al [34] who studied the characteristics of vulnerable code changes. However, our study differs from it in the following three ways: a) we study changes that fix a vulnerability rather than ones that introduce it, b) we study known, severe and exploitable vulnerabilities rather than those found by code reviews and c) we aim at categorizing and describing the origin of the Android vulnerabilities and not generic changes that might indicate the introduction of one.…”