SSPCatcher: Learning to catch security patches

Sawadogo, Arthur D.; Bissyandé, Tegawendé F.; Moha, Naouel; Allix, Kevin; Klein, Jacques; Li, Li; Traon, Yves Le

doi:10.1007/s10664-022-10168-9

Cited by 11 publications

(6 citation statements)

References 43 publications

(58 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Either incorrect commits or line changes were selected. Substantial work has been conducted for ML-based models to identify correct vulnerability patches [12], [72]. Semantic filters or heuristics for correct vulnerability fixing lines is currently lacking.…”

Section: Discussionmentioning

confidence: 99%

Data Quality for Software Vulnerability Datasets

Croft¹,

Babar²,

Kholoosi³

2023

Preprint

View full text Add to dashboard Cite

The use of learning-based techniques to achieve automated software vulnerability detection has been of longstanding interest within the software security domain. These data-driven solutions are enabled by large software vulnerability datasets used for training and benchmarking. However, we observe that the quality of the data powering these solutions is currently ill-considered, hindering the reliability and value of produced outcomes. Whilst awareness of software vulnerability data preparation challenges is growing, there has been little investigation into the potential negative impacts of software vulnerability data quality. For instance, we lack confirmation that vulnerability labels are correct or consistent. Our study seeks to address such shortcomings by inspecting five inherent data quality attributes for four state-of-the-art software vulnerability datasets and the subsequent impacts that issues can have on software vulnerability prediction models. Surprisingly, we found that all the analyzed datasets exhibit some data quality problems.In particular, we found 20-71% of vulnerability labels to be inaccurate in real-world datasets, and 17-99% of data points were duplicated. We observed that these issues could cause significant impacts on downstream models, either preventing effective model training or inflating benchmark performance. We advocate for the need to overcome such challenges. Our findings will enable better consideration and assessment of software vulnerability data quality in the future.

show abstract

Section: Discussionmentioning

confidence: 99%

Data Quality for Software Vulnerability Datasets

Croft¹,

Babar²,

Kholoosi³

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…It's crucial in software maintenance to ensure that patches not only resolve issues but also maintain the overall functionality and performance of the software. Vulnerability detection: The FixMe dataset can be applied in various software security tasks, including vulnerability detection [1,13,14,28,37], CWE type prediction [38], and vulnerability fix detection [39][40][41]. Labeling the new code fixed by the commits or code after can be considered benign sample and code before can be taken as vulnerable samples for vulnerability detection as binary classification.…”

Section: Applications Of Fixmementioning

confidence: 99%

An Incremental Lightweight Method for Vulnerability Data Collection for Security Patches

Bhandari,

Gavric

2024

Preprint

View full text Add to dashboard Cite

Automated Program Repair enhances developers’ efficiency by reducing the time dedicated to debugging and fixing source code vulnerabilities. However, the existing datasets utilized to train patch generation models suffer from various limitations, notably the limited bug context, limited size, and reliance on synthetic and unrealistic source code samples. Additionally, existing dataset extraction tools lack an incremental approach, rendering the inclusion of new data a resource-intensive endeavor in terms of both time and computational resources. In this paper, we introduce FixMe , a novel framework designed for generating patches across diverse programming languages. This open-source tool facilitates an incremental approach to gathering vulnerability records from the Common Vulnerabilities and Exposures (CVE) database. Incorporating an incremental methodology accelerates the data acquisition process, encompassing newly identified vulnerabilities and their corresponding patch pairs. Our dataset curation method involves the extraction of security issues, acquiring vulnerability-fixing commits, and extracting source code from relevant projects. The resultant dataset comprises 18,063 CVEs from 2,133 commits, encompassing 19,109 patch sets with 44,059 vulnerability-fix pairs (patch hunks). This dataset spans 884 open-source projects hosted in git-based systems. Additionally , the study explores fostering the potential of APR techniques in reducing debugging costs and improving software quality by generating patches for fixing vulnerabilities automatically.

show abstract

“…For example, Mirhosseini et al [37] found that projects that use automated pull requests upgrade dependencies 1.6x often as projects that did not use any tools. Other studies propose methods for detecting or obtaining more information about vulnerabilities from different software artifacts, including commits [39]- [43], bug reports [29], [44], [45], and mailing lists [46], [47]. Unlike these studies, we do not aim to detect vulnerabilities but to determine which libraries are vulnerable based on a vulnerability report.…”

Section: Related Workmentioning

confidence: 99%