[Engineering Paper] Enabling the Continuous Analysis of Security Vulnerabilities with VulData7

Jiménez, M.; Traon, Yves Le; Papadakis, Mike

doi:10.1109/scam.2018.00014

Cited by 18 publications

(19 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We collected all vulnerabilities reported in NVD for the three systems under study using the VulData7 framework [23]. VulData7 automatically retrieves all declared bug reports and patches by crawling NVD.…”

Section: Data Collectionmentioning

confidence: 99%

“…However, to reduce this threat, we studied three large open-source systems with a large number (3-4 times larger than in previous studies) of real (reported in NVD) vulnerabilities. Moreover, we use a publicly available tool (VulData7 [23]) for data collection and report the followed procedure in order to allow other researcher to replicate and extend our work.…”

Section: Threats To Validitymentioning

confidence: 99%

See 1 more Smart Citation

The importance of accounting for real-world labelling when predicting software vulnerabilities

Jiménez

Rwemalika

Papadakis

et al. 2019

Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of

Self Cite

View full text Add to dashboard Cite

Previous work on vulnerability prediction assume that predictive models are trained with respect to perfect labelling information (includes labels from future, as yet undiscovered vulnerabilities). In this paper we present results from a comprehensive empirical study of 1,898 real-world vulnerabilities reported in 74 releases of three security-critical open source systems (Linux Kernel, OpenSSL and Wiresark). Our study investigates the effectiveness of three previously proposed vulnerability prediction approaches, in two settings: with and without the unrealistic labelling assumption. The results reveal that the unrealistic labelling assumption can profoundly mislead the scientific conclusions drawn; suggesting highly effective and deployable prediction results vanish when we fully account for realistically available labelling in the experimental methodology. More precisely, MCC mean values of predictive effectiveness drop from 0.77, 0.65 and 0.43 to 0.08, 0.22, 0.10 for Linux Kernel, OpenSSL and Wiresark, respectively. Similar results are also obtained for precision, recall and other assessments of predictive efficacy. The community therefore needs to upgrade experimental and empirical methodology for vulnerability prediction evaluation and development to ensure robust and actionable scientific findings. CCS CONCEPTS • Software and its engineering → Software defect analysis.

show abstract

Section: Data Collectionmentioning

confidence: 99%

Section: Threats To Validitymentioning

confidence: 99%

The importance of accounting for real-world labelling when predicting software vulnerabilities

Jiménez

Rwemalika

Papadakis

et al. 2019

Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of

Self Cite

View full text Add to dashboard Cite

show abstract

“…Jimenez et al [8] developed VulData7, an extensible framework (and dataset) of real vulnerabilities, automatically collected from software archives. VulData7 is a general framework that contains vulnerabilities for four security-critical open source project languages at the file level.…”

Section: Related Workmentioning

confidence: 99%

“…Nonetheless, the practical usability of a model depends highly on its ability to localize its prediction (i.e., to provide as fine-grained hit as possible). Most of the state-of-the-art vulnerability prediction models provide prediction results at file [3,8,12,16,21], binary [22] or class [19] level, but even the most fine-grained results stop at the level of functions/methods [4,19]. It means that developers need to explore large chunks of code to confirm and identify the exact spot and type of vulnerability predicted by a model.…”

Section: Introductionmentioning

confidence: 99%

Towards a Prototype Based Explainable JavaScript Vulnerability Prediction Model

Mosolygó

Vándor

Antal

et al. 2021

2021 International Conference on Code Quality (ICCQ)

View full text Add to dashboard Cite

Security has become a central and unavoidable aspect of today's software development. Practitioners and researchers have proposed many code analysis tools and techniques to mitigate security risks. These tools apply static and dynamic analysis or, more recently, machine learning. Machine learning models can achieve impressive results in finding and forecasting possible security issues in programs. However, most of the current approaches fall short of developer demands in two areas at least: explainability and granularity of predictions.In this paper, we propose a novel and simple yet, promising approach to identify potentially vulnerable source code in JavaScript programs. The model improves the state-of-the-art in terms of explainability and prediction granularity as it gives results at the level of individual source code lines, which is fine-grained enough for developers to take immediate actions. Additionally, the model explains each predicted line (i.e., provides the most similar vulnerable line from the training set) using a prototype-based approach. In a study of 186 real-world and confirmed JavaScript vulnerability fixes of 91 projects, the approach could flag 60% of the known vulnerable lines on average by marking only 10% of the code-base, but in particular cases, the model identified 100% of the vulnerable code lines while flagging only 8.72% of the code-base.

show abstract

“…In their work, Jimenez et al [8] proposed an extensible framework (Vul-Data7) and dataset of real vulnerabilities, automatically collected from software archives. They presented the capabilities of their framework on 4 large systems, but one can extend the framework to meet specific needs.…”

Section: Rq2: How Do Vulnerability Mitigation Patches Affect Test Code?mentioning

confidence: 99%

Inspecting JavaScript Vulnerability Mitigation Patches with Automated Fix Generation in Mind

Hegedűs

2020

Computational Science and Its Applications – ICCSA 2020

View full text Add to dashboard Cite

Software security has become a primary concern for both the industry and academia in recent years. As dependency on critical services provided by software systems grows globally, a potential security threat in such systems poses higher and higher risks (e.g. economical damage, a threat to human life, criminal activity). Finding potential security vulnerabilities at the code level automatically is a very popular approach to aid security testing. However, most of the methods based on machine learning and statistical models stop at listing potentially vulnerable code parts and leave their validation and mitigation to the developers. Automatic program repair could fill this gap by automatically generating vulnerability mitigation code patches. Nonetheless, it is still immature, especially in targeting security-relevant fixes. In this work, we try to establish a path towards automatic vulnerability fix generation techniques in the context of JavaScript programs. We inspect 361 actual vulnerability mitigation patches collected from vulnerability databases and GitHub. We found that vulnerability mitigation patches are not short on average and in many cases affect not just program code but test code as well. These results point towards that a general automatic repair approach targeting all the different types of vulnerabilities is not feasible. The analysis of the code properties and fix patterns for different vulnerability types might help in setting up a more realistic goal in the area of automatic JavaScript vulnerability repair.

show abstract

[Engineering Paper] Enabling the Continuous Analysis of Security Vulnerabilities with VulData7

Cited by 18 publications

References 10 publications

The importance of accounting for real-world labelling when predicting software vulnerabilities

The importance of accounting for real-world labelling when predicting software vulnerabilities

Towards a Prototype Based Explainable JavaScript Vulnerability Prediction Model

Inspecting JavaScript Vulnerability Mitigation Patches with Automated Fix Generation in Mind

Contact Info

Product

Resources

About