Large-Scale Empirical Study of Important Features Indicative of Discovered Vulnerabilities to Assess Application Security

Zhang, Mengyuan; Carnavalet, Xavier de Carné de; Wang, Lingyu; Ragab, Ahmed

doi:10.1109/tifs.2019.2895963

Cited by 31 publications

(43 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…All these studies, which were based on different datasets, observed that a weak but statistically significant correlation exists between the software metrics and the existence of vulnerabilities, whereas they also produced metric-based vulnerability prediction models of satisfactory accuracy. In addition to this, recent studies have shown that the combination of different software metrics lead to better vulnerability predictors, and, thus, it may render a meaningful approach for enhancing security assessment Zhang et al, 2019).…”

Section: Existing Software Security Assessment Approachesmentioning

confidence: 99%

“…However, the reliability of these models is hindered since they are based exclusively on OO metrics, which were found to be only weak indicators of vulnerabilities (Shin & Williams, 2008;Chowdhury & Zulkernine, 2011;Shin et al, 2011;Siavvas et al, 2017;Siavvas et al, 2017b;Moshtari et al, 2013;Moshtari & Sami, 2016;Stuckman et al, 2017;Ferenc et al, 2019;Jimenez et al, 2019;Zhang et al, 2019). In addition, their parameters (i.e., thresholds, weights, etc.)…”

Section: Existing Software Security Assessment Approachesmentioning

confidence: 99%

See 1 more Smart Citation

A hierarchical model for quantifying software security based on static analysis alerts and software metrics

et al. 2021

View full text Add to dashboard Cite

Despite the acknowledged importance of quantitative security assessment in secure software development, current literature still lacks an efficient model for measuring internal software security risk. To this end, in this paper, we introduce a hierarchical security assessment model (SAM), able to assess the internal security level of software products based on low-level indicators, i.e., security-relevant static analysis alerts and software metrics. The model, following the guidelines of ISO/IEC 25010, and based on a set of thresholds and weights, systematically aggregates these low-level indicators in order to produce a high-level security score that reflects the internal security level of the analyzed software. The proposed model is practical, since it is fully automated and operationalized in the form of a standalone tool and as part of a broader Computer-Aided Software Engineering (CASE) platform. In order to enhance its reliability, the thresholds of the model were calibrated based on a repository of 100 popular software applications retrieved from Maven Repository. Furthermore, its weights were elicited in a way to chiefly reflect the knowledge expressed by the Common Weakness Enumeration (CWE), through a novel weights elicitation approach grounded on popular decision-making techniques. The proposed model was evaluated on a large repository of 150 open-source software applications retrieved from GitHub and 1200 classes retrieved from the OWASP Benchmark. The results of the experiments revealed the capacity of the proposed model to reliably assess internal security at both product level and class level of granularity, with sufficient discretion power. They also provide preliminary evidence for the ability of the model to be used as the basis for vulnerability prediction. To the best of our knowledge, this is the first fully automated, operationalized and sufficiently evaluated security assessment model in the modern literature.

show abstract

Section: Existing Software Security Assessment Approachesmentioning

confidence: 99%

Section: Existing Software Security Assessment Approachesmentioning

confidence: 99%

A hierarchical model for quantifying software security based on static analysis alerts and software metrics

et al. 2021

View full text Add to dashboard Cite

show abstract

“…The classification approach is the preferable one in the Vulnerability Prediction (VP) domain. SVPs can be based on different types of features: Software Metrics (SM) [2,29,30], Text Mining (TM) [36,[41][42][43] features, ASA alerts [27,44], and hybrid ones [45][46][47]. To create SVPs, different algorithms are used: decision trees [43,45], random forests [43,48,49], boosted trees [45], Support Vector Machines (SVM) [50], linear discriminant analysis [2], Bayesian Networks [2], linear regression [45], the naive Bayes classifier [41], K-nearest neighbors [43], as well as artificial neural networks and deep learning [36,47,51,52].…”

Section: Related Workmentioning

confidence: 99%

Efficient Feature Selection for Static Analysis Vulnerability Prediction

Filus

Boryszko

Domańska

et al. 2021

Sensors

View full text Add to dashboard Cite

Common software vulnerabilities can result in severe security breaches, financial losses, and reputation deterioration and require research effort to improve software security. The acceleration of the software production cycle, limited testing resources, and the lack of security expertise among programmers require the identification of efficient software vulnerability predictors to highlight the system components on which testing should be focused. Although static code analyzers are often used to improve software quality together with machine learning and data mining for software vulnerability prediction, the work regarding the selection and evaluation of different types of relevant vulnerability features is still limited. Thus, in this paper, we examine features generated by SonarQube and CCCC tools, to identify those that can be used for software vulnerability prediction. We investigate the suitability of thirty-three different features to train thirteen distinct machine learning algorithms to design vulnerability predictors and identify the most relevant features that should be used for training. Our evaluation is based on a comprehensive feature selection process based on the correlation analysis of the features, together with four well-known feature selection techniques. Our experiments, using a large publicly available dataset, facilitate the evaluation and result in the identification of small, but efficient sets of features for software vulnerability prediction.

show abstract

“…A large number of VPMs has been proposed in the literature over the past decade [ 3 ]. As stated in [ 14 ], the main VPMs that can be found in the literature utilize software metrics [ 10 , 17 , 18 ], text mining [ 9 , 15 ], and security-related static analysis alerts [ 12 , 13 ] to predict vulnerabilities in software products. However, as stated before, while these models have demonstrated promising results in predicting the existence of vulnerabilities in the software projects on which they have been built (i.e., within-project vulnerability prediction), they have failed to demonstrate sufficient performance in cross-project vulnerability prediction.…”

Section: Related Workmentioning

confidence: 99%

“…Several VPMs have been proposed over the years utilizing various software factors as inputs for predicting the existence of vulnerable components, including software metrics [ 10 ], text mining [ 9 , 11 ], and static analysis alerts [ 12 , 13 ]. Although these models have demonstrated promising results in predicting the existence of vulnerabilities in the projects on which they have been trained (i.e., within-project vulnerability prediction), they failed to sufficiently predict the existence of vulnerabilities in previously unknown software projects (i.e., cross-project vulnerability prediction) [ 3 , 14 ].…”

Section: Introductionmentioning

confidence: 99%

Cross-Project Vulnerability Prediction Based on Software Metrics and Deep Learning

Kalouptsoglou

Siavvas

Tsoukalas

et al. 2020

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Vulnerability prediction constitutes a mechanism that enables the identification and mitigation of software vulnerabilities early enough in the development cycle, improving the security of software products, which is an important quality attribute according to ISO/IEC 25010. Although existing vulnerability prediction models have demonstrated sufficient accuracy in predicting the occurrence of vulnerabilities in the software projects with which they have been trained, they have failed to demonstrate sufficient accuracy in cross-project prediction. To this end, in the present paper we investigate whether the adoption of deep learning along with software metrics may lead to more accurate cross-project vulnerability prediction. For this purpose, several machine learning (including deep learning) models are constructed, evaluated, and compared based on a dataset of popular real-world PHP software applications. Feature selection is also applied with the purpose to examine whether it has an impact on cross-project prediction. The results of our analysis indicate that the adoption of software metrics and deep learning may result in vulnerability prediction models with sufficient performance in cross-project vulnerability prediction. Another interesting conclusion is that the performance of the models in cross-project prediction is enhanced when the projects exhibit similar characteristics with respect to their software metrics.

show abstract

Large-Scale Empirical Study of Important Features Indicative of Discovered Vulnerabilities to Assess Application Security

Cited by 31 publications

References 32 publications

A hierarchical model for quantifying software security based on static analysis alerts and software metrics

A hierarchical model for quantifying software security based on static analysis alerts and software metrics

Efficient Feature Selection for Static Analysis Vulnerability Prediction

Cross-Project Vulnerability Prediction Based on Software Metrics and Deep Learning

Contact Info

Product

Resources

About