Why Johnny Can’t Pentest: An Analysis of Black-Box Web Vulnerability Scanners

Doupé, Adam; Cova, Marco; Vigna, Giovanni

doi:10.1007/978-3-642-14215-4_7

Cited by 156 publications

(139 citation statements)

References 5 publications

Supporting

Mentioning

129

Contrasting

Unclassified

Order By: Relevance

“…We also used the Web Input Vector Extractor Teaser (WIVET) [30] benchmarking project to examine the crawling capabilities of the evaluated scanners. To assess the accuracy of each scanner, we calculated (1) the true positive rates (TPRs); (2) the true negative rates (TNRs); (3) the false positive rates (FPRs); (4) the false negative rates (FNRs); (5) the positive predictive values (PPVs); (6) the negative predictive values (NPVs); (7) and the false omission rates (FORs) [31,32]. We also measured the vulnerability detection accuracy of the evaluated scanners and their associatedmeasures.…”

Section: Web Vulnerability Scanners Selectionmentioning

confidence: 99%

“…Prior work has also addressed the performance of web security vulnerability scanners by either evaluating the detection effectiveness of a set of scanners [18,19,32,58] or developing techniques that can be incorporated into these tools to increase their detection accuracy [5,60,61]. Other research attempts focused on comparing the effectiveness of dynamic testing with other security testing approaches (e.g., static testing or manual code review) [62].…”

Section: Web Security Scanners: Detection Effectivenessmentioning

confidence: 99%

See 1 more Smart Citation

Performance-Based Comparative Assessment of Open Source Web Vulnerability Scanners

Alsaleh

Alomar

Alshreef

et al. 2017

Security and Communication Networks

View full text Add to dashboard Cite

The widespread adoption of web vulnerability scanners and the differences in the functionality provided by these tool-based vulnerability detection approaches increase the demand for testing their detection effectiveness. Despite the advantages of dynamic testing approaches, the literature lacks studies that systematically evaluate the performance of open source web vulnerability scanners. The main objectives of this study are to assess the performance of open source scanners from multiple perspectives and to examine their detection capability. This paper presents the results of a comparative evaluation of the security features as well as the performance of four web vulnerability detection tools. We followed this comparative assessment with a case study in which we evaluate the level of agreement between the results reported by two open source web vulnerability scanners. Given that the results of our comparative evaluation did not show significant performance differences among the scanners while the results of the conducted case study revealed high level of disagreement between the reports generated by different scanners, we conclude that the inconsistencies between the reports generated by different scanners might not necessarily correlate with their performance properties. We also present some recommendations for helping developers of web vulnerabilities scanners to improve their tools' capabilities.

show abstract

Section: Web Vulnerability Scanners Selectionmentioning

confidence: 99%

Section: Web Security Scanners: Detection Effectivenessmentioning

confidence: 99%

Performance-Based Comparative Assessment of Open Source Web Vulnerability Scanners

Alsaleh

Alomar

Alshreef

et al. 2017

Security and Communication Networks

View full text Add to dashboard Cite

show abstract

“…Previous research analyzing the effectiveness of black-box vulnerability scanners in discovering vulnerabilities in traditional client-server web applications demonstrated that these scanners are full of limitations and fall far short of the "point-and-click" tools that they aim to be. This research additionally concluded that the ability to crawl a web application is as important as the actual ability to detect vulnerabilities [31].…”

Section: Black-box Vulnerability Scannersmentioning

confidence: 93%

“…One common approach that these tools employ is "fuzzing," in which they bombard the application's input parameters with input data in an attempt to trigger a vulnerability. Fuzzing is a simple technique that offers a high benefit-cost ratio, especially because it accesses a web application in the same way users do and can be utilized independent of the technology that powers the web-application [31,76]. Although more sophisticated tools use intelligent heuristics to perform their attacks, fuzzing alone cannot provide a complete picture of the overall security and must be used in conjunction with other techniques.…”

Section: The Crawl Modulementioning

confidence: 99%

See 1 more Smart Citation

Measurement and detection of security properties of client-side web applications

Weissbacher¹

View full text Add to dashboard Cite

Model-based security testing: a taxonomy and systematic classification

Felderer

Zech

Breu

et al. 2015

Softw. Test. Verif. Reliab.

View full text Add to dashboard Cite

Model-based security testing relies on models to test whether a software system meets its security requirements. It is an active research field of high relevance for industrial applications, with many approaches and notable results published in recent years. This article provides a taxonomy for model-based security testing approaches. It comprises filter criteria (i.e. model of system security, security model of the environment and explicit test selection criteria) as well as evidence criteria (i.e. maturity of evaluated system, evidence measures and evidence level). The taxonomy is based on a comprehensive analysis of existing classification schemes for model-based testing and security testing. To demonstrate its adequacy, 119 publications on model-based security testing are systematically extracted from the five most relevant digital libraries by three researchers and classified according to the defined filter and evidence criteria. On the basis of the classified publications, the article provides an overview of the state of the art in model-based security testing and discusses promising research directions with regard to security properties, coverage criteria and the feasibility and return on investment of model-based security testing. 120 M. FELDERER ET AL. 9126 [4] defining security as a functional quality characteristic. However, it seems desirable that security testing directly targets the previous security properties, as opposed to taking the detour of functional tests of security mechanisms. This view is supported by the ISO/IEC 25010 [2] standard that revises ISO/IEC 9126 and introduces security as a new quality characteristic that is not included in the characteristic functionality any more.Because the former kind of (non-functional) security properties describes all executions of a system, this kind of security testing is intrinsically hard. Because testing cannot show the absence of faults, an immediately useful perspective directly considers the violation of these properties. This has resulted in the development of specific testing techniques such as penetration testing that simulates attacks to exploit vulnerabilities. Penetration tests are difficult to craft because tests often do not directly cause observable security exploits, and because the testers must think like an attacker [5], which requires specific expertise. During penetration testing, testers build a mental model of security properties, security mechanisms and possible attacks against the system and its environment. It seems intuitive that security testing can benefit from specifying these security test models in an explicit and processable way. Security test models provide guidance for the systematic and effective specification and documentation of security test objectives and security test cases, as well as for their automated generation and evaluation.The variant of testing that relies on explicit models that encode information on the system under test and/or its environment is called model-based testing (MBT) [6,7]. Especially in ...

show abstract

Why Johnny Can’t Pentest: An Analysis of Black-Box Web Vulnerability Scanners

Cited by 156 publications

References 5 publications

Performance-Based Comparative Assessment of Open Source Web Vulnerability Scanners

Performance-Based Comparative Assessment of Open Source Web Vulnerability Scanners

Measurement and detection of security properties of client-side web applications

Model-based security testing: a taxonomy and systematic classification

Contact Info

Product

Resources

About