LEOPARD: Identifying Vulnerable Code for Vulnerability Assessment Through Program Metrics

Du, Xianfeng; Chen, Bihuan; Li, Yuekang; Guo, Jun; Zhou, Yaqin; Liu, Yang; Jiang, Yu

doi:10.1109/icse.2019.00024

Cited by 75 publications

(44 citation statements)

References 63 publications

(107 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…LEOPARD [15] keeps queue culling yet add an additional step, prioritizing the selected inputs from queue culling by a function-level coverage metrics, rather than choosing randomly in AFL. The approach is able to cover all visited edge in each fuzzing loop, but it requires to preprocess the targeting programs for function complexity analysis and thus brings performance overhead.…”

Section: ) Advanced Input Prioritization Approachesmentioning

confidence: 99%

“…Current anti-fuzzing techniques make coverage-guided fuzzers much less effective in vulnerability discovery, causing 85%+ performance decrease in exploring paths. Unfortunately, many of the presented edge-coverage-based fuzzers [12,15,43,53,60] suffer from the current anti-fuzzing techniques. VUzzer is affected due to the use of concolic execution.…”

Section: Anti-fuzzing Techniquesmentioning

confidence: 99%

“…Another way to select vulnerable functions is code analysis and there is a recent relevant paper LEOPARD [15] which is according to the complexity score and vulnerability score of functions. The score is calculated based on several code features including loops and memory accesses, which is more like the combination of all three coverage accounting metrics we propose.…”

Section: ) Loopsmentioning

confidence: 99%

“…This may result in a slow increase of samples in Queue and fewer samples with the favor attributes. To solve this problem, TortoiseFuzz uses the original coverage-sensitive top rated entries TopCov to re-cull the Queue while there is no favor sample in the Queue (line [15][16][17][18][19][20][21][22][23][24]. Also, whenever the TopAccounting is changed (line 5), TortoiseFuzz will switch back to the security-sensitive strategy.…”

Section: ) Updating Top Rated Candidatesmentioning

confidence: 99%

“…AFL-Sensitive [53] and Angora [12] add more metrics complementary to edges, but edges are still considered equally, so the issue still exists for the inputs with the same value in the complementary metrics. LEOPARD [15] considers function coverage instead of edges and it weights functions differently, but it requires static analysis to preprocess, which causes extra performance overhead. Even worse, these approaches are all vulnerable to anti-fuzzing techniques [23,28] (See II-D).…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Not All Coverage Measurements Are Equal: Fuzzing by Coverage Accounting for Input Prioritization

Wang¹,

Jia²,

Liu³

et al. 2020

Proceedings 2020 Network and Distributed System Security Symposium

View full text Add to dashboard Cite

Coverage-based fuzzing has been actively studied and widely adopted for finding vulnerabilities in real-world software applications. With coverage information, such as statement coverage and transition coverage, as the guidance of input mutation, coverage-based fuzzing can generate inputs that cover more code and thus find more vulnerabilities without prerequisite information such as input format. Current coveragebased fuzzing tools treat covered code equally. All inputs that contribute to new statements or transitions are kept for future mutation no matter what the statements or transitions are and how much they impact security. Although this design is reasonable from the perspective of software testing that aims at full code coverage, it is inefficient for vulnerability discovery since that 1) current techniques are still inadequate to reach full coverage within a reasonable amount of time, and that 2) we always want to discover vulnerabilities early so that it can be fixed promptly. Even worse, due to the non-discriminative code coverage treatment, current fuzzing tools suffer from recent anti-fuzzing techniques and become much less effective in finding vulnerabilities from programs enabled with anti-fuzzing schemes. To address the limitation caused by equal coverage, we propose coverage accounting, a novel approach that evaluates coverage by security impacts. Coverage accounting attributes edges by three metrics based on three different levels: function, loop and basic block. Based on the proposed metrics, we design a new scheme to prioritize fuzzing inputs and develop TortoiseFuzz, a greybox fuzzer for finding memory corruption vulnerabilities. We evaluated TortoiseFuzz on 30 real-world applications and compared it with 6 state-of-the-art greybox and hybrid fuzzers: AFL, AFLFast, FairFuzz, MOPT, QSYM, and Angora. Statistically, TortoiseFuzz found more vulnerabilities than 5 out of 6 fuzzers (AFL, AFLFast, FairFuzz, MOPT, and Angora), and it had a comparable result to QSYM yet only consumed around 2% of QSYM's memory usage on average. We also compared coverage accounting metrics with two other metrics, AFL-Sensitive and LEOPARD, and TortoiseFuzz performed significantly better than both metrics in finding vulnerabilities. Furthermore, we applied the coverage accounting metrics to QSYM and noticed that coverage accounting helps increase the number of discovered vulnerabilities by 28.6% on average. TortoiseFuzz found 20 zero-day vulnerabilities with 15 confirmed with CVE identifications.

show abstract

Section: ) Advanced Input Prioritization Approachesmentioning

confidence: 99%

Section: Anti-fuzzing Techniquesmentioning

confidence: 99%

Section: ) Loopsmentioning

confidence: 99%

Section: ) Updating Top Rated Candidatesmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Not All Coverage Measurements Are Equal: Fuzzing by Coverage Accounting for Input Prioritization

Wang¹,

Jia²,

Liu³

et al. 2020

Proceedings 2020 Network and Distributed System Security Symposium

View full text Add to dashboard Cite

show abstract

A catalog of metrics at source code level for vulnerability prediction: A systematic mapping study

Codabux,

Zakia Sultana,

Chowdhury

2023

J Software Evolu Process

View full text Add to dashboard Cite

Industry practitioners assess software from a security perspective to reduce the risks of deploying vulnerable software. Besides following security best practice guidelines during the software development life cycle, predicting vulnerability before roll‐out is crucial. Software metrics are popular inputs for vulnerability prediction models. The objective of this study is to provide a comprehensive review of the source code‐level security metrics presented in the literature. Our systematic mapping study started with 1451 studies obtained by searching the four digital libraries from ACM, IEEE, ScienceDirect, and Springer. After applying our inclusion/exclusion criteria as well as the snowballing technique, we narrowed down 28 studies for an in‐depth study to answer four research questions pertaining to our goal. We extracted a total of 685 code‐level metrics. For each study, we identified the empirical methods, quality measures, types of vulnerabilities of the prediction models, and shortcomings of the work. We found that standard machine learning models, such as decision trees, regressions, and random forests, are most frequently used for vulnerability prediction. The most common quality measures are precision, recall, accuracy, and ‐measure. Based on our findings, we conclude that the list of software metrics for measuring code‐level security is not universal or generic yet. Nonetheless, the results of our study can be used as a starting point for future studies aiming at improving existing security prediction models and a catalog of metrics for vulnerability prediction for software practitioners.

show abstract

The progress, challenges, and perspectives of directed greybox fuzzing

Wang,

Zhou,

Yue

et al. 2023

Software Testing Verif & Rel

View full text Add to dashboard Cite

SummaryGreybox fuzzing is a scalable and practical approach for software testing. Most greybox fuzzing tools are coverage‐guided as reaching high code coverage is more likely to find bugs. However, since most covered codes may not contain bugs, blindly extending code coverage is less efficient, especially for corner cases. Unlike coverage‐guided greybox fuzzing which increases code coverage in an undirected manner, directed greybox fuzzing (DGF) spends most of its time allocation on reaching specific targets (e.g. the bug‐prone zone) without wasting resources stressing unrelated parts. Thus, DGF is particularly suitable for scenarios such as patch testing, bug reproduction, and special bug detection. For now, DGF has become an active research area. However, DGF has general limitations and challenges that are worth further studying. Based on the investigation of 42 state‐of‐the‐art fuzzers that are closely related to DGF, we conducted the first in‐depth study to summarize the empirical evidence on the research progress of DGF. This paper studies DGF from a broader view, which takes into account not only the location‐directed type that targets specific code parts but also the behavior‐directed type that aims to expose abnormal program behaviors. By analyzing the benefits and limitations of DGF research, we try to identify gaps in current research, meanwhile, reveal new research opportunities and suggest areas for further investigation.

show abstract

LEOPARD: Identifying Vulnerable Code for Vulnerability Assessment Through Program Metrics

Cited by 75 publications

References 63 publications

Not All Coverage Measurements Are Equal: Fuzzing by Coverage Accounting for Input Prioritization

Not All Coverage Measurements Are Equal: Fuzzing by Coverage Accounting for Input Prioritization

A catalog of metrics at source code level for vulnerability prediction: A systematic mapping study

The progress, challenges, and perspectives of directed greybox fuzzing

Contact Info

Product

Resources

About