Impact of code complexity on software analysis

Oliveira, Charles Daniel De; Fong, Elizabeth; Black, Paul E.

doi:10.6028/nist.ir.8165

Cited by 1 publication

(1 citation statement)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The Intelligence Advanced Research Projects Activity (IARPA) attempted to combine all three properties of an ideal test suite in its Securely Taking On New Executable Software of Uncertain Provenance STONESOUP program [25,26]. IARPA created 7770 test cases by injecting small code snippets, containing weaknesses, into sixteen open source base programs.…”

Section: Related Workmentioning

confidence: 99%

SATE V report: ten years of static analysis tool expositions

Delaitre¹,

Stivalet²,

Black³

et al. 2018

Self Cite

View full text Add to dashboard Cite

Software assurance has been the focus of the National Institute of Standards and Technology (NIST) Software Assurance Metrics and Tool Evaluation (SAMATE) team for many years. The Static Analysis Tool Exposition (SATE) is one of the team's prominent projects to advance research in and adoption of static analysis, one of several software assurance methods. This report describes our approach and methodology. It then presents and discusses the results collected from the fifth edition of SATE. Overall, the goal of SATE was not to rank static analysis tools, but rather to propose a methodology to assess tool effectiveness. Others can use this methodology to determine which tools fit their requirements. The results in this report are presented as examples and used as a basis for further discussion. Our methodology relies on metrics, such as recall and precision, to determine tool effectiveness. To calculate these metrics, we designed test cases that exhibit certain characteristics. Most of the test cases were large pieces of software with cybersecurity implications. Fourteen participants ran their tools on these test cases and sent us a report of their findings. We analyzed these reports and calculated the metrics to assess the tools' effectiveness. Although a few results remained inconclusive, many key elements could be inferred based on our methodology, test cases, and analysis. In particular, we were able to estimate the propensity of tools to find critical vulnerabilities in real software, the degree of noise they produced, and the type of weaknesses they were able to find. Some shortcomings in the methodology and test cases were also identified and solutions proposed for the next edition of SATE.

show abstract

Section: Related Workmentioning

confidence: 99%