Automated Classification of Static Code Analysis Alerts: A Case Study

Abstract

Static code analysis tools automatically generate alerts for potential software faults that can lead to failures. However, developers are usually exposed to a large number of alerts. Moreover, some of these alerts are subject to false positives and there is a lack of resources to inspect all the alerts manually. To address this problem, numerous approaches have been proposed for automatically ranking or classifying the alerts based on their likelihood of reporting a critical fault. One of the promising approaches is the application of machine learning techniques to classify alerts based on a set of artifact characteristics. In this work, we evaluate this approach in the context of an industrial case study to classify the alerts generated for a digital TV software. First, we created a benchmark based on this code base by manually analyzing thousands of alerts. Then, we evaluated 34 machine learning algorithms using 10 different artifact characteristics and identified characteristics that have a significant impact. We obtained promising results with respect to the precision of classification.

Identifying and Documenting False Positive Patterns Generated by Static Code Analysis Tools

“…Yuksel and Sozer [20] developed a binary classifier to distinguish between true and false warnings based on 10 attributes, including the warning severity, type of warning, number of warnings in the file, and length of time the warning has persisted through consecutive runs of the static code analysis tool. While the authors concluded their approach to be viable, their classifier still depended on the developer's initial perception of the error-whether it was an actual error or could be ignored.…”

Integrated static code analysis and runtime verification

“…They complement manual software reviews and testing activities to assist in the development of dependable software systems. A major disadvantage of these tools is the large amount of alerts (i.e., warnings, issues) being exposed to developers [8]. The density of alerts can be as much as 40 alerts per thousand lines of code (KLOC) [3].…”

“…The density of alerts can be as much as 40 alerts per thousand lines of code (KLOC) [3]. Around 3000 alerts can be quite common for large-scale industrial projects [8]. Moreover, these alerts are subject to false positives.…”

“…If we assume the inspection time to be 5 min per alert [3,11], a developer might need up to 250 h of time to inspect 3000 alerts. Indeed, it has been observed that the inspection of SCA tool alerts constitute a significant amount of time and effort during the software development lifecycle [8].…”

Extending Static Code Analysis with Application-Specific Rules by Analyzing Runtime Execution Traces

“…The detected violations are reported in the form of a list of alerts. Although SCAT have been successfully utilized in the industry [7,8,15], they have limitations as well. It is very hard or undecidable to show whether an execution path is feasible or infeasible without the runtime context information [11].…”