A Systematic Evaluation of Static API-Misuse Detectors

Abstract

Application Programming Interfaces (APIs) often have usage constraints, such as restrictions on call order or call conditions. API misuses, i.e., violations of these constraints, may lead to software crashes, bugs, and vulnerabilities. Though researchers developed many API-misuse detectors over the last two decades, recent studies show that API misuses are still prevalent. Therefore, we need to understand the capabilities and limitations of existing detectors in order to advance the state of the art. In this paper, we present the first-ever qualitative and quantitative evaluation that compares static API-misuse detectors along the same dimensions, and with original author validation. To accomplish this, we develop MUC, a classification of API misuses, and MUBENCHPIPE, an automated benchmark for detector comparison, on top of our misuse dataset, MUBENCH. Our results show that the capabilities of existing detectors vary greatly and that existing detectors, though capable of detecting misuses, suffer from extremely low precision and recall. A systematic root-cause analysis reveals that, most importantly, detectors need to go beyond the naive assumption that a deviation from the most-frequent usage corresponds to a misuse and need to obtain additional usage examples to train their models. We present possible directions towards more-powerful API-misuse detectors.

Beyond Unit-Testing in Search-Based Test Case Generation: Challenges and Opportunities

“…An interesting example is represented by testing Application Programming Interface (API) uses. APIs misuses [4] are commons as clients may invoke the APIs violating its implicit preconditions. Interesting paths then are those where the client class invokes the APIs without verifying whether the data passed to the APIs satisfy or not its implicit precondition.…”

Learning to Spot and Refactor Inconsistent Method Names

“…Method names are the intuitive and vital information for developers to understand the behavior of programs or APIs [8]- [11]. Therefore, inconsistent method names can make programs harder to understand and maintain [12]- [18], and may even lead to software defects [19]- [22]. Poor method names are indeed prone to be defective.…”

Effective and efficient API misuse detection via exception propagation and search-based testing

“…The fact that apis are opaque to developers is known to lead to incorrect uses (or api misuses [3,4]) since client applications can violate the (implicit) usage constraints (often referred to as contract) of those apis. For example, a violation occurs when a client application calls a method that expects a non-null formal parameter without validating (i.e., via null checks or error handling) the references used as arguments.…”

“…For example, a violation occurs when a client application calls a method that expects a non-null formal parameter without validating (i.e., via null checks or error handling) the references used as arguments. A classification of these violations has been made by Amann et al [3,4]. api misuses can cause software reliability problems, originating from issues such as poor handling of user input and resource misuse [3], or even increasing the attack surface of client applications [17,38].…”