Evaluating Automatic Program Repair Capabilities to Repair API Misuses

Kechagia, Maria; Mechtaev, Sergey; Sarro, Federica; Harman, Mark

doi:10.1109/tse.2021.3067156

Cited by 22 publications

(11 citation statements)

References 85 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Moreover, most of our evaluation considers only a small set of 37 real API misuses from MUBench. While specifically preprocessed to reduce potential bias (e.g., removing duplicates), future work may validate our results on larger datasets of API misuses such similar to ours on the AU500 dataset (Kang and Lo 2021) as well as recently published ones (Nielebock et al 2021;Kechagia et al 2021).…”

Section: Threats To Validitysupporting

confidence: 62%

See 1 more Smart Citation

Guided pattern mining for API misuse detection by change-based code analysis

et al. 2021

View full text Add to dashboard Cite

Lack of experience, inadequate documentation, and sub-optimal API design frequently cause developers to make mistakes when re-using third-party implementations. Such API misuses can result in unintended behavior, performance losses, or software crashes. Therefore, current research aims to automatically detect such misuses by comparing the way a developer used an API to previously inferred patterns of the correct API usage. While research has made significant progress, these techniques have not yet been adopted in practice. In part, this is due to the lack of a process capable of seamlessly integrating with software development processes. Particularly, existing approaches do not consider how to collect relevant source code samples from which to infer patterns. In fact, an inadequate collection can cause API usage pattern miners to infer irrelevant patterns which leads to false alarms instead of finding true API misuses. In this paper, we target this problem (a) by providing a method that increases the likelihood of finding relevant and true-positive patterns concerning a given set of code changes and agnostic to a concrete static, intra-procedural mining technique and (b) by introducing a concept for just-in-time API misuse detection which analyzes changes at the time of commit. Particularly, we introduce different, lightweight code search and filtering strategies and evaluate them on two real-world API misuse datasets to determine their usefulness in finding relevant intra-procedural API usage patterns. Our main results are (1) commit-based search with subsequent filtering effectively decreases the amount of code to be analyzed, (2) in particular method-level filtering is superior to file-level filtering, (3) project-internal and project-external code search find solutions for different types of misuses and thus are complementary, (4) incorporating prior knowledge of the misused API into the search has a negligible effect.

show abstract

Section: Threats To Validitysupporting

confidence: 62%

“…As it has been shown by previous work, other metrics could further improve the results (Le and Lo 2015;Amann 2018). Additionally, recent research has come up with potential further datasets for API misuses (Nielebock et al 2021;Kechagia et al 2021), which provide a more diverse set of validation data.…”

Section: Misusementioning

confidence: 99%

Guided pattern mining for API misuse detection by change-based code analysis

et al. 2021

View full text Add to dashboard Cite

show abstract

“…Our work aligns with different comparative studies on techniques in the automated software-engineering domain, such as automated program repair in general [17], automated repair of API misuses [27], static code analysis for detecting security vulnerabilities [22], fault localization techniques [69], or the performance of API misuse detectors [4]. To the best of our knowledge, no previous work exists that evaluates graphdistance algorithms for API usages comparisons.…”

Section: Studies On Automated Software-engineering Techniquesmentioning

confidence: 57%

“…Finally, the MUBench dataset may not be a representative set of API misuses, which could cause their limited applicability to detect misuses in the AU500 dataset. Thus, other datasets should be researched, such as the ones provided by Kechagia et al [27] or ourselves [46].…”

Section: B External Validitymentioning

confidence: 99%

Replication Package for "An Experimental Analysis of Graph-Distance Algorithms for Comparing API Usages"

Nielebock,

Blockhaus,

Krüger

et al. 2021

View full text Add to dashboard Cite

Modern software development heavily relies on the reuse of functionalities through Application Programming Interfaces (APIs). However, client developers can have issues identifying the correct usage of a certain API, causing misuses accompanied by software crashes or usability bugs. Therefore, researchers have aimed at identifying API misuses automatically by comparing client code usages to correct API usages. Some techniques rely on certain API-specific graph-based data structures to improve the abstract representation of API usages. Such techniques need to compare graphs, for instance, by computing distance metrics based on the minimal graph edit distance or the largest common subgraphs, whose computations are known to be NP-hard problems. Fortunately, there exist many abstractions for simplifying graph distance computation. However, their applicability for comparing graph representations of API usages has not been analyzed. In this paper, we provide a comparison of different distance algorithms of API-usage graphs regarding correctness and runtime. Particularly, correctness relates to the algorithms' ability to identify similar correct API usages, but also to discriminate similar correct and false usages as well as non-similar usages. For this purpose, we systematically identified a set of eight graph-based distance algorithms and applied them on two datasets of real-world API usages and misuses. Interestingly, our results suggest that existing distance algorithms are not reliable for comparing API usage graphs. To improve on this situation, we identified and discuss the algorithms' issues, based on which we formulate hypotheses to initiate research on overcoming them.

show abstract

“…We found 3 datasets that met our criteria: 1) DDR by He et al [8], which contains patches from 14 repair systems. He et al [8] classified patches using a technique called RGT, which generates new test cases using a groundtruth, human-written oracle patches; 2) one dataset by Liu et al [22], which includes patches from 16 repair systems, and manually evaluated the correctness using guidelines presented by Liu et al [22]; 3) APIRepBench by Kechagia et al [23], which includes patches from 14 repair tools. The patches were manually assessed.…”

Section: Datasetmentioning

confidence: 99%

Test-based Patch Clustering for Automatically-Generated Patches Assessment

Martínez¹,

Kechagia²,

Perera³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Previous studies have shown that Automated Program Repair (apr) techniques suffer from the overfitting problem. Overfitting happens when a patch is run and the test suite does not reveal any error, but the patch actually does not fix the underlying bug or it introduces a new defect that is not covered by the test suite. Therefore, the patches generated by apr tools need to be validated by human programmers, which can be very costly, and prevents apr tools adoption in practice. Our work aims at increasing developer trust in automated patch generation by minimizing the number of plausible patches that they have to review, thereby reducing the time required to find a correct patch. We introduce a novel light-weight test-based patch clustering approach called xTestCluster, which clusters patches based on their dynamic behavior. xTestCluster is applied after the patch generation phase in order to analyze the generated patches from one or more repair tools and to provide

show abstract

Evaluating Automatic Program Repair Capabilities to Repair API Misuses

Cited by 22 publications

References 85 publications

Guided pattern mining for API misuse detection by change-based code analysis

Guided pattern mining for API misuse detection by change-based code analysis

Replication Package for "An Experimental Analysis of Graph-Distance Algorithms for Comparing API Usages"

Test-based Patch Clustering for Automatically-Generated Patches Assessment

Contact Info

Product

Resources

About