API Evolution and Compatibility: A Data Corpus and Tool Evaluation.

Abstract

The development of software components with independent release cycles is nowadays widely supported by multiple languages and frameworks. A critical feature of any such platform is to safeguard composition by ensuring backward compatibility of substituted components. In recent years, some tooling has been developed to help developers and DevOps engineers to establish whether components are backward compatible by means of static analysis. We investigate the state of the art in this space by benchmarking such tools for Java. For this purpose, we have developed a compact benchmark data set of less than 200KB. Using this dataset, we study possible API changes of Java libraries, and whether the tools investigated can detect them. We find that only a small number of tools suitable to analyse API evolution exist. Those tools are only infrequently maintained by small communities. All tools investigated have some shortcomings in that they fail to detect certain API incompatibilities.

BreakBot: Analyzing the Impact of Breaking Changes to Assist Library Evolution

“…Static analysis. Various tools can scan two versions of a library using static analysis to output the list of BCs between them [14]. This approach is more complete than regression testing (although false positives or negatives can arise), therefore it is unlikely that BCs will leak to clients.…”

“…To evaluate BreakBot, we plan to use both quantitative and qualitative evaluation methods. We will first investigate the accuracy of Maracas during the BCs and impact analysis phases using a proper benchmark such as the one introduced by Jezek and Dietrich [14]. Second, we aim at better understanding the current state of practice when dealing with change and the effects of BreakBot on these practices.…”

Model-based testing of breaking changes in Node.js libraries

“…The κ label in the actions is used to distinguish calls to the same function. 8 In an argument read action, κ →arg j , the label κ identifies the function call for which the argument is being read. The purpose of these modifications to the Path mechanism becomes clear when we explain the type regression testing phase in Section 5.…”

“…A few tools exist for helping developers detect breaking changes before an update is released to the clients. Examples include APIDiff, Clirr, and Revapi for Java [8], the elm diff tool 2 for elm, and NoRegrets [15] and dont-break 3 for JavaScript. A common property of these tools is that they compute the changes to the types of the public API of the library for a given update, and then identify the changes that may break clients.…”

“…Node.js libraries are loaded via the built-in require function, as shown in Section 1 8. Because of the introduction of the κ labels, NoRegrets+ does not need to track the number of arguments at calls as done by NoRegrets.…”