SOAR: A Synthesis Approach for Data Science API Refactoring

Abstract

With the growth of the open-source data science community, both the number of data science libraries and the number of versions for the same library are increasing rapidly. To match the evolving APIs from those libraries, open-source organizations often have to exert manual effort to refactor the APIs used in the code base. Moreover, due to the abundance of similar open-source libraries, data scientists working on a certain application may have an abundance of libraries to choose, maintain and migrate between. The manual refactoring between APIs is a tedious and error-prone task. Although recent research efforts were made on performing automatic API refactoring between different languages, previous work relies on statistical learning with collected pairwise training data for the API matching and migration. Using large statistical data for refactoring is not ideal because such training data will not be available for a new library or a new version of the same library. We introduce Synthesis for Open-Source API Refactoring (SOAR), a novel technique that requires no training data to achieve API migration and refactoring. SOAR relies only on the documentation that is readily available at the release of the library to learn API representations and mapping between libraries. Using program synthesis, SOAR automatically computes the correct configuration of arguments to the APIs and any glue code required to invoke those APIs. SOAR also uses the interpreter's error messages when running refactored code to generate logical constraints that can be used to prune the search space. Our empirical evaluation shows that SOAR can successfully refactor 80% of our benchmarks corresponding to deep learning models with up to 44 layers with an average run time of 97.23 seconds, and 90% of the data wrangling benchmarks with an average run time of 17.31 seconds. Index Terms-software maintenance, program translation, program synthesis I. In t r o d u c t i o nModem software development makes heavy use of libraries, frameworks, and associated application programming interfaces (APIs). Libraries provide modular functionality intended for reuse, with prescribing a particular architecture [1], and their widespread use has important productivity advantages [2]. The API for a library defines the interface, or *Both authors contributed equally to this work.

PyMigBench and PyMigTax: A Benchmark and Taxonomy for Python Library Migration

“…In this section, we use PyMigTax to identify the types of real-world migrations that existing library migration techniques support. We specifically discuss four state-ofthe-art API mapping techniques [4][5][6]42] and, to the best of our knowledge, the only existing client code transformation technique for library migration [37]. Note that the API mapping techniques [4][5][6]42] are for Java and the transformation technique [37] is for Python.…”

“…Other techniques use various metrics, based on functional and non-functional characteristics of libraries, to select or recommend such analogous libraries [14,19,24]. Similar approaches have also been used for API mapping, whether through mining previous migration instances or API descriptions to discover analogous APIs [4,6,9,37,42,52], or by building recommender systems around these API mappings [5]. For client code transformation, researchers used various techniques such as program synthesis [37], data flow analysis [32], patch generalization [20,51] and differential testing [20].…”

“…Finally, researchers have evaluated most of the previous version or library migration techniques by migrating code for a few arbitrarily selected library pairs. For example, Ni et al [37] evaluated their library migration technique for only two library pairs. Migration between different library pairs might need different types of code changes.…”

Bind the gap: compiling real software to hardware FFT accelerators

“…A large amount of work has been done on API migration [38,[97][98][99]105],the task of migrating code using one API to use a new API. Likewise, a number of API-recommendation tools [68,137] have been developed, although these do not tell the programmer how to integrate the API.…”

Example-based Synthesis of Static Analysis Rules

“…They do not automate the task of synthesizing a rule from a cluster of code changes. Program synthesis has also been recently applied to other code related applications such as API migration [Gao et al 2021;Ni et al 2021], synthesis of merge conflict resolutions [Pan et al 2021], interactive code search ]. Datalog synthesis: There is a rich body of recent work on datalog synthesis [Albarghouthi et al 2017;Si et al 2018Si et al , 2019Thakkar et al 2021] and its application to code related tasks such as interactive code search .…”