Source File Set Search for Clone-and-Own Reuse Analysis

Abstract

Clone-and-own approach is a natural way of source code reuse for software developers. To assess how known bugs and security vulnerabilities of a cloned component affect an application, developers and security analysts need to identify an original version of the component and understand how the cloned component is different from the original one. Although developers may record the original version information in a version control system and/or directory names, such information is often either unavailable or incomplete. In this research, we propose a code search method that takes as input a set of source files and extracts all the components including similar files from a software ecosystem (i.e., a collection of existing versions of software packages). Our method employs an efficient file similarity computation using b-bit minwise hashing technique. We use an aggregated file similarity for ranking components. To evaluate the effectiveness of this tool, we analyzed 75 cloned components in Firefox and Android source code. The tool took about two hours to report the original components from 10 million files in Debian GNU/Linux packages. Recall of the topfive components in the extracted lists is 0.907, while recall of a baseline using SHA-1 file hash is 0.773, according to the ground truth recorded in the source code repositories.

Extraction of Library Update History Using Source Code Reuse Detection

“…where S (p, l) is an aggregated file similarity value of a product version p and a library version l [8]. In Reuse(P, L), each product version p ∈ P is corresponding to only one version of library l whose source files are the most similar to the product source code.…”

“…Copyright c 2018 The Institute of Electronics, Information and Communication Engineers th = 0.9 to ignore lower similarity values. The threshold accurately identifies source files in a product that are reused from a library [8].…”

“…Hence, an automatic analysis tool is necessary to track copied libraries in a project. To identify the original version of a copied library, Ishio et al [8] proposed a source file comparison method. The method uses aggregated file similarity to effectively identify an original version.…”

“…In this paper, we propose a method to extract and visualize a library update history to investigate when developers adopt, update or change their copied library in the ver- sion history of a product. To identify the original version of a library copy, we employed a modified version of the source file comparison method [8]; we compare source files between repositories. The identified library versions in each of product versions are visualized in a Sankey diagram [9].…”

Siamese: scalable and incremental code clone search via multiple code representations

“…The authors also present other variations of real-time clone search system using multi-level indexing (Keivanloo et al, 2011a,b), and abstract programming solutions . Ishio et al (2017) present a scalable approach for detecting clone-and-own software packages using b-bit minwise hashing technique. Then, an aggregated file similarity is applied to rank the returned search components.…”