SourcererCC

Abstract

Despite a decade of active research, there is a marked lack in clone detectors that scale to very large repositories of source code, in particular for detecting near-miss clones where significant editing activities may take place in the cloned code. We present SourcererCC, a token-based clone detector that targets three clone types, and exploits an index to achieve scalability to large inter-project repositories using a standard workstation. SourcererCC uses an optimized invertedindex to quickly query the potential clones of a given code block. Filtering heuristics based on token ordering are used to significantly reduce the size of the index, the number of code-block comparisons needed to detect the clones, as well as the number of required token-comparisons needed to judge a potential clone.We evaluate the scalability, execution time, recall and precision of SourcererCC, and compare it to four publicly available and state-of-the-art tools. To measure recall, we use two recent benchmarks, (1) a large benchmark of real clones, BigCloneBench, and (2) a Mutation/Injection-based framework of thousands of fine-grained artificial clones. We find SourcererCC has both high recall and precision, and is able to scale to a large inter-project repository (250MLOC) using a standard workstation.

Aroma: code recommendation via structural code search

“…We tested these alternative setups for search recall on the same micro-benchmark dataset. [Sajnani et al 2016] is a state-of-the-art clone detector that supports Type-3 clone detection. We wanted to compare Aroma with SourcererCC to examine whether a current-generation clone detector can be used as the light-weight search phase in Aroma.…”

“…Clone detectors are designed to detect syntactically identical or highly similar code. SourcererCC [Sajnani et al 2016] is a token-based clone detector targeting Type 1, 2, and 3 clones. Compared with other clone detectors that also support Type 3 clones, including NiCad [Cordy and Roy 2011], Deckard [Jiang et al 2007], and CCFinder , SourcererCC has high precision and recall and also scales to large-size projects.…”

“…Aroma's recommendations contain more syntactic variety than just API usages; for instance, the recommended code snippet in Listing 3 includes a try-catch block, and Example B in Table 1 recommends adding an if statement that modifies a variable. Code clone detectors [Cordy and Roy 2011;Jiang et al 2007;Sajnani et al 2016] are another set of techniques that could potentially be used to retrieve recommended code snippets. However, code clone detection tools usually retrieve code snippets that are almost identical to a query snippet.…”

The Android update problem

“…They also cannot detect changes within method bodies. Our work combines several existing tools and techniques, such as ChangeDistiller [22], SourcerCC [35], and Refactor-ingMiner [36] to get a comprehensive set of method-level changes.…”

“…Since we do not currently have access to proprietary vendorspecific code, we use a popular community-based variant of Android, LineageOS, as a proxy for a vendor-based version of Android. For each subsystem in AOSP, we track the method-level changes in the source code using a combination of existing code-evolution analysis tools: SourcererCC [35], ChangeDistiller [22], and Refac-toringMiner [36]. We are interested in changes in two directions.…”