Vector space analysis of software clones

Abstract

In this paper, we introduce a technique for applying Independent Component Analysis to vector space representations of software code fragments such as methods or blocks. The distance between these points can be determined, and used as a measure of the similarity between the original source code fragments they represent. It can be reasoned that if the initial matrix representation contains enough information about the syntactic structure of the source code, the vector space representation will be sufficient to predict the similarity of fragments to one another, and can provide the likelihood that the code is a clone.

Using heuristics to estimate an appropriate number of latent topics in source code analysis

“…a clear relationship between proximity in the package structure and the likelihood of finding clones [33]. Using this, together with our observation that clones often share similar semantic information and are frequently identified as semantically related in latent models [30], we believe that the proximity score is a reasonable measure.…”

“…In our earlier research using information retrieval methods to locate clones [30], the results suggested a correlation between clones and latent topics. While it is not true that all clones are conceptually related, they are more likely to be conceptually related than not.…”

“…We experimented with Independent Component Analysis [27] as a way to identify latent structure in source code, and attempted to show how it could be used as a way to identify related blocks of information [28,29]. We also used latent models to identify clones, and showed that some variance in the syntactic data can be resolved through the latent semantic information identified by these models [30]. Although these results have been promising, we remained curious about how to choose how many latent factors are necessary to best fit the data.…”

Detecting High Level Similarities in Source Code and Beyond

“…Grant and Cordy [17] introduced a technique based on independent component analysis to analyze vector space representations of methods in software systems.…”

“…Local weights range in complexity from simple binary values (0 or 1) to functions involving logarithms of term frequencies. Global weighting schemes range from simple normalizations to advanced statistics-based approaches [3,17].…”