How should we measure functional sameness from program source code? an exploratory study on Java methods

Abstract

Program source code is one of the main targets of software engineering research. A wide variety of research has been conducted on source code, and many studies have leveraged structural, vocabulary, and method signature similarities to measure the functional sameness of source code. In this research, we conducted an empirical study to ascertain how we should use three similarities to measure functional sameness. We used two large datasets and measured the three similarities between all the method pairs in the datasets, each of which included approximately 15 million Java method pairs. The relationships between the three similarities were analyzed to determine how we should use each to detect functionally similar code. The results of our study revealed the following.(1) Method names are not always useful for detecting functionally similar code. Only if there are a small number of methods having a given name, the methods are likely to include functionally similar code.(2) Existing file-level, method-level, and block-level clone detection techniques often miss functionally similar code generated by copy-and-paste operations between different projects. (3) In the cases we use structural similarity for detecting functionally similar code, we obtained many false positives. However, we can avoid detecting most false positives by using a vocabulary similarity in addition to a structural one. (4) Using a vocabulary similarity to detect functionally similar code is not suitable for method pairs in the same file because such method pairs use many of the same program elements such as private methods or private fields.

Replacements and Replaceables: Making the Case for Code Variants

“…There are a variety of approaches to compute structural similarity of source code [49,50,51]. We used the de-duplication approach explained in [52] for its simplicity and effectiveness for method-level similarity computation.…”

An Exploratory Study of Interface Redundancy in Code Repositories

“…To cope with this problem, they have randomly selected 100 Java projects from Source-Forge 5 in what was called a first attempt to run a statistically sound experiment in test data generation. According to them, the resulting benchmark, called SF100, is statistically sound and representative of open source projects 6 . This issue also affects the evaluation of code search techniques: many times the target repositories are not a random sample of software projects, but a specific set of projects, which can introduce bias to the study.…”

“…Although such a property has received a lot of attention in the past, recently researchers have been looking at other types of replication, such as vocabulary or temporal redundancy. The idea is that sometimes two fragments of code can be similar with respect to other aspects besides text (think about different implementations of sorting algorithms, for instance, which can have the same function but different structure [6]). Vocabulary redundancy appears when different pieces of code share similar words [6], while temporal redundancy is concerned with the amount of code commits that are composed of previous commits [7].…”

DSpot: Test Amplification for Automatic Assessment of Computational Diversity

“…Beyond this (of course fuzzy) threshold, the diversity and uniqueness of source code appears. Higo and Kusumoto [11] investigate the interplay between structural similarity, vocabulary similarity and method name similarity, to assess functional similarity between methods in Java programs. They show that many contextual factors influence the ability of these similarity measures to spot functional similarity (e.g., the number of methods that share the same name, or the fact that two methods with similar structure are in the same class or not).…”