Spotting working code examples

Keivanloo, Iman; Rilling, Juergen; Zou, Ying

doi:10.1145/2568225.2568292

Cited by 128 publications

(108 citation statements)

References 37 publications

Supporting

Mentioning

106

Contrasting

Order By: Relevance

“…Related work in this field include experiments to evaluate diverse approaches to code search [17]; techniques to improve the searchability of source code (e.g., [9,10]); techniques for extracting information from diverse sources (e.g., [14,19]); and interactive techniques (e.g., [1,21]). …”

Section: Related Workmentioning

confidence: 99%

Automatically Generating Test Templates from Test Names (N)

Zhang

Hill

Clause

2015

2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE)

View full text Add to dashboard Cite

Existing specification-based testing techniques require specifications that either do not exist or are too difficult to create. As a result, they often fall short of their goal of helping developers test expected behaviors. In this paper we present a novel, natural language-based approach that exploits the descriptive nature of test names to generate test templates. Similar to how modern IDEs simplify development by providing templates for common constructs such as loops, test templates can save time and lower the cognitive barrier for writing tests. The results of our evaluation show that the approach is feasible: despite the difficulty of the task, when test names contain a sufficient amount of information, the approach's accuracy is over 80% when parsing the relevant information from the test name and generating the template.

show abstract

Section: Related Workmentioning

confidence: 99%

Automatically Generating Test Templates from Test Names (N)

Zhang

Hill

Clause

2015

2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE)

View full text Add to dashboard Cite

show abstract

“…One may not feasibly try all returned repair candidates but only the top-ranked ones. Another example is searching for working code examples (Keivanloo et al 2014) when one wants to pick only the top ranked solution.…”

Section: Scenario 4 (Ranked Results)mentioning

confidence: 99%

A comparison of code similarity analysers

2017

View full text Add to dashboard Cite

Copying and pasting of source code is a common activity in software engineering. Often, the code is not copied as it is and it may be modified for various purposes; e.g. refactoring, bug fixing, or even software plagiarism. These code modifications could affect the performance of code similarity analysers including code clone and plagiarism detectors to some certain degree. We are interested in two types of code modification in this study: pervasive modifications, i.e. transformations that may have a global effect, and local modifications, i.e. code changes that are contained in a single method or code block. We evaluate 30 code similarity detection techniques and tools using five experimental scenarios for Java source code. These are (1) pervasively modified code, created with tools for source code and bytecode obfuscation, and boiler-plate code, (2) source code normalisation through compilation and decompilation using different decompilers, (3) reuse of optimal configurations over different data sets, (4) tool evaluation using ranked-based measures, and (5) local + global code modifications. Our experimental results show that in the presence of pervasive modifications, some of the general textual similarity measures can offer similar performance to specialised code similarity tools, whilst in the presence of boiler-plate code, highly specialised source code similarity detection techniques and tools outperform textual similarity measures. Our study strongly validates the use of compilation/decompilation as a normalisation technique. Its use reduced false classifications to zero for three of the tools. Moreover, we demonstrate that optimal configurations are very sensitive to a specific data set. After directly applying optimal configurations derived from one data set to another, the tools perform poorly on the new data set. The code similarity analysers are thoroughly evaluated not only based on several well-known pair-based and query-based error measures but also on each specific type of pervasive code modification. This broad, thorough study is the largest in existence and potentially an invaluable guide for future users of similarity detection in source code.

show abstract

“…However, techniques in code search most commonly use an indirect measure to find code of higher quality, for instance through popularity metrics. Popularity itself has many meanings, but it has been measured by counting the number of times code is used by other code (similar to PageRank [85] once used by Google) [67], [76], [123], and it has also been measured by extracting high level patterns from the code indexed and counting how 47 often those patterns occur in the search engine [31], [60], [77]. Another approach proposes to use votes from other developers [43].…”

Section: Code Results Qualitymentioning

confidence: 99%