How Well Do Search Engines Support Code Retrieval on the Web?

Abstract

Software developers search the Web for various kinds of source code for diverse reasons. In a previous study, we found that searches varied along two dimensions: the size of the search target (e.g., block, subsystem, or system) and the motivation for the search (e.g., reference example or as-is reuse). Would each of these kinds of searches require different search technologies? To answer this question, we conducted an experiment with 36 participants to evaluate three diverse approaches (general purpose information retrieval, source code search, and component reuse), as represented by five Web sites (Google, Koders, Krugle, Google Code Search, and SourceForge). The independent variables were search engine, size of search target, and motivation for search. The dependent variable was the participants judgement of the relevance of the first ten hits. We found that it was easier to find reference examples than components for as-is reuse and that participants obtained the best results using a general-purpose information retrieval site. However, we also found an interaction effect: code-specific search engines worked better in searches for subsystems, but Google worked better on searches for blocks. These results can be used to guide the creation of new tools for retrieving source code from the Web.

Understanding the impact of support for iteration on code search

“…We stopped mining a little over 10M classes because of a noticeable delay in processing queries on our servers (unacceptable when users expect queries to be processed in a few seconds). However, we also thought 10M classes allowed us to closely approximate an Internet scale code search engine when compared against the sizes of the Internet code search engines Koders (600K), Krugle (3.5M), and Google Code Search (2.5M) [108].…”

“…Given the advantages and disadvantages of search engines today and the importance of search in developing software (programmers report searching for code frequently as part of their practice [97], [108]), software engineering researchers are investigating how to improve code search engines. Some, for instance, have been investigating how to support more expressive queries (e.g., searching by test case or method signatures) that afford more precise matching of code compared to keywords (e.g., [4], [15], [54], [65], [71], [82], [91], [110], [122], [142]).…”

Solving Semantic Searches for Source Code

“…Recent studies have revealed that programmers typically use general search engines to find code for reuse [21]. More specialized code search engines (e.g., Koders, Krugle, ohloh) incorporate various filtering capabilities (e.g., language, domain, scores) and program syntax into the query to better guide the matching process [21]. Other approaches add natural language processing to increase the potential matches [8], [16].…”

“…INTRODUCTION Today, searching for code is a regular activity for most programmers [21]. Yet, the mechanisms to support this activity have barely evolved in the last decade, and the limitations are becoming more apparent as code repositories get richer and programmers' expertise and needs more diverse.…”

“…The programmer turns to Google (like many others [21]) and issues a search query with the following keywords: extract alias from e-mail address in Java. As expected, the query returns millions of results.…”