A trustability metric for code search based on developer karma

Abstract

The promise of search-driven development is that developers will save time and resources by reusing external code in their local projects. To efficiently integrate this code, users must be able to trust it, thus trustability of code search results is just as important as their relevance. In this paper, we introduce a trustability metric to help users assess the quality of code search results and therefore ease the cost-benefit analysis they undertake trying to find suitable integration candidates. The proposed trustability metric incorporates both user votes and cross-project activity of developers to calculate a "karma" value for each developer. Through the karma value of all its developers a project is ranked on a trustability scale. We present JBender , a proof-of-concept code search engine which implements our trustability metric and we discuss preliminary results from an evaluation of the prototype.

Harnessing Stack Overflow for the IDE

“…This is mainly achieved through three of the specific design decisions that guided the creation of the service: voting, tagging, and editing. Voting addresses the critical issue that arises when reusing code samples found on the web: Before integrating a search result the developer has to assess its trustability to take a decision [4]. In the case of search engine results, developers have to take a decision that can only be based on their knowledge and experience.…”

“…This complete data is then given as input to an Apache Solr 4 instance, which is the basis of our recommendation engine. 4 …”

Understanding the impact of support for iteration on code search

“…Popularity itself has many meanings, but it has been measured by counting the number of times code is used by other code (similar to PageRank [85] once used by Google) [67], [76], [123], and it has also been measured by extracting high level patterns from the code indexed and counting how 47 often those patterns occur in the search engine [31], [60], [77]. Another approach proposes to use votes from other developers [43]. These popularity scores are then used in the final ranking algorithm to organize results differently than just in order of being on topic.…”

Towards a Marketplace of Open Source Software Data

“…Trustability for code search is presented in [27]. They specify a metric for trust of source code that incorporates the trustworthiness of persons (e.g., of the developer) that are associated with a program code.…”