Unsupervised software repositories mining and its application to code search

Abstract

Software repositories are crucial resources for many software tasks, including code retrieval and annotation. Programming forums provide questions and answers (Q&A) from software developers, containing abundant code-description posts for exchanging knowledge about programming issues. However, most posts provide personal opinions of users that are often not adequately confirmed or outdated. Mining software repositories in such open and unrestricted forums is challenging. Since the posts can be arbitrary and noisy, it is difficult to get unified labels for supervised noise elimination. Different from existing mining approaches, this paper proposes Code-Description Mining Framework (CodeMF), an unsupervised framework to eliminate noisy posts and extract high quality software repositories from programming forums.CodeMF treats all social features of the posts as discrete-time signals for kernel principal component analysis and further performs wavelet transform feature fusion to find the delicate changes (noises in temporal signals). We conduct comprehensive experiments on StackOverflow. Experimental results demonstrate that CodeMF can effectively reduce running time and improve precision via mining high-quality software repositories for various programming languages, especially for the large-scale codebases. To further illustrate the effect of CodeMF applied in software tasks, we introduce it to improve the performance of query-expansion code search. Meanwhile, for SQL and C# programs, compared to the state-of-the-art query-expansion method QECK, the improvement of QECK CodeMF is 2% and 6% on Recall@10, and 4% and 14% on mean reciprocal rank, respectively.

Opportunities and Challenges in Code Search Tools

“…However, the major challenge is the semantic gap between query and code. Code is not like a search query and code is written in a highly structured programming language with different syntactic rules and semantic representation [42,83,150]. To address this issue, researchers have proposed many query processing techniques to align this semantic gap, such as replacing query words with appropriate synonyms that occur in the codebase [134]; and expanding query words with code changes (e.g., pull requests and commits) in the development history [44-47, 52, 52, 150].…”

“…It has been observed that APIs are an important factor to complement the missing semantics in queries [8]. Researchers have thus expanded query words with relevant APIs or class names from official API documents [83], codebases [140], or Stack Overflow posts [11,42,95,118,143].…”

“…7(b) shows that nearly 60% of studies tested their proposed tools by using no more than 100 queries. Such query scale can only cover a limited number of real-world queries actually used by software developers [42,43,118]. The main reason that hinders the query scale is the tool evaluation method.…”

“…2) Query Representativeness. Commonly, code search studies selected the top-𝑛 frequently used queries from Q&A forums, or randomly sample a small set of queries from the real world [42,43,118]. However, studies seldom investigated the representativeness of the selected queries.…”

“…Due to the above difficulties, most code search studies chose to identify relevancy with human efforts. However, their evaluation with manual identification is subject to potential serious bias and human errors [37,42,73,81,128].…”