Search4Code: Code Search Intent Classification Using Weak Supervision

Rao, Nikitha; Bansal, Chetan; Guan, Joe

doi:10.1109/msr52588.2021.00077

Cited by 8 publications

(4 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Rao et al note that a significant bottleneck in this area of research is the lack of datasets as search logs can not be made public due to privacy laws (see also Section 5) [4]. In their later work [17], they release a limited dataset of anonymized software development-related search queries mined from Bing logs between September 1, 2019 and August 31, 2020. The dataset contains more than 11,000 real-world search queries related to the C# and Java programming languages, which they group into 7 categories based on their search intent: API, HowTo, Installation, Debug, Learn, Navigational and Miscellaneous.…”

Section: Software Development-related Queriesmentioning

confidence: 99%

“…In this work, we select the Java programming language-related queries released by Rao et al [17], which consists of 6,596 queries. Some of the queries in the dataset are similar (e.g., 'java api', and 'java apis', 'java queue', and 'java queues', 'java for loop', and 'for loop java' ), and some are noisy (e.g., 'java chicken', 'java apple' ).…”

Section: Software Development-related Queriesmentioning

confidence: 99%

See 1 more Smart Citation

Using clarification questions to improve software developers' Web search

Imran,

Damevski

2022

Preprint

View full text Add to dashboard Cite

ContextRecent research indicates that Web queries written by software developers are not very successful in retrieving relevant results, performing measurably worse compared to general purpose Web queries. Most approaches up to this point have addressed this problem with software engineering-specific automated query reformulation techniques, which work without developer involvement but are limited by the content of the original query. In other words, these techniques automatically improve the existing query but can not contribute new, previously unmentioned, concepts.

show abstract

Section: Software Development-related Queriesmentioning

confidence: 99%

Section: Software Development-related Queriesmentioning

confidence: 99%

Using clarification questions to improve software developers' Web search

Imran,

Damevski

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Moreover, they describe the limits of anonymisation of such data and how secure anonymisation could help. Rao et al (2021) discuss how their data was anonymised and how they filter out queries that were entered by less than k users and could potentially contain sensitive information. Yamashita et al (2017) discuss how the history of Git repositories was re-written to remove personal data.…”

Section: Data Showcasementioning

confidence: 99%

Ethics in the mining of software repositories

Gold

Krinke

2021

Empir Software Eng

View full text Add to dashboard Cite

Research in Mining Software Repositories (MSR) is research involving human subjects, as the repositories usually contain data about developers’ and users’ interactions with the repositories and with each other. The ethics issues raised by such research therefore need to be considered before beginning. This paper presents a discussion of ethics issues that can arise in MSR research, using the mining challenges from the years 2006 to 2021 as a case study to identify the kinds of data used. On the basis of contemporary research ethics frameworks we discuss ethics challenges that may be encountered in creating and using repositories and associated datasets. We also report some results from a small community survey of approaches to ethics in MSR research. In addition, we present four case studies illustrating typical ethics issues one encounters in projects and how ethics considerations can shape projects before they commence. Based on our experience, we present some guidelines and practices that can help in considering potential ethics issues and reducing risks.

show abstract

“…To reduce the efforts of annotation, recent weak supervision (WS) frameworks have been proposed which focus on enabling users to leverage a diversity of weaker, often programmatic supervision sources [76,77,75] to label and manage training data in an efficient way. Recently, WS has been widely applied to various machine learning tasks in a diversity of domains: scene graph prediction [9], video analysis [23,92], image classification [12], image segmentation [35], autonomous driving [96], relation extraction [36,107,57], named entity recognition [82,53,50,45,27], text classification [78,100,85,86], dialogue system [63], biomedical [43,19,64], healthcare [20,17,21,80,93,81], software engineering [74], sensors data [24,39], E-commerce [66,103], and multi-agent systems [102].…”

Section: Introductionmentioning

confidence: 99%

WRENCH: A Comprehensive Benchmark for Weak Supervision

Zhang,

Yu,

et al. 2021

Preprint

View full text Add to dashboard Cite

Recent Weak Supervision (WS) approaches have had widespread success in easing the bottleneck of labeling training data for machine learning by synthesizing labels from multiple potentially noisy supervision sources. However, proper measurement and analysis of these approaches remain a challenge. First, datasets used in existing works are often private and/or custom, limiting standardization. Second, WS datasets with the same name and base data often vary in terms of the labels and weak supervision sources used, a significant "hidden" source of evaluation variance. Finally, WS studies often diverge in terms of the evaluation protocol and ablations used. To address these problems, we introduce a benchmark platform, WRENCH, for thorough and standardized evaluation of WS approaches. It consists of 22 varied real-world datasets for classification and sequence tagging; a range of real, synthetic, and procedurally-generated weak supervision sources; and a modular, extensible framework for WS evaluation, including implementations for popular WS methods. We use WRENCH to conduct extensive comparisons over more than 100 method variants to demonstrate its efficacy as a benchmark platform. The code is available at https://github.com/JieyuZ2/wrench.Preprint. Under review.

show abstract

Search4Code: Code Search Intent Classification Using Weak Supervision

Cited by 8 publications

References 20 publications

Using clarification questions to improve software developers' Web search

Using clarification questions to improve software developers' Web search

Ethics in the mining of software repositories

WRENCH: A Comprehensive Benchmark for Weak Supervision

Contact Info

Product

Resources

About