An empirical assessment of best-answer prediction models in technical Q&A sites

Abstract

Technical Q&A sites have become essential for software engineers as they constantly seek help from other experts to solve their work problems. Despite their success, many questions remain unresolved, sometimes because the asker does not acknowledge any helpful answer. In these cases, an information seeker can only browse all the answers within a question thread to assess their quality as potential solutions. We approach this time-consuming problem as a binary-classification task where a best-answer prediction model is built to identify the accepted answer among those within a resolved question thread, and the candidate solutions to those questions that have received answers but are still unresolved. In this paper, we report on a study aimed at assessing 26 best-answer prediction models in two steps. First, we study how models perform when predicting best answers in Stack Overflow, the most popular Q&A site for software engineers. Then, we assess performance in a cross-platform setting where the prediction models are trained on Stack Overflow and tested on other technical Q&A sites. Our findings show that the choice of the classifier and automatied parameter tuning have a large impact on the prediction of the best answer. We also demonstrate that our approach to the bestanswer prediction problem is generalizable across technical Q&A sites. Finally, we provide practical recommendations to Q&A platform designers to curate and preserve the crowdsourced knowledge shared through these sites.

I Know What You Are Searching for: Code Snippet Recommendation from Stack Overflow Posts

“…• Traditional Classifiers Considering that our CodeSelector ranks the code snippet candidates by doing classification between QC pairs, it is hence natural to compare our approach with traditional classifiers. Recently Calefato et al [9] proposed an approach for best answer prediction problem by formulating it as a a binary-classification task. The binary-classification methods output a score referring to a probability of relevance.…”

“…Conventional techniques for retrieving answers primarily focus on complementary features of the CQA sites. Calefato et al [9] transform the answer selection task to a binary classification problem, they empirically evaluated 26 answer prediction model in Stack Overflow. Xu et al [69] proposed a novel framework for generating relevant, useful and diverse answer summary for technical questions in Stack Overflow.…”

Technical Q8A Site Answer Recommendation via Question Boosting

“…The detailed algorithms of DeepAns for offline training and online recommendation are presented in Algorithm 1 and Algorithm 2 respectively. To be more specific, during the offline training, we use the data from technical Q&A sites to train the question boosting model (lines 1-3) and answer recommendation model (lines [4][5][6][7][8][9][10][11][12][13][14][15][16][17][18][19][20]. When it comes to the online recommendation, for a given user query, we first collect a pool of answer candidates via finding its similar questions (lines 1-8).…”

An empirical study on the characteristics of question-answering process on developer forums

“…There has been extensive research devoted to analyzing Stack Overflow, such as analyzing obsolete answers [11], discussing bestanswer prediction models [1], learning to answer SO questions [5].…”

“…Extensive research [1,5,11] has studied StackOverflow (SO). However, despite the importance of API forums, little research has focused on API-specific forums.…”