Experiences from using snowballing and database searches in systematic literature studies

Abstract

Background: Systematic literature studies are commonly used in software engineering.There are two main ways of conducting the searches for these type of studies; they are snowballing and database searches. In snowballing, the reference list (backward snowballing -BSB) and citations (forward snowballing -FSB) of relevant papers are reviewed to identify new papers whereas in a database search, different databases are searched using predefined search strings to identify new papers. Objective: Snowballing has not been in use as extensively as database search. Hence it is important to evaluate its efficiency and reliability when being used as a search strategy in literature studies. Moreover, it is important to compare it to database searches. Method: In this paper, we applied snowballing in a literature study, and reflected on the outcome. We also compared database search with backward and forward snowballing. Database search and snowballing were conducted independently by different researchers. The searches of our literature study were compared with respect to the efficiency and reliability of the findings. Results: Out of the total number of papers found, snowballing identified 83% of the papers in comparison to 46% of the papers for the database search. Snowballing failed to identify a few relevant papers, which potentially could have been addressed by identifying a more comprehensive start set. Conclusion: The efficiency of snowballing is comparable to database search. It can potentially be more reliable than a database search however, the reliability is highly dependent on the creation of a suitable start set.

'Follow the moon' development

“…There are other approaches to being systematic in the conduct of reviews [19,1], which could equally be considered.…”

Optimization of Micro-Electrical Discharge Machining Parameters of Ti6AL4V Component: A Mapping Study

“…In this study, we used backward snowballing, identifying articles from the reference lists of some relevant articles, and afterwards, we applied forward snowballing identifying articles that have cited the articles we have already found. We confidently used the snowballing approach since it has been proved (Badampudi, Wohlin, & Petersen, 2015) that key word search of databases and snowballing give conclusions and patterns quite similar. At the very end, having a robust knowledge about many of the studies, we investigated the Google Scholars database.…”

Conceptualising a multidimensional model of information communication and technology project complexity

“…Forward and backward snowballing was also employed when performing the content analysis. Forward snowballing searches literature which has cited the literature in question, while backward snowballing searches the reference list of literature (Badampudi, Wohlin & Petersen 2015;Jalali & Wohlin 2012;Schön et al 2017). The final process was to conduct a manual scan of the literature sources to ensure adequate concept representation and enable conceptual model construction (Asher 2013;Dube & Marnewick 2016;Schön et al 2017).…”