An alternative screening approach for Google Search identifies an accurate and manageable number of results for a systematic review (case study)

Abstract

BackgroundA challenge when using Google Search to identify studies for a systematic review is managing the high number of results, which can number in the hundreds of thousands or even more. Studies and guidance on web searching suggest limiting the screening process, e.g. to the first 100 results.ObjectivesOur objective in this case study is to demonstrate an alternative approach to screening the results retrieved by Google Search which is based on our experience that the viewable number of results is often far fewer than the estimated number calculated by the search engine.MethodsWe screened the results of three searches of Google Search using our approach, which involves increasing the number of results displayed per page from 10 to the maximum of 100. We then calculated the viewable number of results and compared this with the estimated number.ResultsThe mean of the estimated number of results for the three searches was 569,454,000. The mean of the viewable number results was 463 (0.00008% of the mean of the estimated number of results).ConclusionOur findings challenge the commonly reported view that the number of results retrieved when using Google Search is too high to screen in full.

Feasibility and desirability of screening search results from Google Search exhaustively for systematic reviews: A cross‐case analysis

“…the results of searches exhaustively. 13 We have also suggested that, for the two case studies in the present study, it was desirable to screen to the third page of results, where the lowest ranking first appearances of relevant studies were identified (approximately 65% of the retrieved results in total in both cases). The distribution of studies, relevant or not, was much lower on page four, which diminished the desirability of screening these results.…”

“…In order to facilitate data analysis, we set Google Search to display 100 results per page as described in Briscoe and Rogers. 13 This involved using the slide-bar option in the 'See All Settings' submenu of the main settings menu (accessed via the 'gear cog' icon on the Google Search homepage) to increase the results per page from the default of 10 to the maximum of 100. Prior to searching we also ensured that we were logged out of our personal Google accounts and used the option in the Search Settings page to deactivate search customisation, which stops Google from using the user's search history to personalise the ranking of search results according to their previous searches.…”

“…The viewable number was calculated using the following procedure described by Briscoe and Rogers. 13 First, we selected the final page of results using the page numbers at the bottom of the page. We then selected the option to "Repeat the search with the omitted results showing", which includes search results that are similar to the initial set of results, but which Google Search initially omits to avoid potential duplication.…”

“…13 In summary, Briscoe and Rogers showed that the mean number of viewable results for three Google Search searches was 463, in contrast to the mean number of estimated results reported by the search engine of 569,454,000. 13 The viewable number of results was calculated by setting Google Search to display 100 results per page and scrolling to the final page of results. 13 Despite this finding, it is not clear whether exhaustive screening is desirable, particularly in view of Google Search's PageRank algorithm which ranks content according to relevance, which might make screening the results in full unprofitable (in terms of the identification of studies) even if feasible.…”

“…13 The viewable number of results was calculated by setting Google Search to display 100 results per page and scrolling to the final page of results. 13 Despite this finding, it is not clear whether exhaustive screening is desirable, particularly in view of Google Search's PageRank algorithm which ranks content according to relevance, which might make screening the results in full unprofitable (in terms of the identification of studies) even if feasible. 14…”