The NICE OECD countries’ geographic search filters: Part 2—validation of the MEDLINE and Embase (Ovid) filters

Abstract

Objective: We previously developed draft MEDLINE and Embase (Ovid) geographic search filters for Organisation for Economic Co-operation and Development (OECD) countries to assess their feasibility for finding evidence about the countries. Here, we describe the validation of these search filters.Methods: We identified OECD country references from thirty National Institute for Health and Care Excellence (NICE) guidelines to generate gold standard sets for MEDLINE (n=2,065) and Embase (n=2,023). We validated the filters by calculating their recall against these sets. We then applied the filters to existing search strategies for three OECD-focused NICE guideline reviews (NG103 on flu vaccination, NG140 on abortion care, and NG146 on workplace health) to calculate the filters’ impact on the number needed to read (NNR) of the searches.Results: The filters both achieved 99.95% recall against the gold standard sets. Both filters achieved 100% recall for the three NICE guideline reviews. The MEDLINE filter reduced NNR from 256 to 232 for the NG103 review, from 38 to 27 for the NG140 review, and from 631 to 591 for the NG146 review. The Embase filter reduced NNR from 373 to 341 for the NG103 review, from 101 to 76 for the NG140 review, and from 989 to 925 for the NG146 review.Conclusion: The NICE OECD countries’ search filters are the first validated filters for the countries. They can save time for research topics about OECD countries by finding the majority of evidence about OECD countries while reducing search result volumes in comparison to no filter use.

The ScHARR LMIC filter: Adapting a low‐ and middle‐income countries geographic search filter to identify studies on preterm birth prevention and management

“…Future developments for the filter would benefit from such testing and validation, similar to that conducted by the developers of other geographic search filters. 9,24,25,27,[29][30][31] Future research on the use and efficiency of geographic search filters would be of benefit to information specialists and the evidence synthesis community. Ayiku et al 45 provide a useful guide to developing and validating geographic search filters and encourage more to be developed.…”

“…Beyond our initial development, it has not been tested for sensitivity and precision, nor undergone any validation. Future developments for the filter would benefit from such testing and validation, similar to that conducted by the developers of other geographic search filters 9,24,25,27,29–31 …”

Searching for the social determinants of health: observations from evidence synthesis publications

“…As described by Campbell, the terms filters and hedges are sometimes used interchangeably, but more traditionally filters refine a search to studies with specific characteristics, while hedges are focused on describing subject searches [ 20 ]. Under this definition, examples of filters constructed by researchers include ones pertaining to study type [ 21 , 22 ], population [ 23 , 24 ], and geography [ 25 , 26 ], while examples of constructed hedges include ones for deprescribing [ 27 ], adverse effects of medical devices [ 28 ], acute kidney injury [ 29 ], patient-based benefit-risk assessment (BRA) of medicines [ 30 ], and public involvement in health efforts [ 31 ]. For other researchers to be able to reuse the developed strategy with an informed sense of its anticipated performance, it should be validated by formal testing and reporting of its relative recall and precision when tested against a gold standard set of relevant publications.…”

Development and validation of a geographic search filter for MEDLINE (PubMed) to identify studies conducted in Germany

“…To the best of our knowledge, the current availability of geographic search filters is somewhat limited, and very few of them have been validated. Among the validated geographic search filters are those for Spain (12), Africa (13), the United Kingdom (UK) (3,11), German-speaking countries (specific for high-impact factor nursing journals) (14), the group of 37 countries of the Organisation for Economic Co-operation and Development (OECD) (2,15), as well as the United States (US) (16).…”