The NICE UK geographic search filters for MEDLINE and Embase (Ovid): Post‐development study to further evaluate precision and number‐needed‐to‐read when retrieving UK evidence

Abstract

Background The National Institute for Health and Care Excellence's (NICE) United Kingdom (UK) geographic search filters for MEDLINE and Embase (OVID) retrieve evidence in literature searches for UK‐focused research topics with high recall. Their precision and number‐needed‐to‐read (NNR) was examined previously in case studies using a single review. This paper details a larger post‐development study that was conducted to test the NICE UK filters' precision and NNR more extensively. Methods The filters' recall of included UK references from 100 reviews was calculated. As reproducible search strategies were not available for every review, the MEDLINE filter's precision and NNR were calculated using strategies from 25 reviews. Strategies from nine reviews were used for the Embase filter. Results The MEDLINE filter achieved an average of 96.4% recall for the included UK references from the 100 reviews and the Embase filter achieved an average of 97.4% recall. Compared to not using a filter, the MEDLINE filter achieved an average of 98.9% recall for the 25 reviews. Precision was increased by an average of 7.8 times, reducing the NNR from 357 to 46. The Embase filter achieved an average of 97.1% recall for the nine reviews. Precision was increased by an average of 5.1 times, reducing the NNR from 746 to 146. Conclusion There is more evidence to demonstrate that the NICE UK filters retrieve the majority of UK evidence from MEDLINE and Embase while increasing precision and reducing NNR. The filters can save time spent on selecting evidence for UK‐focused research topics.

The NICE OECD countries’ geographic search filters: Part 1—methodology for developing the draft MEDLINE and Embase (Ovid) filters

“…U.K.: MEDLINE [14] and Embase [15] We previously developed and validated the NICE U.K. geographic search filters for MEDLINE and Embase (Ovid) [14,15]. These filters retrieve evidence about the U.K. effectively and efficiently when needed for contextsensitive NICE guidance topics [14][15][16]. They have been used to inform U.K.-focused topics in NICE guidelines [17][18][19] as well as national reports and systematic reviews produced by external organizations [20][21][22].…”

“…Using geographic subject heading terms alone for the draft filters has limitations. For instance, not all references are indexed with geographic subject headings [12][13][14][15][16]; therefore, some irrelevant non-OECD country evidence will be retained. However, this limitation does not pose risks for OECD evidence to be excluded inadvertently.…”

The ScHARR LMIC filter: Adapting a low‐ and middle‐income countries geographic search filter to identify studies on preterm birth prevention and management

“…Future developments for the filter would benefit from such testing and validation, similar to that conducted by the developers of other geographic search filters. 9,24,25,27,[29][30][31] Future research on the use and efficiency of geographic search filters would be of benefit to information specialists and the evidence synthesis community. Ayiku et al 45 provide a useful guide to developing and validating geographic search filters and encourage more to be developed.…”

“…Beyond our initial development, it has not been tested for sensitivity and precision, nor undergone any validation. Future developments for the filter would benefit from such testing and validation, similar to that conducted by the developers of other geographic search filters 9,24,25,27,29–31 …”

What evidence exists on the links between natural climate solutions and climate change mitigation outcomes in subtropical and tropical terrestrial regions? A systematic map protocol

“…With careful development and testing, the use of NOTs can improve the efficiency and precision of a search strategy, particularly when dealing with large corpora (e.g. [ 51 – 53 ]). We tested the likelihood that the use of NOTs would result in a significant portion of missed citations using a random sample and a relevance ranked sample of studies (n = 500 citations) that would have not been captured in a search strategy including NOT terms.…”