Reproducibility of Literature Search Reporting in Medical Education Reviews

Abstract

According to this analysis, literature search strategies in medical education reviews are highly variable and generally not reproducible. The authors provide recommendations to facilitate future high-quality, transparent, and reproducible literature searches.

Compliance of systematic reviews in veterinary journals with Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) literature search reporting guidelines

“…Systematic reviews require a comprehensive search for published and unpublished studies in order to minimize bias; therefore, for most review topics, it is necessary to search multiple research databases to ensure that all of the relevant literature is retrieved [2, 13–15]. The appropriate databases to search will vary with the topic of the review, but failure to search multiple databases can increase the risk that relevant studies will be missed, which could bias the outcome of the review [16, 17].…”

“…Reproducibility is an essential characteristic of all reliable primary biomedical research as well as research synthesis reports such as systematic reviews. Because the studies retrieved by the literature search form the evidence base for a systematic review, it is essential that the search strategy, which is the data collection method for reviews, be reported in sufficient detail to enable the search to be replicated and evaluated [2, 27]. A list of search terms is not sufficient to enable accurate replication of a literature search.…”

“…Unfortunately, incomplete and inaccurate reporting of systematic review literature searches [2, 3] is part of a larger, widespread problem of poor reporting of all types of biomedical research [4–6]. Poor reporting of biomedical research has serious consequences, making it difficult or impossible to assess the quality of, replicate the study, or use the study in clinical decision making or in subsequent systematic reviews [7].…”

Literature Searching in Medical Education: Online Tutorial Development from Idea to Creation

“…Making this somewhat complicated is that no single database provides comprehensive coverage of medical education research. 4 This translates into a need to search multiple databases and information resources and employ a variety of search strategies, such as using controlled vocabularies, including synonyms, and leveraging truncation. However, based on a review of medical education literature search strategies, it appears that medical education researchers do not search much beyond MEDLINE and that their search strategies do not take advantage of search techniques.…”

Some Potential “Pitfalls” in the Construction of Educational Systematic Reviews

“…This challenge is not an issue confined to Academic Psychiatry alone but has been found to occur in other leading educational journals. In one search of reviews in Academic Medicine, Teaching and Learning in Medicine, and Medical Education, only a minority of identified reviews explicitly described searching nonmedical databases or listed Medical Subject Headings (MeSH) and Boolean operators, and none included reproducible searches [16]. Of course, the sensitivity of the search strategy has to be balanced with its specificity so that there is not an overwhelming number of articles to screen.…”