Evidence surveillance to keep up to date with new research

Abstract

Overview of the topicResearch is being published at an ever-increasing rate and it is becoming more and more difficult for systematic reviewers to find research in a timely way and keep existing reviews updated as new studies are published.[1] This is a particular problem for organizations which maintain libraries of systematic reviews, such as the Cochrane and Campbell Collaborations, as the more systematic reviews they publish, the greater the burden of maintenance. It is also a challenge for guideline-producing organizations which, for pragmatic reasons, typically invest significant resources and effort in one-off periodic updates without knowing whether the evidence base has changed or has actually changed so rapidly that more frequent updating would have been warranted. Previous work has shown that systematic reviews can date very quicklywith some out of date as soon as they are published [2]and it is becoming clear that our current methods of research curation are wasteful of societal investment in research, and risk resulting in suboptimal outcomes. [3] This chapter is concerned with this problem of 'data deluge' and the need to maintain a better surveillance of research in order to keep abreast of new developments. It is thus related to work on Living Systematic Reviews (LSRs), that are 'continually updated, incorporating relevant new evidence as it becomes available'. [4,5] The chapter outlines developments in automation technologies that are already making the systematic review process more efficient and then focuses on the way that global research curation systems are organized. The chapter suggests that new approaches are needed in order to support the production of evidence syntheses in efficient and timely ways. Case studies 9.1 and 9.2 explain how these new developments are being put into practice to realise these benefits. 9.2Discussion New ways of working that integrate and capitalize on automation are necessary to tackle the growing burden of identifying and synthesizing research. New technologieswhich range from the mundane (such as identifying duplicates in bibliographic records) to full Artificial Intelligence (AI) systemsare under constant development and are already assisting various aspects of the evidence curation (see box) and discovery process. It is possible to break these new tools down into two broad categories:1. Tools which can make existing manual processes more efficient.2. Tools which aim to change the way systematic reviews are carried out in more fundamental ways by linking tools together and changing the sequencing of activities.The following section examines the potential for automation to assist in existing processes, and the section after that considers the potential for linking these tools into integrated surveillance systems for LSRs and other types of living evidence, such as guidelines.3 What is 'curation'?A key concept in this chapter is 'evidence curation'. 'Curation' as an idea has been around a long time, and concerns the activities necessary to manage, sort and ar...

Fit for the Challenge

“…Because of the vast amounts of training data generated by Cochrane Crowd, these classifiers can very accurately determine the likelihood that a record reports a randomised trial. Cochrane is now routinely employing machine classifiers centrally (via its evidence pipeline) to efficiently harvest all randomised trials which are then fed directly into Cochrane's trials register in the Cochrane Library (Marshall et al, 2018) (Thomas et al, 2019).…”

Reflections from New Zealand

“…Because of the vast amounts of training data generated by Cochrane Crowd, these classifiers can very accurately determine the likelihood that a record reports a randomised trial. Cochrane is now routinely employing machine classifiers centrally (via its evidence pipeline) to efficiently harvest all randomised trials which are then fed directly into Cochrane's trials register in the Cochrane Library (Marshall et al, 2018) (Thomas et al, 2019).…”