Evaluation of the sensitivity, accuracy and currency of the Cochrane COVID-19 Study Register for supporting rapid evidence synthesis production

Abstract

IntroductionThe Cochrane COVID-19 Study Register (CCSR) is a public, continually updated, curated database of COVID-19 study references. The aim of this study-based register is to support rapid and living evidence synthesis, including a project to build an evidence ecosystem of COVID-19 research (CEOsys). In November and December 2020 we conducted an evaluation of the CCSR for CEOsys, measured its performance and identified areas for improvement.MethodsFor the evaluation we generated a purposive sample of 286 studies from 20 reviews to calculate the CCSR’s sensitivity (comprehensiveness), accuracy (correctly classified and linked studies) and currency (time to publish and process references).ResultsThe CCSR had an overall sensitivity of 77.2%, with the highest sensitivity for interventional studies (94.4%) and lowest sensitivity for modelling studies (63.6%). The study register had 100% sensitivity for trial registry records, 86.5% for journal articles and 52.4% for preprints. 98.3% of references were correctly classified with regard to study type, and 93.4% with regard to study aim. 89% of studies were correctly linked. 81.4% of references were published to the register in under 30 days, with 0.5 day (median) for trial registry records, 2 days for journal articles and 56 days for preprints.ConclusionThe CCSR had high sensitivity, accurate study classifications and short publishing times for journal articles and trial registry records. We identified that the CCSR’s coverage and publishing times for preprints needed improvement. Finally, the evaluation illustrated the value of a study-based register for identifying additional study references for analysis in evidence synthesis.