Do published search filters to identify diagnostic test accuracy studies perform adequately?

Abstract

Objectives : To assess the performance of published search filters in finding diagnostic test accuracy studies. Methods : Diagnostic test accuracy search filters were identified by searching medline , our own files and by requesting unpublished filters from colleagues. We applied the filters to a case study review of diagnostic test accuracy studies for urinary tract infections (UTI) in young children. The included studies with records in medline formed the gold standard. The performance of the filters in finding those gold standard records was assessed. Results : We identified twenty-three diagnostic test accuracy search filters for use with medline . The case study systematic review of UTI included 179 studies of diagnostic test accuracy, of which 160 were available in medline . The filters showed a wide range of sensitivities (range: 20.6% to 86.9%) and precision (range: 1% to 9.4%). Conclusions : Our results broadly support those reported in two other studies. The search filters tested do not offer an adequate trade-off between sensitivity and precision to be used to identify studies for systematic reviews. However, there are methods available to explore whether search filters are viable based on an objective statistical analysis of the text and indexing used in records.

Assessing the performance of methodological search filters to improve the efficiency of evidence information retrieval: five literature reviews and a qualitative study

“…Twenty-one studies were identified as potentially meeting the inclusion criteria for this review based on titles and abstracts 2,10,14,15,17,19,22,23,25,33,48,49,[55][56][57][58][59][60][61][62][63] Of these studies, 10 reported the development of one or more search filters, whose performance was then compared against the performance of existing filters 10,14,15,17,19,22,23,25,56,57 and 11 reported the comparative performance of existing filters. 2,33,48,49,55,[58][59][60][61][62][63] On receipt of the full articles, three studies 55,60,62 were excluded from the review based on the criteria outlined in the methods section.…”

“…2,33,48,49,55,[58][59][60][61][62][63] On receipt of the full articles, three studies 55,60,62 were excluded from the review based on the criteria outlined in the methods section. The 18 included studies are listed in Tables 8 and 9 and the excluded studies are listed in Table 10.…”

“…Although there are many published search filters, few have been extensively validated beyond the data offered in the original publications. [47][48][49] This means that their performance in the real-world setting of day-to-day information retrieval across a range of search topics is unknown. 50 Furthermore, search filters are seldom assessed against common data sets, which makes a comparison of performance across filters problematic.…”

“…Methods exist to test search filter performance and to build the performance picture, including reviews of search filter performance. 48,49,[52][53][54] There is no formal guidance, however, on the best methods for testing filter performance, on which performance measures are valued by searchers and on which measures should ideally be reported to assist searchers in choosing between filters. The performance picture for filters across different disciplines, questions and databases is therefore largely unknown.…”

The value of FDG positron emission tomography/computerised tomography (PET/CT) in pre-operative staging of colorectal cancer: a systematic review and economic evaluation.

“…34,35 Available guidance on search methods for diagnostic accuracy tests was followed. 36 Intermediate search results were tested against known relevant publications and revised to ensure enough sensitivity to include those known publications with the intention that additional, unknown publications would not be missed.…”