Performance of conceptual framework elements for the retrieval of qualitative health literature: a case study

Abstract

Objective: A growing volume of studies address methods for performing systematic reviews of qualitative studies. One such methodological aspect is the conceptual framework used to structure the review question and plan the search strategy for locating relevant studies. The purpose of this case study was to evaluate the retrieval potential of each element of conceptual frameworks in qualitative systematic reviews in the health sciences.Methods: The presence of elements from conceptual frameworks in publication titles, abstracts, and controlled vocabulary in CINAHL and PubMed was analyzed using a set of qualitative reviews and their included studies as a gold standard. Using a sample of 101 publications, we determined whether particular publications could be retrieved if a specific element from the conceptual framework was used in the search strategy.Results: We found that the relative recall of conceptual framework elements varied considerably, with higher recall for patient/population (99%) and research type (97%) and lower recall for intervention/phenomenon of interest (74%), outcome (79%), and context (61%).Conclusion: The use of patient/population and research type elements had high relative recall for qualitative studies. However, other elements should be used with great care due to lower relative recall.

Supplementary strategies identified additional eligible studies in qualitative systematic reviews

“…Our findings suggest that it may be challenging to retrieve publications for qualitative reviews, even though these publications are indexed in PubMed. Traditional database searches managed to retrieve a total of 62.4% (63 identifiable publications from 11 reviews with a total of 101 publication) of the included studies in a set of qualitative reviews [13], and supplementing with alternative search strategies and citation searching recall increased to 87.1% (63 publications identifiable through traditional database searches plus 25 publications identifiable through citation searches and alternative search strategies out of 101 total publications). Again, it should be stressed that all the publications not retrieved are in fact indexed in PubMed, and therefore should be expected to be identifiable in some way.…”

“…Another important aspect that should be brought into this discussion, is the fact that our analyses are based on qualitative reviews and hence, qualitative studies and publications. As mentioned previously, traditional database searches only retrieved 62.4% of the (PubMed-indexed) publications included in a set of qualitative reviews [13] (when the specific elements of conceptualizing models that are recommended were used to form the search blocks). You would therefore have to rely on supplementary searches.…”

“…When searching for qualitative health literature it is particularly challenging to retrieve all potentially relevant studies [11,12]. Relevant studies may be indexed in the databases although not retrieved using traditional search strategies in the databases [13]. Challenges with indexing and abstract content make it difficult to devise an efficient search strategy in traditional bibliographic databases and it is therefore necessary to go beyond the traditional database search strategies [14] or use a tailored approach to identify eligible studies [15].…”

“…We use these as a gold standard to evaluate the efficiency of alternative search strategies. The 12 reviews included 101 PubMed-indexed publications [13]. One of the 12 reviews had only one included publication that were nonidentifiable in PubMed, when following current recommendations (i.e., including the conceptual elements (P: patients/population), I (intervention/phenomenon of interest) and research type to structure the literature search).…”