Quality appraisal for systematic literature reviews of health state utility values: a descriptive analysis

Abstract

Background Health state utility values (HSUVs) are an essential input parameter to cost-utility analysis (CUA). Systematic literature reviews (SLRs) provide summarized information for selecting utility values from an increasing number of primary studies eliciting HSUVs. Quality appraisal (QA) of such SLRs is an important process towards the credibility of HSUVs estimates; yet, authors often overlook this crucial process. A scientifically developed and widely accepted QA tool for this purpose is lacking and warranted. Objectives To comprehensively describe the nature of QA in published SRLs of studies eliciting HSUVs and generate a list of commonly used items. Methods A comprehensive literature search was conducted in PubMed and Embase from 01.01.2015 to 15.05.2021. SLRs of empirical studies eliciting HSUVs that were published in English were included. We extracted descriptive data, which included QA tools checklists or good practice recommendations used or cited, items used, and the methods of incorporating QA results into study findings. Descriptive statistics (frequencies of use and occurrences of items, acceptance and counterfactual acceptance rates) were computed and a comprehensive list of QA items was generated. Results A total of 73 SLRs were included, comprising 93 items and 35 QA tools and good recommendation practices. The prevalence of QA was 55% (40/73). Recommendations by NICE and ISPOR guidelines appeared in 42% (16/40) of the SLRs that appraised quality. The most commonly used QA items in SLRs were response rates (27/40), statistical analysis (22/40), sample size (21/40) and loss of follow up (21/40). Yet, the most commonly featured items in QA tools and GPRs were statistical analysis (23/35), confounding or baseline equivalency (20/35), and blinding (14/35). Only 5% of the SLRS used QA to inform the data analysis, with acceptance rates of 100% (in two studies) 67%, 53% and 33%. The mean counterfactual acceptance rate was 55% (median 53% and IQR 56%). Conclusions There is a considerably low prevalence of QA in the SLRs of HSUVs. Also, there is a wide variation in the QA dimensions and items included in both SLRs and extracted tools. This underscores the need for a scientifically developed QA tool for multi-variable primary studies of HSUVs.

Lessons learned from implementing the Non-Communicable Diseases Kit in a humanitarian emergency: an operational evaluation in Sudan

“…Digital health interventions, such as the use of electronic health records, NCD-specific registers or tracing systems, could play a pivotal role in achieving these improvements. 22 …”

“…health interventions, such as the use of electronic health records, NCD-specific registers or tracing systems, could play a pivotal role in achieving these improvements. 22 Logistical capacity, repackaging and delays with kit distribution Leadership and governance Assessing leadership and governance were beyond the scope of this evaluation, and as such, no specific data were collected in this area. It is worth noting that the IRC is responsible for the management of both facilities, and a decision was made before deployment to have the basic module for NCDs and mental health medicines assembled in the IRC central warehouse in Khartoum, and to facilitate faster delivery and avoid delays as previously experienced in Yemen, Libya and South Sudan.…”

Quality Appraisal in Systematic Literature Reviews of Studies Eliciting Health State Utility Values: Conceptual Considerations

“…Recent years have witnessed an exponential increase in primary studies eliciting HSUVs, with researchers employing direct (standard gamble, time trade-off or visual analogue scale) and indirect methods (i.e. generic preference-based measures such as the European Quality of Life Five Dimension, Short-Form Six Dimensions, Health Utility Index or mapping algorithms) [ 1 , 5 , 6 ]. Consequently, systematic reviews and meta-analyses have become indispensable tools for synthesising these studies across various decision-making contexts.…”

“…However, the conduct of QA in systematic reviews of studies that elicit HSUVs needs to be improved. Recent reviews [ 5 , 14 ] of studies that elicited HSUVs estimated that only 55% appraised the quality of individual studies in the systematic reviews, which is far lower than that in other research fields [ 15 – 18 ]. The low prevalence of conducting a QA in this field could be partly attributed to the lack of a widely accepted and scientifically developed QA tool [ 5 , 14 ] specific to this context.…”

“…Recent reviews [ 5 , 14 ] of studies that elicited HSUVs estimated that only 55% appraised the quality of individual studies in the systematic reviews, which is far lower than that in other research fields [ 15 – 18 ]. The low prevalence of conducting a QA in this field could be partly attributed to the lack of a widely accepted and scientifically developed QA tool [ 5 , 14 ] specific to this context. This gap may arise from the unique features of these studies, which include multiple applicable study designs and elicitation methods [ 5 ].…”