Protocol for a systematic review and meta-analysis of minimal important differences for generic multiattribute utility instruments

Abstract

IntroductionGeneric multiattribute utility instruments (MAUIs) are efficient tools for determining and enumerating health-related quality of life. MAUIs accomplish this by generating health state utilities (HSUs) via algorithms. Minimal important differences (MIDs) assist with the interpretation of HSUs by estimating minimum changes that are clinically significant. The overall goal of the proposed systematic review and meta-analysis is the development of comprehensive guidelines for MID estimation.Methods and analysisThis protocol defines a systematic review and meta-analysis of MIDs for generic MAUIs. The proposed research will involve a comprehensive investigation of 10 databases (EconLit, IDEAs database, INAHTA database, Medline, PsycINFO, Embase, Emcare, JBIEBP and CINAHL) from 1 June 2022 to 7 June 2022, and will be performed and reported in accordance with several validated guidelines, principally the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. The quality of papers, considered for inclusion in the review, will be appraised using the COnsensus-based Standards for the selection of health Measurement INstruments, inter alia.Narrative analysis will involve identifying the characteristics of MIDs including methods of calculation, sources of heterogeneity, and validation. Meta-analysis will also be conducted. The descriptive element of meta-analysis will involve the generation of I2statistics and Galbraith plots of MID heterogeneity. Together with narrative analysis, this will allow sources of MID heterogeniety to be identified. A multilevel mixed model, estimated via restricted maximum likelihood estimation, will be constructed for the purposes of meta-regression. Meta-regression will attempt to enumerate the effects of sources of heterogeneity on MID estimates. Meta-analysis will be concluded with pooling of MIDs via a linear random-effects model.Ethics and disseminationEthics approval is not required for this review, as it will aggregate data from published literature. Methods of dissemination will include publication in a peer-reviewed journal, as well as presentation at conferences and seminars.PROSPERO registration numberCRD42021261821.

Enhancing the Interpretation of Continuous Outcomes and Subgroup Analyses in Systematic Reviews

“…2 An alternative approach that could provide more meaningful insights is transforming various scales into a common instrument score. This method, coupled with modeling the risk difference for reaching the minimally important difference, 3 can better convey if the observed effects are clinically significant. Such an approach would enhance the utility of the findings for both clinicians and patients.…”

“…First, none of the 23 randomized clinical trials included in our review reported MIDs, and the outcomes measured by multiple scales across these trials were not reported consistently enough to calculate MIDs to achieve appropriate homogeneity. Second, there are no guidelines for MID estimation of generalized multiattribute utility instruments that were included in our study, 3 and determining MIDs sourced from previous work or establishing MIDs by consensus may introduce bias due to methodological subjectivity. 4 Moreover, the Cummings treatment effect model 5 indicates that MID will not appear in the early stage of the treatment; the difference in effect compared with the control group gradually increases as time progresses and, theoretically, it is expected to achieve and exceed MID.…”

“…Case studies and challenges are published due to their inherent difficulty, and human raters typically succeed less than 40% of the time. 2,3 Challenges are usually presented with multiple-choice options, which helps test clinical knowledge and avoid nonconceptual reasons for incorrect responses (eg, being too general). In Barile et al, 1 11 diagnoses were rejected for being nonspecific, which may stem from a vague prompt or lack of forced choice.…”

“…This metric is on par with past performance of human raters. 2,3 The authors also exclude essential 4 figures, videos, and tables from the LLM input that are usually available to humans. Clinical information stored in visuals (eg, rash photographs, fluoroscopy videos, 4 and laboratory results) are often necessary for diagnostics.…”

“…This metric is more than twice as accurate as reported for GPT-3.5 1 and matches previously reported clinician benchmarks that usually included some visuals and multiple choices. 2,3 LLMs are not a suitable replacement for the critical thinking and experience of trained pediatric clinicians. Even highly performant models are not necessarily "wise" or "good."…”