Use of E-values for addressing confounding in observational studies—an empirical assessment of the literature

Abstract

Background E-values are a recently introduced approach to evaluate confounding in observational studies. We aimed to empirically assess the current use of E-values in published literature. Methods We conducted a systematic literature search for all publications, published up till the end of 2018, which cited at least one of two inceptive E-value papers and presented E-values for original data. For these case publications we identified control publications, matched by journal and issue, where the authors had not calculated E-values. Results In total, 87 papers presented 516 E-values. Of the 87 papers, 14 concluded that residual confounding likely threatens at least some of the main conclusions. Seven of these 14 named potential uncontrolled confounders. 19 of 87 papers related E-value magnitudes to expected strengths of field-specific confounders. The median E-value was 1.88, 1.82, and 2.02 for the 43, 348, and 125 E-values where confounding was felt likely to affect the results, unlikely to affect the results, or not commented upon, respectively. The 69 case-control publication pairs dealt with effect sizes of similar magnitude. Of 69 control publications, 52 did not comment on unmeasured confounding and 44/69 case publications concluded that confounding was unlikely to affect study conclusions. Conclusions Few papers using E-values conclude that confounding threatens their results, and their E-values overlap in magnitude with those of papers acknowledging susceptibility to confounding. Facile automation in calculating E-values may compound the already poor handling of confounding. E-values should not be a substitute for careful consideration of potential sources of unmeasured confounding. If used, they should be interpreted in the context of expected confounding in specific fields.

Commentary: Developing best-practice guidelines for the reporting of E-values

“…Blum et al . 1 sampled a set of control papers from the same journals as those that reported E-values. Of these 69 papers, 52 (75.3%) apparently had no discussion whatsoever of unmeasured confounding.…”

“…As noted in our paper 3 and by Blum et al . 1 , the E-value is not context-free. The E-value needs to be evaluated in light of the measured confounders, the outcome, the exposure and the potentially known unmeasured confounders.…”

“…Blum et al . 1 report that, of the 87 articles in their sample, 34 (39%) named specific variables that could be confounders and were unaccounted for. Such reporting should be improved.…”

“…We would like to thank Blum et al . 1 for their interesting analysis of the current reporting practices around the use of the E-value to evaluate sensitivity to unmeasured confounding. 2 , 3 As with nearly any quantitative tool, the E-value is potentially subject to misuse, 4 , 5 examples of which are indeed documented in their paper.…”

Assessment of Acute Obstetrical Needs and the Potential Utility of Point-Of-Care Ultrasound in the North East Region of Haiti: A Cross-Sectional Study

“…Although the multivariable analyses controlled for factors associated with the outcomes of interest, unmeasured confounders such as disease specific characteristics and variations in treatments may have influenced the findings. To address this limitation, alternative statistical methods could be employed such as the use of E-Value analytical approaches to assess for the effect of measured confounding, however the most optimal approach would be the completion of a dedicated prospective study of the implementation of POCUS in the study setting [24].…”