Simulating ComBat: how batch correction can lead to the systematic introduction of false positive results in DNA methylation microarray studies

Abstract

Background: Systematic technical effects-also called batch effects-are a considerable challenge when analyzing DNA methylation (DNAm) microarray data, because they can lead to false results when confounded with the variable of interest. Methods to correct these batch effects are error-prone, as previous findings have shown. Results: Here, we demonstrate how using the R function ComBat to correct simulated Infinium HumanMethylation450 BeadChip (450 K) and Infinium MethylationEPIC BeadChip Kit (EPIC) DNAm data can lead to a large number of false positive results under certain conditions. We further provide a detailed assessment of the consequences for the highly relevant problem of p-value inflation with subsequent false positive findings after application of the frequently used ComBat method. Using ComBat to correct for batch effects in randomly generated samples produced alarming numbers of false discovery rate (FDR) and Bonferroni-corrected (BF) false positive results in unbalanced as well as in balanced sample distributions in terms of the relation between the outcome of interest variable and the technical position of the sample during the probe measurement. Both sample size and number of batch factors (e.g. number of chips) were systematically simulated to assess the probability of false positive findings. The effect of sample size was simulated using n = 48 up to n = 768 randomly generated samples. Increasing the number of corrected factors led to an exponential increase in the number of false positive signals. Increasing the number of samples reduced, but did not completely prevent, this effect.

Removal of Scanner Effects in Covariance Improves Multivariate Pattern Analysis in Neuroimaging Data

“…While we do not observe CovBat introducing false positives in our investigation, care must be taken in implementing CovBat protecting for the outcome of interest especially in unbalanced study designs. We reiterate previous advice that analyses should be performed with and without harmonization and that associations detected only after harmonization should be interpreted with caution (Zindler et al, 2020).…”

“…Associations with covariates not included in the harmonization step can certainly be removed alongside scanner effects and recent papers have even identified situations where spurious associations can be introduced (Nygaard et al, 2016;Zindler et al, 2020). While we do not observe CovBat introducing false positives in our investigation, care must be taken in implementing CovBat protecting for the outcome of interest especially in unbalanced study designs.…”

A computational solution for bolstering reliability of epigenetic clocks: Implications for clinical trials and longitudinal tracking

“…Batch correction is a standard preprocessing step during DNA methylation analysis (Morris & Beck 2015). However, there is a balance between adequate batch correction and over-correction that can lead to false positive and false negative results (Zindler et al 2020). In many datasets there may be residual batch effects after correction that affect epigenetic clock predictions.…”

Evaluation of critical data processing steps for reliable prediction of gene co-expression from large collections of RNA-seq data

“…However, our previous study was limited to microarray data, and considered only one data normalization method and one batch correction method, ComBat (Johnson et al ., 2007). While we found that treating batch effects increased the quality of resulting gene co-expression estimates, other studies have shown that treating batch effects can also result in unwanted artifacts such as exaggerated differences between covariates in gene expression and DNA methylation data (Harper et al ., 2013; Nygaard and Rødland, 2016; Price and Robinson, 2018; Zindler et al ., 2020).…”