Objective:The quest for epilepsy biomarkers is on the rise. Variables with statistically significant group-level differences are often misinterpreted as biomarkers with sufficient discriminative power. This study aimed to demonstrate the relationship between significant group-level differences and a variable's power to discriminate between individuals.
Methods:We simulated normal-distributed datasets from hypothetical populations with varying sample sizes (25-800), effect sizes (Cohen's d: .25-2.50), and variability (standard deviation: 10-35) to assess the impact of these parameters on significance and discriminative power. The simulation data were illustrated by assessing the discriminative power of a potential real-case biomarker-the EEG beta band power-to diagnose generalized epilepsy, using data from 66 children with generalized epilepsy and 385 controls. Additionally, we evaluated recently reported epilepsy biomarkers by comparing their effect sizes to our simulation-derived effect size criterion.Results: Group size affects significance but not discriminative power.Discriminative power is much more related to variability and effect size. Our real data example supported these simulation results by demonstrating that group-level significance does not translate, one to one, into discriminative power. Although we found a significant difference in the beta band power between children with and without epilepsy, the discriminative power was poor due to a small effect size. A Cohen's d of at least 1.25 is required to reach good discriminative power in univariable prediction modeling. Slightly over 60% of the biomarkers in our literature search met this criterion.Significance: Rather than statistical significance of group-level differences, effect size should be used as an indicator of a variable's biomarker potential. The minimal required effects size for individual biomarkers-a Cohen's d of 1.25-is large.This calls for multivariable approaches, in which combining multiple variables with smaller effect sizes could increase the overall effect size and discriminative power.