A diverse set of driver genes, such as regulators of DNA methylation, RNA splicing, and chromatin remodeling, have been associated with pre-malignant clonal expansion of hematopoietic stem cells (HSCs). The factors mediating expansion of these mutant clones remain largely unknown, partially due to a paucity of large cohorts with longitudinal blood sampling. To circumvent this limitation, we developed and validated a method to infer clonal expansion rate from single timepoint data called PACER (passenger-approximated clonal expansion rate). Applying PACER to 5,071 persons with clonal hematopoiesis accurately recapitulated the known fitness effects due to different driver mutations. A genome-wide association study of PACER revealed that a common inherited polymorphism in the TCL1A promoter was associated with slower clonal expansion. Those carrying two copies of this protective allele had up to 80% reduced odds of having driver mutations in TET2, ASXL1, SF3B1, SRSF2, and JAK2, but not DNMT3A. TCL1A was not expressed in normal or DNMT3A-mutated HSCs, but the introduction of mutations in TET2 or ASXL1 by CRISPR editing led to aberrant expression of TCL1A and expansion of HSCs in vitro. These effects were abrogated in HSCs from donors carrying the protective TCL1A allele. Our results indicate that the fitness advantage of multiple common driver genes in clonal hematopoiesis is mediated through TCL1A activation. PACER is an approach that can be widely applied to uncover genetic and environmental determinants of pre-malignant clonal expansion in blood and other tissues.