Disappointing results from clinical trials designed to delay structural brain decline and the accompanying increase in risk for dementia in older adults have precipitated a shift in testing promising interventions from late in life toward midlife before irreversible damage has accumulated. This shift, however, requires targeting midlife biomarkers that are associated with clinical changes manifesting only in late life. Here we explored possible links between one putative biomarker, distributed integrity of brain white matter, and two intervention targets, cardiovascular fitness and healthy lifestyle behaviors, in midlife. At age 45, fractional anisotropy (FA) derived from diffusion weighted MRI was used to estimate the microstructural integrity of distributed white matter tracts in a population-representative birth cohort. Age-45 cardiovascular fitness (VO2Max; N = 801) was estimated from heart rates obtained during submaximal exercise tests; age-45 healthy lifestyle behaviors were estimated using the Nyberg Health Index (N = 854). Ten-fold cross-validated elastic net predictive modeling revealed that estimated VO2Max was modestly associated with distributed FA. In contrast, there was no significant association between Nyberg Health Index scores and FA. Our findings suggest that cardiovascular fitness levels, but not healthy lifestyle behaviors, are associated with the distributed integrity of white matter in the brain in midlife. These patterns could help inform future clinical intervention research targeting ADRDs.