Background: White matter hyperintensities (WMHs) are a risk factor for stroke. Their etiology is considered to be cerebral microvascular abnormality. However, the association between WMHs and arteriosclerosis is not yet clear. The aim of this hospital-based cohort study was to identify the arteriosclerotic characteristics associated with WMHs. Methods: We cross-sectionally included 240 consecutive patients with no history of stroke. We measured the brachial-ankle pulse wave velocity (baPWV), ankle brachial pressure index, and intima-media thickness of the common carotid artery, and we performed magnetic resonance brain imaging. WMHs were defined as periventricular hyperintensity (Fazekas grade ≧3) and/or separate deep white matter hyperintense signals (Fazekas grade ≧2). We determined the prevalence of WMHs, silent brain infarction (SBI), hypertension, hypercholesterolemia, diabetes mellitus, ischemic heart disease, and smoking. We compared 2 groups of patients, defined by the presence or absence of WMHs, using multiple logistic regression analyses. Results: In multivariable analysis, SBI (OR 3.38; 95% CI 1.52–7.72), hypertension (OR 2.23; 95% CI 1.03–5.15), female sex (OR 1.95; 95% CI 1.03–3.76), baPWV (OR 1.12; 95% CI 1.02–1.23), and age (OR 1.09; 95% CI 1.04–1.14) were independently associated with WMHs. Conclusions: An increased baPWV is associated with WMHs. Management of increased baPWV may help to prevent the progression of WMHs and stroke.