Background The relationship between vitamin D and diabetes peripheral neuropathy (DPN) is not consistent among epidemiologic studies. Thus, we aimed to investigate this relationship in different age groups. Methods In this cross‐sectional study, 1461 patients admitted to the Department of Endocrinology at Xinhua Hospital from June 2016 to September 2017 were divided into three age groups: a Youth group (24‐44 years, n = 127), a Middle‐age group (45‐64 years, n = 779), and an Elderly group (≥65 years, n = 555). Basic information and laboratory results were collected from medical records. Results Among the patients, 32.72% had DPN, with 12.59% in the Youth group, 33.63% in the Middle‐age group, and 36.04% in the Elderly group. For the total sample and the Elderly group, serum 25(OH)D concentrations in DPN patients were lower than in non‐DPN patients (P < .05). The results of multivariate logistic regression indicated a low vitamin D concentration to be a risk factor for DPN in the Elderly group (P < .05), but such relationship was not found in the Youth or Middle‐age groups. Moreover, according to ROC analysis, a serum 25(OH)D level < 34.87 nmol/L suggests the occurrence of DPN in elderly patients with type 2 diabetes (P < .001). Conclusions This study is the first to report that a low vitamin D level is associated with DPN in diabetic patients over 65 years of age and might be used as a predictor of DPN in this population. The interaction between vitamin D and age in the development of DPN and its underlying mechanisms need to be further explored.