Older adults can experience glucose metabolism dysfunction, and although manganese may help regulate glucose metabolism, there is little information regarding this association among older people. This cross-sectional study included 2402 Chinese adults who were ≥60 years old in 2013 (Tianjin, China), and evaluated the associations of serum manganese with prediabetes and diabetes. Serum manganese levels were measured using inductively coupled plasma mass spectrometry. Multivariable logistic regression models were used to evaluate the sex-specific associations of manganese levels with diabetes and prediabetes after adjusting for confounding factors (age, sex, life style factors, and health status). Based on the WHO criteria, prediabetes was observed in 15.1% of men and 13.4% of women, while diabetes was observed in 30.0% of men and 34.4% of women. In the final model, the odds ratios (95% confidence interval) for prediabetes according to manganese quartile were 1.000, 0.463 (0.269–0.798), 0.639 (0.383–1.065), and 0.614 (0.365–1.031) among men and 1.000, 0.773 (0.498–1.200), 0.602 (0.382–0.947), and 0.603 (0.381–0.953) among women (p for trend = 0.134 and 0.015, respectively). The lowest prevalence of diabetes among men occurred at a moderate range of serum manganese (p < 0.05). Therefore, appropriate serum manganese levels may help prevent and control prediabetes and diabetes.