Objectives To explore associations of dietary diversity with cognitive function among Chinese elderly. Design This cross-sectional study was conducted in 2011–2012, data were analyzed using multiple linear regression and logistic regression models. Setting community-based setting in the 23 provinces in China. Subjects 8,571 elderly participants, including 2984 younger elderly aged 65–79 and 5587 oldest old aged 80+ participated in this study. Measurement Intake frequencies of food groups was collected and dietary diversity (DD) was assessed based on the mean of DD score. Cognitive function was assessed using the Chinese version of Mini-Mental State Examination (MMSE), and cognitive impairment was defined using education-based cutoffs. Information about socio-demographics, lifestyles, resilience and health status was also collected. Results Poor dietary diversity was significantly associated with cognitive function, with β (95% CI) of −0.11(−0.14, −0.08) for − log 31-MMSE score) and odds ratio (95% CI) of 1.29 (1.14, 1.47) for cognitive impairment. Interaction effect of age with DD was observed on cognitive impairment (P interaction=0.018), but not on −log 31-MMSE score) (P interaction=0.08). Further separate analysis showed that poor DD was significantly associated with increased risk of cognitive impairment in the oldest old (p<0.01), with odds ratio (95% CI) of 1.34 (1.17, 1.54), while not in the younger elderly (p>0.05), with OR (95% CI) being 1.09 (0.80, 1.47) in the fully adjusted model. Similar results were obtained when DD was categorized into four groups. Conclusions Poor dietary diversity was associated with worse global cognitive function among Chinese elderly, and particularly for the oldest old. This finding would be very meaningful for prevention of cognitive impairment.