Background: Worldwide, we are observing a rising prevalence of dementia and mild cognitive impairments that often co-occur with the heightened incidence of non-communicable diseases in the elderly. It is suggested that type 2 diabetes and defects in glucose metabolism might predispose to poorer cognitive performances and more rapid decline in old age. Methods: to address existing knowledge gaps in this area, we systematically reviewed the literature to identify whether patients with type 2 diabetes (T2DM) and pre-diabetes are at a higher risk of poorer cognitive performance, and whether the risk (if any) might affect specific cognitive abilities. We concentrated the review on elderly individuals (65 years or older) at intake. In total, 3251 original articles were retrieved, of which 17 met our inclusion and quality control criteria, which comprised 12 structured questions used to define the articles. Results: 11 of 17 studies found a statistically significant decline in cognition among individuals who had T2DM or pre-diabetes compared to their non-diabetic counterparts. The association between diabetes and cognitive decline was not always clear, and the extent of the cognitive tests used seemed to have the greatest effect on the results. Conclusion: Focusing on a population age 65 years and over, we found insufficient evidence to support an association between pre-diabetes stages and mild cognitive impairment. However, there is consistent evidence to support diabetes as an independent risk factor for low cognitive ability in the elderly. Finally, we found insufficient evidence to support effect of T2DM on distinct cognitive ability due to the scarcity of comparable findings.