Chemotherapy-related cognitive impairment (CRCI) is a controversial concept not much explored on colorectal cancer patients. Materials and Methods We identified eleven prospective studies: eight studies on 696 colorectal cancer patients who received chemotherapy and three studies on 346 rectal cancer patients who received neoadjuvant chemoradiotherapy. Standardized mean differences (SMDs) of neuropsychological test results and the cognitive quality-of-life scale were calculated using random effect models. A meta-regression was conducted to investigate the association between mean study population age and effect sizes. Results The association between chemotherapy and cognitive impairment was not clear in colorectal cancer patients (SMD, 0.003; 95% confidence interval,-0.080 to 0.086). However, a metaregression showed that older patients are more vulnerable to CRCI than younger patients (β=-0.016, p < 0.001). Conclusion Chemotherapy has an overall positive negligible effect size on the cognitive function of colorectal patients. Age is a significant moderator of CRCI.