Aim: To test the association between cognitive performance and APOE genotype, and to assess potential modifications of this association by sociodemographic and neuroanatomical factors in a sample of 74 healthy elders. Methods: Firstly, we explored the isolated role of the APOE ɛ4 genotype (i.e., APOE4) in different neuropsychological tests, and then the effects of its interaction with sociodemographic (i.e., age, gender, and educational level) and neuroanatomical (i.e., hippocampal volumes) variables. Subsequently, we performed the same analyses after dividing the sample into two subgroups according to their Mini-Mental State Examination scores (control-high group ≥29 and control-low group < 29). Results: In the whole group, APOE4 carriers exhibited a significantly poorer execution in several cognitive domains including global cognitive functioning, episodic memory, verbal fluency, and naming. This effect was more noticeable in older and less educated subjects. The separated analyses revealed that APOE4 carriers in the control-low group exhibited lower scores in global cognitive functioning and episodic memory, while no effects were observed in the control-high group. Neither gender nor hippocampal volumes showed a significant interaction effect with APOE genotype. Conclusions: Current results point out that APOE4 genotype influences healthy aged cognition, although factors such age or educational attainment seem to modulate its effects.