Alzheimer's disease (AD) is a multifactorial neurodegenerative disorder that is influenced by genetic and environmental risk factors, such as inheritance of ε4 allele of APOE (APOE4), sex and diet. Here, we examined the effect of high fat diet (HFD) on amyloid pathology and expression profile in brains of AD model mice expressing human APOE isoforms (APP/E3 and APP/E4 mice). APP/E3 and APP/E4 mice were fed HFD or Normal diet for 3months. We found that HFD significantly increased amyloid plaques in male and female APP/E4, but not in APP/E3 mice. To identify differentially expressed genes and gene-networks correlated to diet, APOE isoform and sex, we performed RNA sequencing and applied Weighted Gene Co-expression Network Analysis. We determined that the immune response network with major hubs Tyrobp/DAP12, Csf1r, Tlr2, C1qc and Laptm5 correlated significantly and positively to the phenotype of female APP/E4-HFD mice. Correspondingly, we found that in female APP/E4-HFD mice, microglia coverage around plaques, particularly of larger size, was significantly reduced. This suggests altered containment of the plaque growth and sex-dependent vulnerability in response to diet. The results of our study show concurrent impact of diet, APOE isoform and sex on the brain transcriptome and AD-like phenotype.