E2F transcription factors play critical roles in the cell cycle. Therefore, their activity is expected to reflect tumor aggressiveness and responsiveness to therapy. We scored 3905 tumors of nine breast cancer cohorts for this activity based on their gene expression for the Hallmark E2F targets gene set. As expected, tumors with a high score had an increased expression of cell proliferation-related genes. A high score was significantly associated with shorter patient survival, greater MKI67 expression, histological grade, stage, and genomic aberrations. Furthermore, metastatic tumors had higher E2F scores than the primary tumors from which they arose. Although tumors with a high score had greater infiltration by both pro- and anti-cancerous immune cells, they had an increased expression of immune checkpoint genes. Estrogen receptor (ER)-positive/human epidermal growth factor receptor 2 (HER2)-negative cancer with a high E2F score achieved a significantly higher pathological complete response (pCR) rate to neoadjuvant chemotherapy. The E2F score was significantly associated with the expression of cyclin-dependent kinase (CDK)-related genes and strongly correlated with sensitivity to CDK inhibition in cell lines. In conclusion, the E2F score is a marker of breast cancer aggressiveness and predicts the responsiveness of ER-positive/HER2-negative patients to neoadjuvant chemotherapy and possibly to CDK and immune checkpoint inhibitors.