Background: Breast cancer subtypes are known to have different metastatic recurrence sites. Distant metastases are often observed during the post-operative course in patients with human epidermal growth factor receptor 2 (HER2)-enriched breast cancer and triple-negative breast cancer, but are relatively rare in those with hormone receptor-positive and HER2-negative (HR+/HER2-) breast cancer. Tumor-infiltrating lymphocytes (TILs) serve as an index to monitor tumor immune microenvironment and may possibly predict the prognosis and therapeutic effect in breast cancer. This study aimed to investigate the correlation between TIL density and recurrence site in HR+/HER2- breast cancer.Methods: Four-hundred and seventy-one patients with HR+/HER2- breast cancer underwent surgery as the first treatment and received adjuvant endocrine therapy (except adjuvant chemotherapy) at the Osaka City University Hospital from April 2007 to October 2015. Needle biopsy specimens were obtained from all patients to evaluate tumor morphology and examine TILs. Morphological assessment was conducted using conventional hematoxylin and eosin staining.Results: Breast cancer recurrence was observed in 42 patients. In patients with no TIL density, local recurrence was significantly less (p = 0.022), while distant metastases were significantly more (p = 0.015) compared to those in patients with TIL density. Therefore, for the prediction of distant metastases in HR+/HER2- breast cancer without chemotherapy, TILs could be used as predictors in univariate analysis (p = 0.015, odds ratio [OR] = 0.127), although not as independent factors (p = 0.285, OR = 0.144).Conclusions: Our findings indicate that TILs may predict distant metastatic recurrence in stages I–II of HR+/HER2- breast cancer.