Tumor-associated macrophages (TAMs) is a key element in the breast tumor microenvironment. CD163 and CD206 have been utilized for TAM identification, but the clinical implications of TAMs identified by these markers have not been thoroughly explored. This study conducted a comparative analysis of CD163 and CD206 TAMs using digital image analysis, focusing on their spatial distribution and prognostic significance in relation to tumor-infiltrating lymphocytes (TILs). Distinct clinico-pathological and prognostic characteristics were noted between the two types of TAMs. CD163 TAMs were linked to high-grade tumors (p = 0.006), whereas CD206 TAMs were associated with a higher incidence of nodal metastasis (p = 0.033). CD206 TAMs were predominantly found in the stroma, with more cases being stromal CD206-high (sCD206-high) than tumoral CD206-high (tCD206-high) (p = 0.024). Regarding prognostication, patients stratified according to stromal and tumoral densities of CD163 showed different disease-free survival (DFS) time. Specifically, those that were sCD163-low but tCD163-high exhibited the poorest DFS (chi-square = 10.853, p = 0.013). Furthermore, a high sCD163-to-stromal-TILs ratio was identified as an independent predictor of unfavorable survival outcomes (DFS: HR = 3.477, p = 0.018). The spatial distribution and interactions with TILs enhanced the prognostic value of CD163 TAMs, while CD206 TAMs appeared to have limited prognostic utility in breast cancer cases.