Head and neck squamous cell carcinomas (HNSCC) can be induced by smoking or alcohol consumption, but a growing part of cases relate to a persistent high-risk papillomavirus (HPV) infection. Viral etiology has a beneficial impact on the prognosis, which may be explained by a specific immune response. Tumor associated macrophages (TAMs) represent the main immune population of the tumor microenvironment with a controversial influence on the prognosis. In this study, the level, phenotype, and spatial distribution of TAMs were evaluated, and the expression of TAM-associated markers was compared in HPV positive (HPV+) and HPV negative (HPV−) tumors. Seventy-three formalin and embedded in paraffin (FFPE) tumor specimens were examined using multispectral immunohistochemistry for the detection of TAM subpopulations in the tumor parenchyma and stroma. Moreover, the mRNA expression of TAM markers was evaluated using RT-qPCR. Results were compared with respect to tumor etiology, and the prognostic significance was evaluated. In HPV− tumors, we observed more pro-tumorigenic M2 in the stroma and a non-macrophage arginase 1 (ARG1)-expressing population in both compartments. Moreover, higher mRNA expression of M2 markers—cluster of differentiation 163 (CD163), ARG1, and prostaglandin-endoperoxide synthase 2 (PTGS2)—was detected in HPV− patients, and of M1 marker nitric oxide synthase 2 (NOS2) in HPV+ group. The expression of ARG1 mRNA was revealed as a negative prognostic factor for overall survival of HNSCC patients.