(1) Background: The independent negative prognostic value of isolated tumor cells or micro-metastases in axillary lymph nodes has been established in triple-negative breast cancers (BC). However, the prognostic significance of pN0(i+) or pN1mi in HER2-positive BCs treated by primary surgery remains unexplored. Therefore, our objective was to investigate the impact of pN0(i+) or pN1mi in HER2-positive BC patients undergoing up-front surgery on their outcomes. (2) Methods: We retrospectively analyzed 23,650 patients treated in 13 French cancer centers from 1991 to 2013. pN status was categorized as pN0, pN0(i+), pN1mi, and pNmacro. The effect of pN0(i+) or pN1mi on outcomes was investigated both in the entire cohort of patients and in pT1a-b tumors. (3) Results: Of 1771 HER2-positive BC patients included, pN status distributed as follows: 1047 pN0 (59.1%), 60 pN0(i+) (3.4%), 118 pN1mi (6.7%), and 546 pN1 macro-metastases (30.8%). pN status was significantly associated with sentinel lymph node biopsy, axillary lymph node dissection, age, ER status, tumor grade, and size, lymphovascular invasion, adjuvant systemic therapy (ACt), and radiation therapy. With 61 months median follow-up (mean 63.2; CI 95% 61.5–64.9), only pN1 with macro-metastases was independently associated with a negative impact on overall, disease-free, recurrence-free, and metastasis-free survivals in multivariate analysis. In the pT1a-b subgroup including 474 patients, RFS was significantly decreased in multivariate analysis for pT1b BC without ACt (HR 2.365, 1.04–5.36, p = 0.039) and for pN0(i+)/pN1mi patients (HR 2.518, 1.03–6.14, p = 0.042). (4) Conclusions: Survival outcomes were not adversely affected by pN0(i+) and pN1mi in patients with HER2-positive BC. However, in the case of pT1a-b HER2-positive BC, a negative impact on RFS was observed specifically for patients with pN0(i+) and pN1mi diseases, particularly among those with pT1b tumors without ACt. Our findings highlight the importance of considering the pN0(i+) and pN1mi status in the decision-making process when discussing trastuzumab-based ACt for these patients.