There are few data from sub-Saharan Africa on the virologic outcomes associated with second-line ART based on protease inhibitors or dolutegravir (DTG). We compared viral load (VL) suppression among people living with HIV (PLWH) on atazanavir (ATV/r)- or dolutegravir (DTG)-based second-line ART with PLWH on efavirenz (EFV)-based first-line ART. We analyzed data from the electronic medical records system of Newlands Clinic in Harare, Zimbabwe. We included patients aged >=12 years when commencing first-line EFV-based ART or switching to second-line DTG- or ATV/r-based ART with >=24 weeks follow-up after start or switch. We computed suppression rates (HIV VL <50 copies/mL) at weeks 12, 24, 48, 72, and 96 and estimated the probability of VL suppression by treatment regimen, time since start/switch of ART, sex, age, and CD4 cell count (at start/switch) using logistic regression in a Bayesian framework. We included 7013 VL measurements of 1049 PLWH (61% female) initiating first-line ART and 1114 patients (58% female) switching to second-line ART. Among those switching, 872 (78.3%) were switched to ATV/r and 242 (21.7%) to DTG. VL suppression was lower in second-line ART patients than first-line ART patients, except at week 12, when those on DTG showed higher suppression than those on EFV (aOR 2.10, 95%-credible interval [CrI] 1.48-3.00) and ATV/r-based regimens (aOR 1.87, 95%-CrI 1.32-2.71). In weeks >=48, first-line patients had around 3 times the odds of VL suppression compared to second-line patients. There was no evidence of a difference between DTG and ATV/r for follow-up times >24 weeks in PLWH on second-line. Second-line ART patients had lower VL suppression rates than those receiving first-line ART, and there was no difference in suppression between DTG- and ATV/r-based second-line ART >6 months after switching. Virologic monitoring and adherence support remain essential to prevent second-line treatment failure in settings with limited treatment options.