Background Healthcare-associated illnesses, of which surgical site infection is the most common are significant causes of morbidity and mortality. Therefore, this study aimed to determine the prevalence and root causes of surgical site infections in public versus private hospitals in Ethiopia. Methods An institution based retrospective observational cohort study was conducted among patients who underwent surgical procedures at public and private health facilities from March 15 to April 15, 2018. Samples were selected by the simple random sampling technique, and data extracted from the patient’s medical chart, operation, and anesthesia notes. Data were entered using Epi info version 7 and analyzed using STATA 14. Binary logistic regression was fitted to identify factors associated with surgical site infections in private and public hospitals. Crude and adjusted odds ratios (OR) with a 95% confidence interval (CI) were computed to assess the strength of associations. Variables with a p -value less than 0.05 in the multivariable logistic regression model considered as significant predictors of surgical site infections. Result The overall prevalence of surgical site infections was 9.9% (95%CI: 7.8, 12.5). The prevalence of the infections was higher in procedures performed in public hospitals (13.4%) compared to private hospitals (6.5%). Rural residence (AOR = 0.13, 95%CI: 0.034 0.55), clean-contaminated and dirty wound (AOR = 12.81, 95%CI: 4.42 37.08) were significant predictors of the infections in private hospitals. Similarly, clean-contaminated and dirty wounds (AOR = 4.37, 95%CI: 1.88 10.14), length of hospital stay≥6 days (AOR = 2.86, 95%CI: 1.11 7.33), and surgical operation time of over 1 h (AOR = 15.24, 95%CI: 4.48 51.83) were such factors in public hospitals. Conclusion The prevalence of surgical site infections was high, and significant differences were also observed between public and private hospitals. Clean-contaminated and dirty wounds, prolonged operation, and length of hospital stay were predictors of surgical site infections among patients in public hospitals, whereas clean-contaminated wound and rural dwellings were predicted the infections among patients operated in the private hospital.