Background Late antenatal care initiation is linked to a higher risk of maternal death. Women who do not start ANC at an early stage may experience the effects of pregnancy-related health difficulties, as well as long-term health issues and pregnancy complications. Therefore, our study aimed to determine the prevalence of late initiation of antenatal care and associated factors among pregnant women in Jimma Zone public Hospitals. Methods A facility-based cross-sectional study design was employed in Jimma zone public hospitals from February 1 up to 30 March 2020 and 409 pregnant women were participated in the study by using a systematic random sampling method. Structured questionnaire was used to collect data that contain socio demographic variables, socio cultural variables, pregnancy related factors and predisposing factor related variables. The data was entered into EPI data version 3.1 and exported to SPSS version 20 for statistical analysis. Binary and multivariable logistic regression analysis were performed by using 95%CI and significance was declared at P < 0.05. Result Forty-eight percent of pregnant women were initiated their first ANC late. Primary education (AOR = 0.242; 95% CI, 0.071–0.828) and college diploma and above was (AOR = 0.142; 95% CI, 0.040- 0.511), mothers with an unplanned pregnancy (AOR = 11.290; 95%CI, 4.109–31.023), time taken to arrive the health facility greater than sixty (60) minutes (AOR = 8.285; 95% CI, 2.794–24.564) and inadequate knowledge about ANC service (AOR = 4.181; 95%CI, 1.693–10.348) were associated with late first Antenatal care initiating. Conclusion The prevalence of late initiation of ANC still remains a major public health concern in the study area. Level of education, unplanned pregnancy, distance from house to health facility, and lack of understanding about ANC services were all found to be significant variables in late ANC starting. As a result, healthcare workers can provide ongoing health education on the need of starting antenatal care visits early to avoid unfavorable pregnancy outcomes by considering all identified factors.