Postpartum blood calcium (Ca) concentration is related to the reproduction and health of cattle. Oral calcium supplements were given to dairy cows after calving to increase blood Ca concentration and reduce the risk of hypocalcemia. However, studies have shown that oral Ca has different effects in preventing disease. The purposes of this study were (i) to conduct a meta-analysis to evaluate the expected effect of oral Ca on incidence of calving-related diseases, pregnancy risk and milk yield in dairy cows, and (ii) to make a quality assessment of these related studies. In total, 22 eligible studies were included in this review. Meta-analysis showed that oral Ca could significantly reduce the incidence of hypocalcemia (clinical hypocalcemia: relative risk (RR) = 0.67, 95% confidence interval (CI) = [0.52, 0.87]; subclinical hypocalcemia: RR = 0.81, CI = [0.72, 0.91]), and incidence of retained placenta (RR = 0.77, CI = [0.62, 0.95]), improved blood Ca concentrations: mean difference (MD) = 0.08; 95% CI = [0.04, 0.11]. For other results, the meta-analysis revealed a lack of evidence of the correlation between oral Ca and serum magnesium (Mg) / phosphorus (P) concentration (Mg: MD = −0.04; 95% CI = [−0.10, 0.02]; P: MD = 0.05; 95% CI = [−0.10, 0.21]) or incidence of other calving-related disorders (metritis: RR = 1.06, CI = [0.94, 1.19]; ketosis: RR = 1.04, CI = [0.91, 1.18]; mastitis: RR = 1.02, CI = [0.86, 1.21]; displacement of the abomasum: RR = 0.81, CI = [0.57, 1.16]) or pregnancy risk (pregnancy risk at first service: RR = 0.99, CI = [0.94, 1.05]; overall pregnancy rate: RR = 1.03, CI = [0.98, 1.08]) or milk yield (MD = 0.44; 95% CI = [−0.24, 1.13]). The distribution of the funnel plot formed by the included studies was symmetrical, and the Egger’s test had a p > 0.05, indicating that there was no significant publication bias. Sensitivity analyses results suggested that the results of meta-analysis are robust. Quality assessment of the included studies revealed that the risk of bias was focused on selection bias, performance bias, detection bias and other sources of bias, and the future research should focus on these aspects.