Human leukocyte antigen (HLA)-DR4/HLA-DRB1*04 has been reported to be a risk factor for Vogt-Koyanagi-Harada disease (VKH) with various strength of association. Its sub-alleles were also found to be associated with VKH. However the results were inconsistent. In this study, we systematically searched the related literature, pooled the odds ratios (ORs) and 95% confidence interval (CI) of association of HLA-DR4/HLA-DRB1*04 or its sub-alleles with VKH from individual studies, and explored the potential source of heterogeneity. A total of 1853 VKH patients and 4164 controls from 21 articles were included in this meta-analysis. The pooled OR of association of HLA-DR4/HLA-DRB1*04 and VKH was 8.42 (95% CI: 5.69–12.45). There were significant heterogeneity (I2 = 71%). Subgroup analysis indicated that ethnicity was the source of heterogeneity (all I2 = 0, ORs ranged from 2.09–13.69 in subgroups). The sub-alleles, HLA-DRB1*0404 (OR = 2.57), 0405 (OR = 10.31) and 0410 (OR = 6.52) increased the risk of VKH; 0401 (OR = 0.21) protected VKH; while other sub-alleles were not associated with VKH. Our meta-analysis confirmed the association between VKH and HLA-DR4/DRB1*04, found the strength of association is different in different ethnic groups, and identified HLA-DRB1*0404, 0405 and 0410 as risk sub-alleles while 0401 as protective sub-allele.