The intricate genus Rhinanthus L. contains many taxonomically unresolved taxa. The assessment of genetic diversity may clarify the differentiation and taxonomic relationships between species and help to determine their mating system, which is important for understanding the future prospects of populations and species. Isozymes were used to assess the mating system, genetic diversity, and differentiation in R. rumelicus, R. osiliensis, R. wagneri, R. alectorolophus, and R. angustifolius from sect. Cleistolemus and R. minor from sect. Rhinanthus. Genetic diversity at species level was found to be higher in widespread R. angustifolius and R. rumelicus compared with insular endemic R. osiliensis. Positive inbreeding coefficient revealed dominant inbreeding in endemic R. osiliensis and R. wagneri. We also detected a significant departure from Hardy-Weinberg equilibrium in some populations of R. rumelicus and R. angustifolius. The common R. alectorolophus and R. minor and several populations of R. rumelicus and R. angustifolius displayed a mixed mating mode. Six studied species divided into three distinct clusters. Rhinanthus alectorolophus was the most differentiated taxon among the investigated species. The species of sect. Cleistolemus formed two distinct groups. The first cluster consisted only of R. alectorolophus while another comprised R. rumelicus, R. osiliensis, R. wagneri, and R. angustifolius, indicating their close affinity. Thus, our isozyme results call into question the monophyly of section Cleistolemus and the taxonomic position of R. alectorolophus.