The evolution of reproductive barriers is fundamental to the formation of new species and can help us understand the diversification of life on Earth. These reproductive barriers often take the form of hybrid incompatibilities, where genes derived from two different species no longer interact properly. Theory predicts that incompatibilities involving multiple genes should be common and that rapidly evolving genes will be more likely to cause incompatibilities, but empirical evidence has lagged behind these predictions. Here, we describe a mitonuclear incompatibility involving three genes within respiratory Complex I in naturally hybridizing swordtail fish. Individuals with specific mismatched protein combinations fail to complete embryonic development while those heterozygous for the incompatibility have reduced function of Complex I and unbalanced representation of parental alleles in the mitochondrial proteome. We localize the protein-protein interactions that underlie the incompatibility and document accelerated evolution and introgression in the genes involved. This work thus provides a precise characterization of the genetic architecture, physiological impacts, and evolutionary origin of a multi-gene incompatibility impacting naturally hybridizing species.

While hybridization between species is increasingly appreciated to be a common occurrence, little is known about the forces that govern the subsequent evolution of hybrid genomes. We considered this question in three independent, naturally-occurring hybrid populations formed between swordtail fish species Xiphophorus birchmanni and X. malinche. To this end, we built a fine-scale genetic map and inferred patterns of local ancestry along the genomes of 690 individuals sampled from the three populations. In all three cases, we found hybrid ancestry to be more common in regions of high recombination and where there is linkage to fewer putative targets of selection. These same patterns are also apparent in a reanalysis of human-Neanderthal admixture. Our results lend support to models in which ancestry from the “minor” parental species persists only where it is rapidly uncoupled from alleles that are deleterious in hybrids, and show the retention of hybrid ancestry to be at least in part predictable from genomic features. Our analyses further indicate that in swordtail fish, the dominant source of selection on hybrids stems from deleterious combinations of epistatically-interacting alleles.One sentence summaryThe persistence of hybrid ancestry is predictable from local recombination rates, in three replicate hybrid populations as well as in humans.

Natural hybridization events provide unique windows into the barriers that keep species apart as well as the consequences of their breakdown. Here we characterize hybrid populations formed between the northern swordtail fish Xiphophorus cortezi and X. birchmanni from collection sites on two rivers. We develop sensitive and accurate local ancestry calling for this system based on low coverage whole genome sequencing. Strikingly, we find that hybrid populations on both rivers consist of two genetically distinct subpopulations: a cluster of nearly pure X. birchmanni individuals and one of phenotypically intermediate hybrids that derive ~85-90% of their genome from X. cortezi. Simulations and empirical data suggest that at both sites initial hybridization occurred ~150 generations ago, with little evidence for contemporary gene flow between subpopulations, likely due to strong assortative mating. The patterns of population structure uncovered here mirror those seen in hybridization between X. birchmanni and its sister species, X. malinche. Future comparisons will provide a window into the repeatability of the outcomes of hybridization not only across independent hybridization events between the same species but also across distinct species pairs.