Modern Humans Did Not Admix with Neanderthals during Their Range Expansion into Europe

Abstract

The process by which the Neanderthals were replaced by modern humans between 42,000 and 30,000 before present is still intriguing. Although no Neanderthal mitochondrial DNA (mtDNA) lineage is found to date among several thousands of Europeans and in seven early modern Europeans, interbreeding rates as high as 25% could not be excluded between the two subspecies. In this study, we introduce a realistic model of the range expansion of early modern humans into Europe, and of their competition and potential admixture with local Neanderthals. Under this scenario, which explicitly models the dynamics of Neanderthals' replacement, we estimate that maximum interbreeding rates between the two populations should have been smaller than 0.1%. We indeed show that the absence of Neanderthal mtDNA sequences in Europe is compatible with at most 120 admixture events between the two populations despite a likely cohabitation time of more than 12,000 y. This extremely low number strongly suggests an almost complete sterility between Neanderthal females and modern human males, implying that the two populations were probably distinct biological species.

Haplotype Trees and Modern Human Origins

“…Currat and Excoffier (2004) produced the smallest estimate of the amount of female-mediated gene flow compatible with the mtDNA, assuming neutrality, with an upper limit of 0.1%. However, within population genetics, gene flow is regarded as significant if the product of the population size times the migration rate is equal to or greater than one (Crow and Kimura, 1970).…”

“…This pattern was interpreted as meaning that there was no or extremely little interbreeding between Neanderthals and their more anatomically modern contemporaries, and that Neanderthals perhaps represent a distinct species from Homo sapiens (Krings et al, 2000;Caramelli et al, 2003;Knight, 2003;Currat and Excoffier, 2004;Serre et al, 2004;Lalueza-Fox et al, 2005). However, there are difficulties with these conclusions.…”

“…Third, the small sample sizes preclude the ability to dismiss significant amounts of gene flow between Neanderthals and moderns (Nordborg, 1998;Wall, 2000;Relethford, 2001a;Currat and Excoffier, 2004;Pearson, 2004). Currat and Excoffier (2004) produced the smallest estimate of the amount of female-mediated gene flow compatible with the mtDNA, assuming neutrality, with an upper limit of 0.1%.…”

“…Thus, if Neanderthals had a population size equal to or greater than 1,000, there could be biologically significant gene flow. The simulations used by Currat and Excoffier (2004) to obtain the limit of 0.1% started with a Neanderthal population size of 35,000. Hence, even this conservative evaluation indicates that a biologically significant, female-mediated gene flow of neutral mtDNA cannot be rejected.…”

Genetic evidence and the modern human origins debate

“…The question of Neandertal ancestry has now become a question of the probability of haplotype survival, and as such is dependent on the parameters used to determine these probabilities, including population sizes, rates of gene flow and duration of contact between Neandertals and modern humans in Europe. Studies to date have suggested that the total possible amount of Neandertal ancestry for living humans was not very large, but it has not yet been possible to distinguish between a model of some limited Neandertal ancestry and a model of no Neandertal ancestry (Currat and Excoffier, 2004;Serre et al, 2004).…”