Matrix game between full siblings in Mendelian populations

Abstract

We demonstrate that static evolutionary stability implies the stability of the corresponding interior equilibrium point in genotype dynamics, while a certain form of monotonicity ensures the global stability of a homozygote state. We apply our findings to familial selection in a diploid, panmictic population, where the survival rates of siblings within monogamous and exogamous families are determined by a matrix game, and the behavior is uniquely determined by an autosomal recessive-dominant or intermediate allele pair. We provide conditions for the existence of each homozygote. In our numerical investigations of the Prisoner's Dilemma between siblings, we distinguish two scenarios: cooperation (collaborating case) or defector-cooperator strategy pair (alternating case) that maximizes the siblings' survival rates. Based on the stability of the pure cooperator and defector states, we provide a potential classification of genotype dynamics. We find that the pure cooperator population cannot fixate in the alternating case. However, in the collaborating case, fixation is possible but not necessary, since bistability, coexistence, moreover, the monostable fixation of pure defector state can also occur due to the interplay between the phenotypic payoff function and the genotype-phenotype mapping, which collectively determine the outcome of natural selection. In donation game, the classical Hamilton’s rule implies the fixation of the cooperation in all considered genotype-phenotype mappings.