Maximum likelihood estimation of individual inbreeding coefficients and null allele frequencies

Abstract

In this paper, we developed and compared several expectation-maximization (EM) algorithms to find maximum likelihood estimates of individual inbreeding coefficients using molecular marker information. The first method estimates the inbreeding coefficient for a single individual and assumes that allele frequencies are known without error. The second method jointly estimates inbreeding coefficients and allele frequencies for a set of individuals that have been genotyped at several loci. The third method generalizes the second method to include the case in which null alleles may be present. In particular, it is able to jointly estimate individual inbreeding coefficients and allele frequencies, including the frequencies of null alleles, and accounts for missing data. We compared our methods with several other estimation procedures using simulated data and found that our methods perform well. The maximum likelihood estimators consistently gave among the lowest root-mean-square-error (RMSE) of all the estimators that were compared. Our estimator that accounts for null alleles performed particularly well and was able to tease apart the effects of null alleles, randomly missing genotypes and differing degrees of inbreeding among members of the datasets we analysed. To illustrate the performance of our estimators, we analysed previously published datasets on mice (Mus musculus) and white-tailed deer (Odocoileus virginianus).

Estimating inbreeding coefficients from NGS data: Impact on genotype calling and allele frequency estimation

“…In all scenarios examined, the new method presented here largely outperformed the original Hall et al (2012) method based on called genotypes, especially in cases of extremely low coverage, small sample sizes, and high error rates. Because the original method has been previously shown to outperform other methods based directly on genotypes (Hall et al 2012), the advantage of our method, in the presence of genotype uncertainty, should extend to these methods as well.…”

“…Several methods have been published to infer per-individual inbreeding coefficients (Vogl et al 2002;Leutenegger et al 2003;Wang et al 2006;Moltke et al 2011), but all were designed for genotype (marker) data. Although all present slight improvements, Hall et al (2012) recently incorporated most features into a single EM algorithm and showed that it outperformed previous methods. Here, we have modified this algorithm to accommodate for NGS data, as well as an approximate EM algorithm that can help speed up convergence.…”

“…Because the original method has been previously shown to outperform other methods based directly on genotypes (Hall et al 2012), the advantage of our method, in the presence of genotype uncertainty, should extend to these methods as well. Our analyses of simulated data further show that failing to use a correct prior can greatly affect downstream analyses.…”

“…Estimates of this parameter can shed light into the species' mating system and past history (domestication), as well as be used as a prior to improve genotype-calling algorithms. To this end, we extended a recently published algorithm by Hall et al (2012) to estimate per-individual inbreeding coefficients directly from genotype likelihoods.…”

Estimating Relatedness in the Presence of Null Alleles

“…These two effects can cause an over-or underestimation of the relatedness coefficient. Some methods can estimate the frequency of null alleles (e.g., Brookfield 1996; Vogl et al 2002;van Oosterhout et al 2004;Kalinowski et al 2006;Hall et al 2012;Dabrowski et al 2013). We thus model the probability that a genotype pair (Anderson and Weir 2007) or a genotype pattern or a genotype pair pattern is observed under the assumption that the true frequencies of alleles are already known.…”