A new likelihood estimator and its comparison with moment estimators of individual genome-wide diversity

The inbreeding coefficient of an individual, F, is one of the central parameters in population genetics theory. It has found important applications in evolutionary biology, conservation and ecology, such as the study of inbreeding depression. In the absence of detailed and reliable pedigree records, researchers have developed quite a few estimators to estimate F or the genome-wide homozygosity from genetic marker data. The statistical properties and comparative performances of these metrics are rarely known, however, which impedes an informed choice of the most appropriate one in practical applications. In this investigation, I propose a new likelihood F estimator that makes efficient use of marker information and takes into account of allelic dropouts, null alleles and prior knowledge of inbreeding. I compare the likelihood estimator with three moment estimators of F and three metrics of genomic homozygosity (or heterozygosity) by analysing both simulated and empirical datasets. It is shown that the likelihood estimator invariably outperforms the other estimators and metrics across all datasets analysed. For a typical dataset in heterozygosity-fitness correlation studies involving 10-20 microsatellites and 50 individuals, the correlation between the likelihood estimator and F (the simulated true inbreeding coefficient) is about 8 ～ 35% higher than that between the moment estimators and F. A frequently applied metric, multilocus heterozygosity (MLH), and an F estimator based on the consideration of the proportion of alleles in homozygous conditions, F R'), are shown to have particularly poor performances. The low correlation between MLH and fitness traits, which is widely observed in numerous empirical studies, might be partially caused by the adoption of this inefficient estimator of genomic inbreeding.