Multiple merger gene genealogies in two species: Monophyly, paraphyly, and polyphyly for two examples of Lambda coalescents

Probabilities of monophyly, paraphyly, and polyphyly of two-species gene genealogies are computed for modest sample sizes and compared for two different Λ coalescent processes. Coalescent processes belonging to the Λ coalescent family admit asynchronous multiple mergers of active ancestral lineages. Assigning a timescale to the time of divergence becomes a central issue when different populations have different coalescent processes running on different timescales. Clade probabilities in single populations are also computed, which can be useful for testing for taxonomic distinctiveness of an observed set of monophyletic lineages. The coalescence rates of multiple merger coalescent processes are functions of coalescent parameters. The effect of coalescent parameters on the probabilities studied depends on the coalescent process, and if the population is ancestral or derived. The probability of reciprocal monophyly tends to be somewhat lower, when associated with a Λ coalescent, under the null hypothesis that two groups come from the same population. However, even for fairly recent divergence times, the probability of monophyly tends to be higher as a function of the number of generations for coalescent processes that admit multiple mergers, and is sensitive to the parameter of one of the example processes.     

Ancestral processes in population genetics-the coalescent

A special stochastic process, called the coalescent, is of fundamental interest in population genetics. For a large class of population models this process is the appropriate tool to analyse the ancestral structure of a sample of n individuals or genes, if the total number of individuals in the population is sufficiently large. A corresponding convergence theorem was first proved by Kingman in 1982 for the Wright-Fisher model and the Moran model. Generalizations to a large class of exchangeable population models and to models with overlying mutation processes followed shortly later. One speaks of the "robustness of the coalescent, as this process appears in many models as the total population size tends to infinity. This publication can be considered as an introduction to the theory of the coalescent as well as a review of the most important "convergence-to-the-coalescent-theorems. Convergence theorems are not only presented for the classical exchangeable haploid case but also for larger classes of population models, for example for diploid, two-sex or non-exchangeable models. A review-like summary of further examples and applications of convergence to the coalescent is given including the most important biological forces like mutation, recombination and selection. The general coalescent process allows for simultaneous multiple mergers of ancestral lines.     

A coalescent process with simultaneous multiple mergers for approximating the gene genealogies of many marine organisms

We describe a forward-time haploid reproduction model with a constant population size that includes life history characteristics common to many marine organisms. We develop coalescent approximations for sample gene genealogies under this model and use these to predict patterns of genetic variation. Depending on the behavior of the underlying parameters of the model, the approximations are coalescent processes with simultaneous multiple mergers or Kingman’s coalescent. Using simulations, we apply our model to data from the Pacific oyster and show that our model predicts the observed data very well. We also show that a fact which holds for Kingman’s coalescent and also for general coalescent trees–that the most-frequent allele at a biallelic locus is likely to be the ancestral allele–is not true for our model. Our work suggests that the power to detect a “sweepstakes effect” in a sample of DNA sequences from marine organisms depends on the sample size.     

Convergence to the island-model coalescent process in populations with restricted migration

In this article we apply some graph-theoretic results to the study of coalescence in a structured population with migration. The graph is the pattern of migration among subpopulations, or demes, and we use the theory of random walks on graphs to characterize the ease with which ancestral lineages can traverse the habitat in a series of migration events. We identify conditions under which the coalescent process in populations with restricted migration, such that individuals cannot traverse the habitat freely in a single migration event, nonetheless becomes identical to the coalescent process in the island migration model in the limit as the number of demes tends to infinity. Specifically, we first note that a sequence of symmetric graphs with Diaconis-Stroock constant bounded above has an unstructured Kingman-type coalescent in the limit for a sample of size two from two different demes. We then show that circular and toroidal models with long-range but restricted migration have an upper bound on this constant and so have an unstructured-migration coalescent in the limit. We investigate the rate of convergence to this limit using simulations.     

On the meaning and existence of an effective population size

We investigate conditions under which a model with stochastic demography or population structure converges to the coalescent with a linear change in timescale. We argue that this is a necessary condition for the existence of a meaningful effective population size. We find that such a linear timescale change is obtained when demographic fluctuations and coalescence events occur on different timescales. Simple models of population structure and randomly fluctuating population size are used to exemplify the ideas and provide an intuitive feel for the meaning of the conditions.     

Extending coalescent theory to autotetraploids

We develop coalescent models for autotetraploid species with tetrasomic inheritance. We show that the ancestral genetic process in a large population without recombination may be approximated using Kingman's standard coalescent, with a coalescent effective population size 4N. Numerical results suggest that this approximation is accurate for population sizes on the order of hundreds of individuals. Therefore, existing coalescent simulation programs can be adapted to study population history in autotetraploids simply by interpreting the timescale in units of 4N generations. We also consider the possibility of double reduction, a phenomenon unique to polysomic inheritance, and show that its effects on gene genealogies are similar to partial self-fertilization.     

Extending the coalescent to multilocus systems: the case of balancing selection

Natural populations are structured spatially into local populations and genetically into diverse 'genetic backgrounds' defined by different combinations of selected alleles. If selection maintains genetic backgrounds at constant frequency then neutral diversity is enhanced. By contrast, if background frequencies fluctuate then diversity is reduced. Provided that the population size of each background is large enough, these effects can be described by the structured coalescent process. Almost all the extant results based on the coalescent deal with a single selected locus. Yet we know that very large numbers of genes are under selection and that any substantial effects are likely to be due to the cumulative effects of many loci. Here, we set up a general framework for the extension of the coalescent to multilocus scenarios and we use it to study the simplest model, where strong balancing selection acting on a set of n loci maintains 2n backgrounds at constant frequencies and at linkage equilibrium. Analytical results show that the expected linked neutral diversity increases exponentially with the number of selected loci and can become extremely large. However, simulation results reveal that the structured coalescent approach breaks down when the number of backgrounds approaches the population size, because of stochastic fluctuations in background frequencies. A new method is needed to extend the structured coalescent to cases with large numbers of backgrounds.     

Coalescence 2.0: a multiple branching of recent theoretical developments and their applications

Population genetics theory has laid the foundations for genomic analyses including the recent burst in genome scans for selection and statistical inference of past demographic events in many prokaryote, animal and plant species. Identifying SNPs under natural selection and underpinning species adaptation relies on disentangling the respective contribution of random processes (mutation, drift, migration) from that of selection on nucleotide variability. Most theory and statistical tests have been developed using the Kingman coalescent theory based on the Wright‐Fisher population model. However, these theoretical models rely on biological and life history assumptions which may be violated in many prokaryote, fungal, animal or plant species. Recent theoretical developments of the so‐called multiple merger coalescent models are reviewed here (Λ‐coalescent, beta‐coalescent, Bolthausen‐Sznitman, Ξ‐coalescent). We explain how these new models take into account various pervasive ecological and biological characteristics, life history traits or life cycles which were not accounted in previous theories such as (i) the skew in offspring production typical of marine species, (ii) fast adapting microparasites (virus, bacteria and fungi) exhibiting large variation in population sizes during epidemics, (iii) the peculiar life cycles of fungi and bacteria alternating sexual and asexual cycles and (iv) the high rates of extinction‐recolonization in spatially structured populations. We finally discuss the relevance of multiple merger models for the detection of SNPs under selection in these species, for population genomics of very large sample size and advocate to potentially examine the conclusion of previous population genetics studies.     

The effects of subdivision on the genetic divergence of populations and species

Abstract. An island model of migration is used to study the effects of subdivision within populations and species on sample genealogies and on between-population or between-species measures of genetic variation. The model assumes that the number of demes within each population or species is large. When populations (or species), connected either by gene flow or historical association, are themselves subdivided into demes, changes in the migration rate among demes alter both the structure of genealogies and the time scale of the coalescent process. The time scale of the coalescent is related to the effective size of the population, which depends on the migration rate among demes. When the migration rate among demes within populations is low, isolation (or speciation) events seem more recent and migration rates among populations seem higher because the effective size of each population is increased. This affects the probability of reciprocal monophyly of two samples, the chance that a gene tree of a sample matches the species tree, and relative likelihoods of different types of polymorphic sites. It can also have a profound effect on the estimation of divergence times.     

Exact coalescent for the Wright-Fisher model

The Kingman coalescent, which has become the foundation for a wide range of theoretical as well as empirical studies, was derived as an approximation of the Wright-Fisher (WF) model. The approximation heavily relies on the assumption that population size is large and sample size is much smaller than the population size. Whether the sample size is too large compared to the population size is rarely questioned in practice when applying statistical methods based on the Kingman coalescent. Since WF model is the most widely used population genetics model for reproduction, it is desirable to develop a coalescent framework for the WF model, which can be used whenever there are concerns about the accuracy of the Kingman coalescent as an approximation. This paper described the exact coalescent theory for the WF model and develops a simulation algorithm, which is then used, together with an analytical approach, to study the properties of the exact coalescent as well as its differences to the Kingman coalescent. We show that the Kingman coalescent differs from the exact coalescent by: (1) shorter waiting time between successive coalescent events; (2) different probability of observing a topological relationship among sequences in a sample; and (3) slightly smaller tree length in the genealogy of a large sample. On the other hand, there is little difference in the age of the most recent common ancestor (MRCA) of the sample. The exact coalescent makes up the longer waiting time between successive coalescent events by having multiple coalescence at the same time. The most significant difference among various summary statistics of a coalescent examined is the sum of lengths of external branches, which can be more than 10% larger for exact coalescent than that for the Kingman coalescent. As a whole, the Kingman coalescent is a remarkably accurate approximation to the exact coalescent for sample and population sizes falling considerably outside the region that was originally anticipated.     

A logistic branching process for population genetics

A logistic (regulated population size) branching process population genetic model is presented. It is a modification of both the Wright-Fisher and (unconstrained) branching process models, and shares several properties including the coalescent time and shape, and structure of the coalescent process with those models. An important feature of the model is that population size fluctuation and regulation are intrinsic to the model rather than externally imposed. A consequence of this model is that the fluctuation in population size enhances the prospects for fixation of a beneficial mutation with constant relative viability, which is contrary to a result for the Wright-Fisher model with fluctuating population size. Explanation of this result follows from distinguishing between expected and realized viabilities, in addition to the contrast between absolute and relative viabilities.     

Sequence diversity under the multispecies coalescent with Yule process and constant population size

The study of sequence diversity under phylogenetic models is now classic. Theoretical studies of diversity under the Kingman coalescent appeared shortly after the introduction of the coalescent. In this paper we revisit this topic under the multispecies coalescent, an extension of the single population model to multiple populations. We derive exact formulas for the sequence dissimilarity of two sequences drawn at random under a basic multispecies setup. The multispecies model uses three parameters—the species tree birth rate under the pure birth process (Yule), the species effective population size and the mutation rate. We also discuss the effects of relaxing some of the model assumptions.     

Using phylochronology to reveal cryptic population histories: review and synthesis of 29 ancient DNA studies

The evolutionary history of a population involves changes in size, movements and selection pressures through time. Reconstruction of population history based on modern genetic data tends to be averaged over time or to be biased by generally reflecting only recent or extreme events, leaving many population historic processes undetected. Temporal genetic data present opportunities to reveal more complex population histories and provide important insights into what processes have influenced modern genetic diversity. Here we provide a synopsis of methods available for the analysis of ancient genetic data. We review 29 ancient DNA studies, summarizing the analytical methods and general conclusions for each study. Using the serial coalescent and a model-testing approach, we then re-analyse data from two species represented by these data sets in a common interpretive framework. Our analyses show that phylochronologic data can reveal more about population history than modern data alone, thus revealing 'cryptic' population processes, and enable us to determine whether simple or complex models best explain the data. Our re-analyses point to the need for novel methods that consider gene flow, multiple populations and population size in reconstruction of population history. We conclude that population genetic samples over large temporal and geographical scales, when analysed using more complex models and the serial coalescent, are critical to understand past population dynamics and provide important tools for reconstructing the evolutionary process.     

THE EFFECT OF COLLECTIVE DISPERSAL ON THE GENETIC STRUCTURE OF A SUBDIVIDED POPULATION

Correlated dispersal paths between two or more individuals are widespread across many taxa. The population genetic implications of this collective dispersal have received relatively little attention. Here we develop two‐sample coalescent theory that incorporates collective dispersal in a finite island model to predict expected coalescence times, genetic diversities, and F‐statistics. We show that collective dispersal reduces mixing in the system, which decreases expected coalescence times and increases F_ST. The effects are strongest in systems with high migration rates. Collective dispersal breaks the invariance of within‐deme coalescence times to migration rate, whatever the deme size. It can also cause F_ST to increase with migration rate because the ratio of within‐ to between‐deme coalescence times can decrease as migration rate approaches unity. This effect is most biologically relevant when deme size is small. We find qualitatively similar results for diploid and gametic dispersal. We also demonstrate with simulations and analytical theory the strong similarity between the effects of collective dispersal and anisotropic dispersal. These findings have implications for our understanding of the balance between drift–migration–mutation in models of neutral evolution. This has applied consequences for the interpretation of genetic structure (e.g., chaotic genetic patchiness) and estimation of migration rates from genetic data.     

Inference of Epidemiological Dynamics Based on Simulated Phylogenies Using Birth-Death and Coalescent Models

Quantifying epidemiological dynamics is crucial for understanding and forecasting the spread of an epidemic. The coalescent and the birth-death model are used interchangeably to infer epidemiological parameters from the genealogical relationships of the pathogen population under study, which in turn are inferred from the pathogen genetic sequencing data. To compare the performance of these widely applied models, we performed a simulation study. We simulated phylogenetic trees under the constant rate birth-death model and the coalescent model with a deterministic exponentially growing infected population. For each tree, we re-estimated the epidemiological parameters using both a birth-death and a coalescent based method, implemented as an MCMC procedure in BEAST v2.0. In our analyses that estimate the growth rate of an epidemic based on simulated birth-death trees, the point estimates such as the maximum a posteriori/maximum likelihood estimates are not very different. However, the estimates of uncertainty are very different. The birth-death model had a higher coverage than the coalescent model, i.e. contained the true value in the highest posterior density (HPD) interval more often (2–13% vs. 31–75% error). The coverage of the coalescent decreases with decreasing basic reproductive ratio and increasing sampling probability of infecteds. We hypothesize that the biases in the coalescent are due to the assumption of deterministic rather than stochastic population size changes. Both methods performed reasonably well when analyzing trees simulated under the coalescent. The methods can also identify other key epidemiological parameters as long as one of the parameters is fixed to its true value. In summary, when using genetic data to estimate epidemic dynamics, our results suggest that the birth-death method will be less sensitive to population fluctuations of early outbreaks than the coalescent method that assumes a deterministic exponentially growing infected population.     

Gene genealogies in a metapopulation

A simple genealogical process is found for samples from a metapopulation, which is a population that is subdivided into a large number of demes, each of which is subject to extinction and recolonization and receives migrants from other demes. As in the migration-only models studied previously, the genealogy of any sample includes two phases: a brief sample-size adjustment followed by a coalescent process that dominates the history. This result will hold for metapopulations that are composed of a large number of demes. It is robust to the details of population structure, as long as the number of possible source demes of migrants and colonists for each deme is large. Analytic predictions about levels of genetic variation are possible, and results for average numbers of pairwise differences within and between demes are given. Further analysis of the expected number of segregating sites in a sample from a single deme illustrates some previously known differences between migration and extinction/recolonization. The ancestral process is also amenable to computer simulation. Simulation results show that migration and extinction/recolonization have very different effects on the site-frequency distribution in a sample from a single deme. Migration can cause a U-shaped site-frequency distribution, which is qualitatively similar to the pattern reported recently for positive selection. Extinction and recolonization, in contrast, can produce a mode in the site-frequency distribution at intermediate frequencies, even in a sample from a single deme.     

The joint allele frequency spectrum of multiple populations: a coalescent theory approach

The allele frequency spectrum is a series of statistics that describe genetic polymorphism, and is commonly used for inferring population genetic parameters and detecting natural selection. Population genetic theory on the allele frequency spectrum for a single population has been well studied using both coalescent theory and diffusion equations. Recently, the theory was extended to the joint allele frequency spectrum (JAFS) for three populations using diffusion equations and was shown to be very useful in inferring human demographic history. In this paper, I show that the JAFS can be analytically derived with coalescent theory for a basic model of two isolated populations and then extended to multiple populations and various complex scenarios, such as those involving population growth and bottleneck, migration, and positive selection. Simulation study is used to demonstrate the accuracy and applicability of the theoretical model. The coalescent theory-based approach for the JAFS can characterize the demographic history with comprehensive statistical models as the diffusion approach does, and in addition gains several novel advantages: the computational complexity of calculating the JAFS with coalescent theory is reduced, and thus it is feasible to analytically obtain the JAFS for multiple populations; the hitchhiking effect can be efficiently modeled in coalescent theory, enabling the development of methodologies for detecting selection via multi-population polymorphism data. As an alternative to the diffusion approximation approach, the coalescent theory for the JAFS also provides a foundation for population genetic inference with the advent of large-scale genomic polymorphism data.     

Coalescent approximation for structured populations in a stationary random environment

We establish convergence to the Kingman coalescent for the genealogy of a geographically-or otherwise-structured version of the Wright-Fisher population model with fast migration. The new feature is that migration probabilities may change in a random fashion. This brings a novel formula for the coalescent effective population size (EPS). We call it a quenched EPS to emphasize the key feature of our model — random environment. The quenched EPS is compared with an annealed (mean-field) EPS which describes the case of constant migration probabilities obtained by averaging the random migration probabilities over possible environments.     

The coalescent in an island model of population subdivision with variation among demes

A simple genealogical structure is found for a general finite island model of population subdivision. The model allows for variation in the sizes of demes, in contributions to the migrant pool, and in the fraction of each deme that is replaced by migrants every generation. The ancestry of a sample of non-recombining DNA sequences has a simple structure when the sample size is much smaller than the total number of demes in the population. This allows an expression for the probability distribution of the number of segregating sites in the sample to be derived under the infinite-sites mutation model. It also yields easily computed estimators of the migration parameter for each deme in a multi-deme sample. The genealogical process is such that the lineages ancestral to the sample tend to accumulate in demes with low migration rates and/or which contribute disproportionately to the migrant pool. In addition, common ancestor or coalescent events tend to occur in demes of small size. This provides a framework for understanding the determinants of the effective size of the population, and leads to an expression for the probability that the root of a genealogy occurs in a particular geographic region, or among a particular set of demes.     

Inference of past population expansion from the timing of coalescence events in a gene genealogy

We investigate the expected coalescent in populations growing exponentially. The distribution of expected times to coalescence events may show a linear relationship with a number of ancestral lineages, when the latter is subjected to the "epidemic transformation". However, in a number of viral populations, upward curves are created when the epidemically transformed number of ancestral lineages is plotted against time. We consider possible causes of such upward curves. These include the possibility that a curved line is created through a transformation failure due to a sample size that is too large. We suggest a new formula for predicting such failure. The second cause is a population size increasing at an accelerating rate. However, the combination of recent coalescent events and an upward curve is created by an accelerating population increase only under restricted conditions. Specifically, such a pattern is expected only when, were population growth not to have accelerated, the transformation would have failed anyway. The third cause of nonlinearity arises in the estimated coalescent, as distinct from the real coalescent, if the mutation rate is small. However, coalescence times estimated from data typically give a straight line following epidemic transformation, but the rate of exponential increase, or r value, will be underestimated.