A separation-of-timescales approach to the coalescent in a continuous population

This article presents an analysis of a model of isolation by distance in a continuous, two-dimensional habitat. An approximate expression is derived for the distribution of coalescence times for a pair of sequences sampled from specific locations in a rectangular habitat. Results are qualitatively similar to previous analyses of isolation by distance, but account explicitly for the location of samples relative to the habitat boundaries. A separation-of-timescales approach takes advantage of the fact that the sampling locations affect only the recent coalescent behavior. When the population size is larger than the number of generations required for a lineage to cross the habitat range, the long-term genealogical process is reasonably well described by Kingman's coalescent with time rescaled by the effective population size. This long-term effective population size is affected by the local dispersal behavior as well as the geometry of the habitat. When the population size is smaller than the time required to cross the habitat, deep branches in the genealogy are longer than would be expected under the standard neutral coalescent, similar to the pattern expected for a panmictic population whose population size was larger in the past.     

Genealogy and subpopulation differentiation under various models of population structure

 The structured coalescent is used to calculate some quantities relating to the genealogy of a pair of homologous genes and to the degree of subpopulation differentiation, under a range of models of subdivided populations and assuming the infinite alleles model of neutral mutation. The classical island and stepping-stone models of population structure are considered, as well as two less symmetric models. For each model, we calculate the Laplace transform of the distribution of the coalescence time of a pair of genes from specified locations and the corresponding mean and variance. These results are then used to calculate the values of Wright’s coefficient F _ST , its limit as the mutation rate tends to zero and the limit of its derivative with respect to the mutation rate as the mutation rate tends to zero. From this derivative it is seen that F _ST can depend strongly on the mutation rate, for example in the case of an essentially one-dimensional habitat with many subpopulations where gene flow is restricted to neighbouring subpopulations. Received: 1 October 1997 / Revised version: 15 March 1998     

Coalescent time distributions in trees of arbitrary size

The relationship between speciation times and the corresponding times of gene divergence is of interest in phylogenetic inference as a means of understanding the past evolutionary dynamics of populations and of estimating the timing of speciation events. It has long been recognized that gene divergence times might substantially pre-date speciation events. Although the distribution of the difference between these has previously been studied for the case of two populations, this distribution has not been explicitly computed for larger species phylogenies. Here we derive a simple method for computing this distribution for trees of arbitrary size. A two-stage procedure is proposed which (i) considers the probability distribution of the time from the speciation event at the root of the species tree to the gene coalescent time conditionally on the number of gene lineages available at the root; and (ii) calculates the probability mass function for the number of gene lineages at the root. This two-stage approach dramatically simplifies numerical analysis, because in the first step the conditional distribution does not depend on an underlying species tree, while in the second step the pattern of gene coalescence prior to the species tree root is irrelevant. In addition, the algorithm provides intuition concerning the properties of the distribution with respect to the various features of the underlying species tree. The methodology is complemented by developing probabilistic formulae and software, written in R. The method and software are tested on five-taxon species trees with varying levels of symmetry. The examples demonstrate that more symmetric species trees tend to have larger mean coalescent times and are more likely to have a unimodal gamma-like distribution with a long right tail, while asymmetric trees tend to have smaller mean coalescent times with an exponential-like distribution. In addition, species trees with longer branches generally have shorter mean coalescent times, with branches closest to the root of the tree being most influential.     

Recombination as a Point Process along Sequences

Histories of sequences in the coalescent model with recombination can be simulated using an algorithm that takes as input a sample of extant sequences. The algorithm traces the history of the sequences going back in time, encountering recombinations and coalescence (duplications) until the ancestral material is located on one sequence for homologous positions in the present sequences. Here an alternative algorithm is formulated not as going back in time and operating on sequences, but by moving spatially along the sequences, updating the history of the sequences as recombination points are encountered. This algorithm focuses on spatial aspects of the coalescent with recombination rather than on temporal aspects as is the case of familiar algorithms. Mathematical results related to spatial aspects of the coalescent with recombination are derived.     

Coalescent methods for estimating species trees from phylogenomic data

Genome-scale sequence data have become increasingly available in the phylogenetic studies for understanding the evolutionary histories of species. However, it is challenging to develop probabilistic models to account for heterogeneity of phylogenomic data. The multispecies coalescent model describes gene trees as independent random variables generated from a coalescence process occurring along the lineages of the species tree. Since the multispecies coalescent model allows gene trees to vary across genes, coalescent-based methods have been popularly used to account for heterogeneous gene trees in phylogenomic data analysis. In this paper, we summarize and evaluate the performance of coalescent-based methods for estimating species trees from genome-scale sequence data. We investigate the effects of deep coalescence and mutation on the performance of species tree estimation methods. We found that the coalescent-based methods perform well in estimating species trees for a large number of genes, regardless of the degree of deep coalescence and mutation. The performance of the coalescent methods is negatively correlated with the lengths of internal branches of the species tree.     

Evolutionary Consequences of Dispersal Ability in Cactus-feeding Insects

Although gene flow is an important determinant of evolutionary change, the role of ecological factors such as specialization in determining migration and gene flow has rarely been explored empirically. To examine the consequences of dispersal ability and habitat patchiness on gene flow, migration rates were compared in three cactophagous longhorn beetles using coalescent analyses of mtDNA sequences. Analyses of covariance were used to identify the roles of dispersal ability and habitat distribution in determining migration patterns. Dispersal ability was a highly significant predictor of gene flow (p< 0.001), and was more important than any other factor. These findings predict that dispersal ability may be an import factor shaping both microevolutionary and macroevolutionary patterns; this prediction is borne out by comparisons of species diversity in cactus-feeding groups.     

The coalescence time of sampled genes in the structured coalescent model

The aim of this article is to investigate the distribution of the coalescence time (T) for sampled genes in the structured coalescent. We obtain some exact solutions for small samples and approximate distributions for n sampled genes in strong and weak migration. We also conduct computer simulation to evaluate efficiencies of these approximations and show the dependency of the distribution of the coalescence time on the geographical structure and the intensity of migration. In a panmictic population, we prove that the conditional distribution of the coalescence time given the number of segregating sites (S) among sampled genes is given by the weighted mean of the convolution of gamma distributions. We also study the joint distribution of T and S in the structured coalescent model and show some exact solutions.     

Exact coalescent for the Wright-Fisher model

The Kingman coalescent, which has become the foundation for a wide range of theoretical as well as empirical studies, was derived as an approximation of the Wright-Fisher (WF) model. The approximation heavily relies on the assumption that population size is large and sample size is much smaller than the population size. Whether the sample size is too large compared to the population size is rarely questioned in practice when applying statistical methods based on the Kingman coalescent. Since WF model is the most widely used population genetics model for reproduction, it is desirable to develop a coalescent framework for the WF model, which can be used whenever there are concerns about the accuracy of the Kingman coalescent as an approximation. This paper described the exact coalescent theory for the WF model and develops a simulation algorithm, which is then used, together with an analytical approach, to study the properties of the exact coalescent as well as its differences to the Kingman coalescent. We show that the Kingman coalescent differs from the exact coalescent by: (1) shorter waiting time between successive coalescent events; (2) different probability of observing a topological relationship among sequences in a sample; and (3) slightly smaller tree length in the genealogy of a large sample. On the other hand, there is little difference in the age of the most recent common ancestor (MRCA) of the sample. The exact coalescent makes up the longer waiting time between successive coalescent events by having multiple coalescence at the same time. The most significant difference among various summary statistics of a coalescent examined is the sum of lengths of external branches, which can be more than 10% larger for exact coalescent than that for the Kingman coalescent. As a whole, the Kingman coalescent is a remarkably accurate approximation to the exact coalescent for sample and population sizes falling considerably outside the region that was originally anticipated.     

The joint effects of background selection and genetic recombination on local gene genealogies

Background selection, the effects of the continual removal of deleterious mutations by natural selection on variability at linked sites, is potentially a major determinant of DNA sequence variability. However, the joint effects of background selection and genetic recombination on the shape of the neutral gene genealogy have proved hard to study analytically. The only existing formula concerns the mean coalescent time for a pair of alleles, making it difficult to assess the importance of background selection from genome-wide data on sequence polymorphism. Here we develop a structured coalescent model of background selection with recombination and implement it in a computer program that efficiently generates neutral gene genealogies for an arbitrary sample size. We check the validity of the structured coalescent model against forward-in-time simulations and show that it accurately captures the effects of background selection. The model produces more accurate predictions of the mean coalescent time than the existing formula and supports the conclusion that the effect of background selection is greater in the interior of a deleterious region than at its boundaries. The level of linkage disequilibrium between sites is elevated by background selection, to an extent that is well summarized by a change in effective population size. The structured coalescent model is readily extendable to more realistic situations and should prove useful for analyzing genome-wide polymorphism data.     

Coalescent process with fluctuating population size and its effective size

We consider a Wright-Fisher model whose population size is a finite Markov chain. We introduce a sequence of two-dimensional discrete time Markov chains whose components describe the coalescent process and the fluctuation of population size. For the limiting process of the sequence of Markov chains, the relationship of the expectation of coalescence time to the harmonic and the arithmetic means of population sizes is shown, and the Laplace transform of the distribution of coalescence time is calculated. We define the coalescence effective population size (cEPS) by the expectation of coalescence time. We show that cEPS is strictly larger (resp. smaller) than the harmonic (resp. arithmetic) mean. As the population size fluctuates more quickly (resp. slowly), cEPS is closer to the harmonic (resp. arithmetic) mean. For the case of a two-valued Markov chain, we show the explicit expression of cEPS and its dependency on the sample size.     

Recombination, gene conversion, and identity-by-descent at three loci

We investigate the probabilities of identity-by-descent at three loci in order to find a signature which differentiates between the two types of crossing over events: recombination and gene conversion. We use a Markov chain to model coalescence, recombination, gene conversion and mutation in a sample of size two. Using numerical analysis, we calculate the total probability of identity-by-descent at the three loci, and partition these probabilities based on a partial ordering of coalescent events at the three loci. We use these results to compute the probabilities of four different patterns of conditional identity and non-identity at the three loci under recombination and gene conversion. Although recombination and gene conversion do make different predictions, the differences are not likely to be useful in distinguishing between them using three locus patterns between pairs of DNA sequences. This implies that measures of genetic identity in larger samples will be needed to distinguish between gene conversion and recombination.     

The effect of gene flow on the coalescent time in the human-chimpanzee ancestral population

The coalescent process in the human-chimpanzee ancestral population is investigated using a model, which incorporates a certain time period of gene flow during the speciation process. a is a parameter to represent the degree and time of gene flow, and the model is identical to the null model with an instantaneous species split when a=infinity. A maximum likelihood (ML) method is developed to estimate a, and its power and reliability is investigated by coalescent simulations. The ML method is applied to nucleotide divergence data between human and chimpanzee. It is found that the null model with an instantaneous species split explains the data best, and no strong evidence for gene flow is detected. The result is discussed in the view of the mode of speciation. Another ML method is developed to estimate the male-female ratio (alpha) of mutation rate, in which the coalescent process in the ancestral population is taken into account.     

Concordant genetic estimators of migration reveal anthropogenically enhanced source-sink population structure in the river sculpin, Cottus gobio

River systems are vulnerable to natural and anthropogenic habitat fragmentation and will often harbor populations deviating markedly from simplified theoretical models. We investigated fine-scale population structure in the sedentary river fish Cottus gobio using microsatellites and compared migration estimates from three FST estimators, a coalescent maximum-likelihood method and Bayesian recent migration analyses. Source-sink structure was evident via asymmetry in migration and genetic diversity with smaller upstream locations emigration biased and larger downstream subpopulations immigration biased. Patterns of isolation by distance suggested that the system was largely, but not entirely, in migration-drift equilibrium, with headwater populations harboring a signal of past colonizations and in some cases also recent population bottlenecks. Up- vs. downstream asymmetry in population structure was partly attributable to the effects of flow direction, but was enhanced by weirs prohibiting compensatory upstream migration. Estimators of migration showed strong correspondence, at least in relative terms, especially if pairwise FST was used as an indirect index of relative gene flow rather than being translated to Nm. Since true parameter values are unknown in natural systems, comparisons among estimators are important, both to determine confidence in estimates of migration and to validate the performance of different methods.     

Approximating the coalescent with recombination

The coalescent with recombination describes the distribution of genealogical histories and resulting patterns of genetic variation in samples of DNA sequences from natural populations. However, using the model as the basis for inference is currently severely restricted by the computational challenge of estimating the likelihood. We discuss why the coalescent with recombination is so challenging to work with and explore whether simpler models, under which inference is more tractable, may prove useful for genealogy-based inference. We introduce a simplification of the coalescent process in which coalescence between lineages with no overlapping ancestral material is banned. The resulting process has a simple Markovian structure when generating genealogies sequentially along a sequence, yet has very similar properties to the full model, both in terms of describing patterns of genetic variation and as the basis for statistical inference.     

Conservation Genetics of Kangaroo Mice, Genus Microdipodops

The two currently recognized species of kangaroo mice, Microdipodops megacephalus and M. pallidus, inhabit sandy soils of the Great Basin Desert in western North America. Given their habitat specificity and the fluctuating climate throughout the Pleistocene, kangaroo mice likely endured a turbulent biogeographic history that resulted in disjunct distributions and isolation of genetic lineages. Recent phylogenetic investigations using mitochondrial data have revealed several mitochondrial clades within this genus that may represent cryptic species. These mitochondrial clades are genetically unique, occupy relatively small distributions, and, as such, may be at an increased risk of extinction due to climate change and extensive recent habitat alteration. Herein, we apply haplotype network, population genetic, and historical demographic analyses to mitochondrial data of each Micropdipodops species and mitochondrial clade to assess conservation genetics within kangaroo mice. Results indicate that each mitochondrial clade is a distinct lineage with little to no gene flow occurring among clades. Additionally, historical demographic analyses support past population expansions and identify locations of past refugium for each distinct lineage. Although mitochondrial data indicate that the clades appear to be in approximate genetic equilibrium and have not suffered any extreme bottlenecks over time, there is still concern for the survival of smaller and more vulnerable Microdipodops subpopulations due to impending habitat threats in the Great Basin Desert.     

Multiple merger gene genealogies in two species: Monophyly, paraphyly, and polyphyly for two examples of Lambda coalescents

Probabilities of monophyly, paraphyly, and polyphyly of two-species gene genealogies are computed for modest sample sizes and compared for two different Λ coalescent processes. Coalescent processes belonging to the Λ coalescent family admit asynchronous multiple mergers of active ancestral lineages. Assigning a timescale to the time of divergence becomes a central issue when different populations have different coalescent processes running on different timescales. Clade probabilities in single populations are also computed, which can be useful for testing for taxonomic distinctiveness of an observed set of monophyletic lineages. The coalescence rates of multiple merger coalescent processes are functions of coalescent parameters. The effect of coalescent parameters on the probabilities studied depends on the coalescent process, and if the population is ancestral or derived. The probability of reciprocal monophyly tends to be somewhat lower, when associated with a Λ coalescent, under the null hypothesis that two groups come from the same population. However, even for fairly recent divergence times, the probability of monophyly tends to be higher as a function of the number of generations for coalescent processes that admit multiple mergers, and is sensitive to the parameter of one of the example processes.     

The legacy of ice ages in mountain species: post‐glacial colonization of mountain tops rather than current range fragmentation determines mitochondrial genetic diversity in an endemic Pyrenean rock lizard

Aim The genetic impact of Quaternary climatic fluctuations on mountain endemic species has rarely been investigated. The Pyrenean rock lizard (Iberolacerta bonnali) is restricted to alpine habitats in the Pyrenees where it exhibits a highly fragmented distribution between massifs and between habitats within massifs. Using mitochondrial DNA markers, we set out: (1) to test whether several evolutionary units exist within the species; (2) to understand how the species persisted through the Last Glacial Maximum and whether the current range fragmentation originates from distribution shifts after the Last Glacial Maximum or from more ancient events; and (3) to investigate whether current mitochondrial diversity reflects past population history or current habitat fragmentation. Location The Pyrenees in south‐western France and northern Spain. Methods We used variation in the hypervariable left domain of the mitochondrial control region of 146 lizards collected in 15 localities, supplemented by cytochrome b sequences downloaded from GenBank to cover most of the species’ distribution range. Measures of population genetic diversity were contrasted with population isolation inferred from topography. Classical (F‐statistics) and coalescence‐based methods were used to assess the level of gene flow and estimate divergence time between populations. We used coalescence‐based simulations to test the congruence of our genetic data with a scenario of simultaneous divergence of current populations. Results Coalescence‐based analyses suggested that these peripheral populations diverged simultaneously at the end of the last glacial episode when their habitats became isolated on mountain summits. High mitochondrial diversity was found in peripheral, isolated populations, while the populations from the core of the species’ range were genetically impoverished. Where mitochondrial diversity has been retained, populations within the same massif exhibited high levels of genetic differentiation. Main conclusions As suggested for many other mountain species, the Pyrenean rock lizard survived glacial maxima through short‐distance range shifts instead of migration or contraction in distant southern refugia. Most of the main Pyrenean range has apparently been re‐colonized from a single or a few source populations, resulting in a loss of genetic diversity in re‐colonized areas. As a result, current levels of intra‐population mitochondrial diversity are better explained by post‐glacial population history than by current habitat fragmentation. Genetic population differentiation within massifs implies severe reduction in female‐mediated gene flow between patches of habitats.     

Are populations like a circuit? Comparing isolation by resistance to a new coalescent‐based method

A number of methods commonly used in landscape genetics use an analogy to electrical resistance on a network to describe and fit barriers to movement across the landscape using genetic distance data. These are motivated by a mathematical equivalence between electrical resistance between two nodes of a network and the ‘commute time’, which is the mean time for a random walk on that network to leave one node, visit the other, and return. However, genetic data are more accurately modelled by a different quantity, the coalescence time. Here, we describe the differences between resistance distance and coalescence time, and explore the consequences for inference. We implemented a Bayesian method to infer effective movement rates and population sizes under both these models, and found that inference using commute times could produce misleading results in the presence of biased gene flow. We then used forwards‐time simulation with continuous geography to demonstrate that coalescence‐based inference remains more accurate than resistance‐based methods on realistic data, but difficulties highlight the need for methods that explicitly model continuous, heterogeneous geography.     

Convergence to the island-model coalescent process in populations with restricted migration

In this article we apply some graph-theoretic results to the study of coalescence in a structured population with migration. The graph is the pattern of migration among subpopulations, or demes, and we use the theory of random walks on graphs to characterize the ease with which ancestral lineages can traverse the habitat in a series of migration events. We identify conditions under which the coalescent process in populations with restricted migration, such that individuals cannot traverse the habitat freely in a single migration event, nonetheless becomes identical to the coalescent process in the island migration model in the limit as the number of demes tends to infinity. Specifically, we first note that a sequence of symmetric graphs with Diaconis-Stroock constant bounded above has an unstructured Kingman-type coalescent in the limit for a sample of size two from two different demes. We then show that circular and toroidal models with long-range but restricted migration have an upper bound on this constant and so have an unstructured-migration coalescent in the limit. We investigate the rate of convergence to this limit using simulations.