Statistical phylogeography: methods of evaluating and minimizing inference errors

Nested clade phylogeographical analysis (NCPA) has become a common tool in intraspecific phylogeography. To evaluate the validity of its inferences, NCPA was applied to actual data sets with 150 strong a priori expectations, the majority of which had not been analysed previously by NCPA. NCPA did well overall, but it sometimes failed to detect an expected event and less commonly resulted in a false positive. An examination of these errors suggested some alterations in the NCPA inference key, and these modifications reduce the incidence of false positives at the cost of a slight reduction in power. Moreover, NCPA does equally well in inferring events regardless of the presence or absence of other, unrelated events. A reanalysis of some recent computer simulations that are seemingly discordant with these results revealed that NCPA performed appropriately in these simulated samples and was not prone to a high rate of false positives under sampling assumptions that typify real data sets. NCPA makes a posteriori use of an explicit inference key for biological interpretation after statistical hypothesis testing. Alternatives to NCPA that claim that biological inference emerges directly from statistical testing are shown in fact to use an a priori inference key, albeit implicitly. It is argued that the a priori and a posteriori approaches to intraspecific phylogeography are complementary, not contradictory. Finally, cross-validation using multiple DNA regions is shown to be a powerful method of minimizing inference errors. A likelihood ratio hypothesis testing framework has been developed that allows testing of phylogeographical hypotheses, extends NCPA to testing specific hypotheses not within the formal inference key (such as the out-of-Africa replacement hypothesis of recent human evolution) and integrates intra- and interspecific phylogeographical inference.     

Fossorial but widespread: the phylogeography of the common spadefoot toad (Pelobates fuscus), and the role of the Po Valley as a major source of genetic variability

Pelobates fuscus is a fossorial amphibian that inhabits much of the European plain areas. To unveil traces of expansion and contraction events of the species' range, we sequenced 702 bp of the mitochondrial cytochrome b gene. To infer the population history we applied phylogeographical methods, such as nested clade phylogeographical analysis (NCPA), and used summary statistics to analyse population structure under a neutral model of evolution. Populations were assigned to different drainage systems and we tested hypotheses of explicit refugial models using information from analysis of molecular variance, nucleotide diversity, effective population size estimation, NCPA, mismatch distribution and Bayesian dating. Coalescent simulations were used as post hoc tests for plausibility of derived or a priori assumed biogeographical hypotheses. Our combination of all approaches enabled the reconstruction of the colonization history and phylogeography of P. fuscus and confirmed a previous assumption of the existence of two major genetic lineages within P. fuscus. Using the Afro-European vicariance of Pelobates cultripes and Pelobates varaldii and applying Bayesian dating we estimated the divergence of these phylogeographical lineages to the Pliocene. We suggest the existence of three different glacial refugia: (i) the area between the Caspian and Black Seas as the origin for the expansion of the 'eastern lineage'; (ii) the Danube system as a centre of diversity for part of the 'western lineage'; (iii) the Po Valley, the largest centre of genetic variability. This fits the hypothesis that climatic fluctuation was a key event for differentiation processes in P. fuscus.     

Congeneric phylogeography: hypothesizing species limits and evolutionary processes in Patagonian lizards of the Liolaemus boulengeri group (Squamata: Liolaemini)

In poorly known groups for which data are insufficient to develop biologically plausible model‐based approaches to phylogeographical analyses, a ‘first hypotheses’ protocol is suggested as offering the best way to generate hypotheses for subsequent model‐based tests. Preliminary hypotheses are formulated about species boundaries and population processes in three species complexes of the Liolaemus boulengeri group in the context of mtDNA ‘congeneric phylogeography’. The temperate South American Liolaemus provides a model with ancient and recent allopatric divergence across ecologically and geologically complex landscapes, incipient speciation, secondary contact, and discordance between molecular and morphological patterns of variation. Moderately dense sampling of widely distributed ‘inertial’ species in the Patagonian Steppe has revealed hidden genetic and probably species diversity, and also hinted at demographic and historical processes that may have shaped the histories of these taxa. Five of the seven focal species of the present study were paraphyletic for mtDNA genealogies, suggesting that they represent complexes of species, and nested clade phylogeographical analysis (NCPA) analyses suggest that different historical and demographic processes have shaped the observed patterns. Introgression and incomplete lineage sorting are hypothesized as being the cause of some of the observed paraphyly. Provisional delimitations of species are proposed and NCPA is used to generate hypotheses of population history, all of which are subject to further testing. Multi‐faceted studies, involving phylogenetic assessments of independent molecular markers and morphological variation across codistributed taxa with estimates of niche breadths in a landscape context, will likely yield the most promising returns for cross‐validation of hypotheses of population and speciation histories. © 2006 The Linnean Society of London, Biological Journal of the Linnean Society, 2006, 89 , 241–275.     

In defence of model‐based inference in phylogeography

Recent papers have promoted the view that model‐based methods in general, and those based on Approximate Bayesian Computation (ABC) in particular, are flawed in a number of ways, and are therefore inappropriate for the analysis of phylogeographic data. These papers further argue that Nested Clade Phylogeographic Analysis (NCPA) offers the best approach in statistical phylogeography. In order to remove the confusion and misconceptions introduced by these papers, we justify and explain the reasoning behind model‐based inference. We argue that ABC is a statistically valid approach, alongside other computational statistical techniques that have been successfully used to infer parameters and compare models in population genetics. We also examine the NCPA method and highlight numerous deficiencies, either when used with single or multiple loci. We further show that the ages of clades are carelessly used to infer ages of demographic events, that these ages are estimated under a simple model of panmixia and population stationarity but are then used under different and unspecified models to test hypotheses, a usage the invalidates these testing procedures. We conclude by encouraging researchers to study and use model‐based inference in population genetics.     

WHY DOES A METHOD THAT FAILS CONTINUE TO BE USED? THE ANSWER

It has been claimed that hundreds of researchers use nested clade phylogeographic analysis (NCPA) based on what the method promises rather than requiring objective validation of the method. The supposed failure of NCPA is based upon the argument that validating it by using positive controls ignored type I error, and that computer simulations have shown a high type I error. The first argument is factually incorrect: the previously published validation analysis fully accounted for both type I and type II errors. The simulations that indicate a 75% type I error rate have serious flaws and only evaluate outdated versions of NCPA. These outdated type I error rates fall precipitously when the 2003 version of single-locus NCPA is used or when the 2002 multilocus version of NCPA is used. It is shown that the tree-wise type I errors in single-locus NCPA can be corrected to the desired nominal level by a simple statistical procedure, and that multilocus NCPA reconstructs a simulated scenario used to discredit NCPA with 100% accuracy. Hence, NCPA is a not a failed method at all, but rather has been validated both by actual data and by simulated data in a manner that satisfies the published criteria given by its critics. The critics have come to different conclusions because they have focused on the pre-2002 versions of NCPA and have failed to take into account the extensive developments in NCPA since 2002. Hence, researchers can choose to use NCPA based upon objective critical validation that shows that NCPA delivers what it promises.     

Bayesian inferences on umbrella orderings

In regression applications with categorical predictors, interest often focuses on comparing the null hypothesis of homogeneity to an ordered alternative. This article proposes a Bayesian approach for addressing this problem in the setting of normal linear and probit regression models. The regression coefficients are assigned a conditionally conjugate prior density consisting of mixtures of point masses at 0 and truncated normal densities, with a (possibly unknown) changepoint parameter included to accommodate umbrella ordering. Two strategies of prior elicitation are considered: (1) a Bayesian Bonferroni approach in which the probability of the global null hypothesis is specified and local hypotheses are considered independent; and (2) an approach which treats these probabilities as random. A single Gibbs sampling chain can be used to obtain posterior probabilities for the different hypotheses and to estimate regression coefficients and predictive quantities either by model averaging or under the preferred hypothesis. The methods are applied to data from a carcinogenesis study.     

Model misspecification and probabilistic tests of topology: evidence from empirical data sets

Probabilistic tests of topology offer a powerful means of evaluating competing phylogenetic hypotheses. The performance of the nonparametric Shimodaira-Hasegawa (SH) test, the parametric Swofford-Olsen-Waddell-Hillis (SOWH) test, and Bayesian posterior probabilities were explored for five data sets for which all the phylogenetic relationships are known with a very high degree of certainty. These results are consistent with previous simulation studies that have indicated a tendency for the SOWH test to be prone to generating Type 1 errors because of model misspecification coupled with branch length heterogeneity. These results also suggest that the SOWH test may accord overconfidence in the true topology when the null hypothesis is in fact correct. In contrast, the SH test was observed to be much more conservative, even under high substitution rates and branch length heterogeneity. For some of those data sets where the SOWH test proved misleading, the Bayesian posterior probabilities were also misleading. The results of all tests were strongly influenced by the exact substitution model assumptions. Simple models, especially those that assume rate homogeneity among sites, had a higher Type 1 error rate and were more likely to generate misleading posterior probabilities. For some of these data sets, the commonly used substitution models appear to be inadequate for estimating appropriate levels of uncertainty with the SOWH test and Bayesian methods. Reasons for the differences in statistical power between the two maximum likelihood tests are discussed and are contrasted with the Bayesian approach.     

Failed refutations: further comments on parsimony and likelihood methods and their relationship to Popper's degree of corroboration

Kluge's (2001, Syst. Biol. 50:322-330) continued arguments that phylogenetic methods based on the statistical principle of likelihood are incompatible with the philosophy of science described by Karl Popper are based on false premises related to Kluge's misrepresentations of Popper's philosophy. Contrary to Kluge's conjectures, likelihood methods are not inherently verificationist; they do not treat every instance of a hypothesis as confirmation of that hypothesis. The historical nature of phylogeny does not preclude phylogenetic hypotheses from being evaluated using the probability of evidence. The low absolute probabilities of hypotheses are irrelevant to the correct interpretation of Popper's concept termed degree of corroboration, which is defined entirely in terms of relative probabilities. Popper did not advocate minimizing background knowledge; in any case, the background knowledge of both parsimony and likelihood methods consists of the general assumption of descent with modification and additional assumptions that are deterministic, concerning which tree is considered most highly corroborated. Although parsimony methods do not assume (in the sense of entailing) that homoplasy is rare, they do assume (in the sense of requiring to obtain a correct phylogenetic inference) certain things about patterns of homoplasy. Both parsimony and likelihood methods assume (in the sense of implying by the manner in which they operate) various things about evolutionary processes, although violation of those assumptions does not always cause the methods to yield incorrect phylogenetic inferences. Test severity is increased by sampling additional relevant characters rather than by character reanalysis, although either interpretation is compatible with the use of phylogenetic likelihood methods. Neither parsimony nor likelihood methods assess test severity (critical evidence) when used to identify a most highly corroborated tree(s) based on a single method or model and a single body of data; however, both classes of methods can be used to perform severe tests. The assumption of descent with modification is insufficient background knowledge to justify cladistic parsimony as a method for assessing degree of corroboration. Invoking equivalency between parsimony methods and likelihood models that assume no common mechanism emphasizes the necessity of additional assumptions, at least some of which are probabilistic in nature. Incongruent characters do not qualify as falsifiers of phylogenetic hypotheses except under extremely unrealistic evolutionary models; therefore, justifications of parsimony methods as falsificationist based on the idea that they minimize the ad hoc dismissal of falsifiers are questionable. Probabilistic concepts such as degree of corroboration and likelihood provide a more appropriate framework for understanding how phylogenetics conforms with Popper's philosophy of science. Likelihood ratio tests do not assume what is at issue but instead are methods for testing hypotheses according to an accepted standard of statistical significance and for incorporating considerations about test severity. These tests are fundamentally similar to Popper's degree of corroboration in being based on the relationship between the probability of the evidence e in the presence versus absence of the hypothesis h, i.e., between p(e|hb) and p(e|b), where b is the background knowledge. Both parsimony and likelihood methods are inductive in that their inferences (particular trees) contain more information than (and therefore do not follow necessarily from) the observations upon which they are based; however, both are deductive in that their conclusions (tree lengths and likelihoods) follow necessarily from their premises (particular trees, observed character state distributions, and evolutionary models). For these and other reasons, phylogenetic likelihood methods are highly compatible with Karl Popper's philosophy of science and offer several advantages over parsimony methods in this context.     

Selecting factors predictive of heterogeneity in multivariate event time data

In multivariate survival analysis, investigators are often interested in testing for heterogeneity among clusters, both overall and within specific classes. We represent different hypotheses about the heterogeneity structure using a sequence of gamma frailty models, ranging from a null model with no random effects to a full model having random effects for each class. Following a Bayesian approach, we define prior distributions for the frailty variances consisting of mixtures of point masses at zero and inverse-gamma densities. Since frailties with zero variance effectively drop out of the model, this prior allocates probability to each model in the sequence, including the overall null hypothesis of homogeneity. Using a counting process formulation, the conditional posterior distributions of the frailties and proportional hazards regression coefficients have simple forms. Posterior computation proceeds via a data augmentation Gibbs sampling algorithm, a single run of which can be used to obtain model-averaged estimates of the population parameters and posterior model probabilities for testing hypotheses about the heterogeneity structure. The methods are illustrated using data from a lung cancer trial.     

Random sampling does not exclude spatial dependence: The importance of neutral models for ecological hypothesis testing

Lájer (2007) notes that, to investigate phytosociological and ecological relationships, many authors apply traditional inferential tests to sets of relevés obtained by non-random methods. Unfortunately, this procedure does not provide reliable support for hypothesis testing because non-random sampling violates the assumptions of independence required by many parametric inferential tests. Instead, a random sampling scheme is recommended. Nonetheless, random sampling will not eliminate spatial autocorrelation. For instance, a classical law of geography holds that everything in a piece of (biotic) space is interrelated, but near objects are more related than distant ones. Because most ecological processes that shape community structure and species coexistence are spatially explicit, spatial autocorrelation is a vital part of almost all ecological data. This means that, independently from the underlying sampling design, ecological data are generally spatially autocorrelated, violating the assumption of independence that is generally required by traditional inferential tests. To overcome this drawback, randomization tests may be used. Such tests evaluate statistical significance based on empirical distributions generated from the sample and do not necessarily require data independence. However, as concerns hypothesis testing, randomization tests are not the universal remedy for ecologists, because the choice of inadequate null models can have significant effects on the ecological hypotheses tested. In this paper, I emphasize the need of developing null models for which the statistical assumptions match the underlying biological mechanisms.     

Testing metapopulation concepts: effects of patch characteristics and neighborhood occupancy on the dynamics of an endangered lagomorph

Metapopulation ecology is a field that is richer in theory than in empirical results. Many existing empirical studies use an incidence function approach based on spatial patterns and key assumptions about extinction and colonization rates. Here we recast these assumptions as hypotheses to be tested using 18 years of historic detection survey data combined with four years of data from a new monitoring program for the Lower Keys marsh rabbit. We developed a new model to estimate probabilities of local extinction and colonization in the presence of nondetection, while accounting for estimated occupancy levels of neighboring patches. We used model selection to identify important drivers of population turnover and estimate the effective neighborhood size for this system. Several key relationships related to patch size and isolation that are often assumed in metapopulation models were supported: patch size was negatively related to the probability of extinction and positively related to colonization, and estimated occupancy of neighboring patches was positively related to colonization and negatively related to extinction probabilities. This latter relationship suggested the existence of rescue effects. In our study system, we inferred that coastal patches experienced higher probabilities of extinction and colonization than interior patches. Interior patches exhibited higher occupancy probabilities and may serve as refugia, permitting colonization of coastal patches following disturbances such as hurricanes and storm surges. Our modeling approach should be useful for incorporating neighbor occupancy into future metapopulation analyses and in dealing with other historic occupancy surveys that may not include the recommended levels of sampling replication.     

On the influence of prior information evaluated by fully Bayesian criteria in a personalized whole-brain model of epilepsy spread

Individualized anatomical information has been used as prior knowledge in Bayesian inference paradigms of whole-brain network models. However, the actual sensitivity to such personalized information in priors is still unknown. In this study, we introduce the use of fully Bayesian information criteria and leave-one-out cross-validation technique on the subject-specific information to assess different epileptogenicity hypotheses regarding the location of pathological brain areas based on a priori knowledge from dynamical system properties. The Bayesian Virtual Epileptic Patient (BVEP) model, which relies on the fusion of structural data of individuals, a generative model of epileptiform discharges, and a self-tuning Monte Carlo sampling algorithm, is used to infer the spatial map of epileptogenicity across different brain areas. Our results indicate that measuring the out-of-sample prediction accuracy of the BVEP model with informative priors enables reliable and efficient evaluation of potential hypotheses regarding the degree of epileptogenicity across different brain regions. In contrast, while using uninformative priors, the information criteria are unable to provide strong evidence about the epileptogenicity of brain areas. We also show that the fully Bayesian criteria correctly assess different hypotheses about both structural and functional components of whole-brain models that differ across individuals. The fully Bayesian information-theory based approach used in this study suggests a patient-specific strategy for epileptogenicity hypothesis testing in generative brain network models of epilepsy to improve surgical outcomes.     

Predicting the past distribution of species climatic niches

Predicting past distributions of species climatic niches, hindcasting, by using climate envelope models (CEMs) is emerging as an exciting research area. CEMs are used to examine veiled evolutionary questions about extinctions, locations of past refugia and migration pathways, or to propose hypotheses concerning the past population structure of species in phylogeographical studies. CEMs are sensitive to theoretical assumptions, to model classes and to projections in non-analogous climates, among other issues. Studies hindcasting the climatic niches of species often make reference to these limitations. However, to obtain strong scientific inferences, we must not only be aware of these potential limitations but we must also overcome them. Here, I review the literature on hindcasting CEMs. I discuss the theoretical assumptions behind niche modelling, i.e. the stability of climatic niches through time and the equilibrium of species with climate. I also summarize a set of 'recommended practices' to improve hindcasting. The studies reviewed: (1) rarely test the theoretical assumptions behind niche modelling such as the stability of species climatic niches through time and the equilibrium of species with climate; (2) they only use one model class (72% of the studies) and one palaeoclimatic reconstruction (62.5%) to calibrate their models; (3) they do not check for the occurrence of non-analogous climates (97%); and (4) they do not use independent data to validate the models (72%). Ignoring the theoretical assumptions behind niche modelling and using inadequate methods for hindcasting CEMs may well entail a cascade of errors and naïve ecological and evolutionary inferences. We should also push integrative research lines linking macroecology, physiology, population biology, palaeontology, evolutionary biology and CEMs for a better understanding of niche dynamics across space and time.     

Microtine ultradian rhythm of activity: an evaluation of different hypotheses on the triggering mechanism

Ultradian activity rhythms are a peculiar behavioural pattern, in which the 24-h day is divided into several short-term activity cycles. Such patterns are particularly prominent in microtine rodents. In this paper we review experimental findings and proposed hypotheses on the proximate triggering mechanism of ultradian rhythmicity. Eight different mechanisms (classified as homeostatic, pacemaker and stochastic hypotheses) were analysed within the same theoretical model-framework, considering feeding, food processing, energy turnover and activity behaviour.
Ultradian activity patterns could consistently be generated by each of the analysed models. None of the models, however, resulted in frequency dependencies in respect to body weight, metabolic rate, food quality and food deprivation, unless additional assumptions were introduced.
From our model evaluations we concluded that the stochastic model does not represent an independent hypothesis. Hence, only homeostatic and pacemaker hypotheses remained as alternative approaches to the triggering mechanism. Although the pacemaker hypothesis appeared more reasonable, a combination with some feedback mechanism to metabolism seemed even more likely. Some key experiments are suggested, which could give insight into the complex mechanism.     

Comparison of probability and likelihood models for peptide identification from tandem mass spectrometry data

We evaluate statistical models used in two-hypothesis tests for identifying peptides from tandem mass spectrometry data. The null hypothesis H(0), that a peptide matches a spectrum by chance, requires information on the probability of by-chance matches between peptide fragments and peaks in the spectrum. Likewise, the alternate hypothesis H(A), that the spectrum is due to a particular peptide, requires probabilities that the peptide fragments would indeed be observed if it was the causative agent. We compare models for these probabilities by determining the identification rates produced by the models using an independent data set. The initial models use different probabilities depending on fragment ion type, but uniform probabilities for each ion type across all of the labile bonds along the backbone. More sophisticated models for probabilities under both H(A) and H(0) are introduced that do not assume uniform probabilities for each ion type. In addition, the performance of these models using a standard likelihood model is compared to an information theory approach derived from the likelihood model. Also, a simple but effective model for incorporating peak intensities is described. Finally, a support-vector machine is used to discriminate between correct and incorrect identifications based on multiple characteristics of the scoring functions. The results are shown to reduce the misidentification rate significantly when compared to a benchmark cross-correlation based approach.     

A navigational guide to variable fitness: common methods of analysis,where they break down,and what you can do instead

Our methods for analyzing stochastic fitness are mostly approximations, and the assumptions behind these approximations are not always well understood. Furthermore, many of these approximations break down when fitness variance is high. This review covers geometric mean growth, diffusion approximations, and Markov processes. It discusses where each is appropriate, the conditions under which they break down, and their advantages and disadvantages, with special attention to the case of high fitness variance. A model of sessile and site-attached coastal species is used as a running example, and fully worked calculations and code are provided. Summary: The logarithm of geometric mean growth is usually only appropriate when (a) an invader growth rate is needed and (b) fitness variability is driven by environmental fluctuations. The usual approximation breaks down when fitness variance is high. Diffusion approximations can provide a reasonable guide to the expected change in frequency over a time step if expected fitnesses and fitness variances are appropriately scaled by the average expected fitness. Diffusion approximations can perform less well for fixation probabilities, especially since further approximations may be required. Fixation probabilities can be calculated exactly using a Markov process, regardless of how large fitness variance is, although an analytic expression is frequently not possible. If an analytic expression is desired, it may be worth using a diffusion approximation and checking it with a Markov process calculation.     

Thematic review series: patient-oriented research. Design and analysis of lipoprotein tracer kinetics studies in humans

Lipoprotein tracer kinetics studies have for many years provided new and important knowledge of the metabolism of lipoproteins. Our understanding of kinetics defects in lipoprotein metabolism has resulted from the use of tracer kinetics studies and mathematical modeling. This review discusses all aspects of the performance of kinetics studies, including the development of hypotheses, experimental design, statistical considerations, tracer administration and sampling schedule, and the development of compartmental models for the interpretation of tracer data. In addition to providing insight into new metabolic pathways, such models provide quantitative information on the effect of interventions on lipoprotein metabolism. Compartment models are useful tools to describe experimental data but can also be used to aid in experimental design and hypothesis generation. The SAAM II program provides an easy-to-use interface with which to develop and test compartmental models against experimental models. The development of a model requires that certain checks be performed to ensure that the model describes the experimental data and that the model parameters can be estimated with precision. In addition to methodologic aspects, several compartment models of apoprotein and lipid metabolism are reviewed.     

Recent volcanism and mitochondrial DNA structuring in the lizard Gallotia atlantica from the island of Lanzarote

The phylogeography of the lacertid lizard Gallotia atlantica from the small volcanic island of Lanzarote (Canary Islands) was analysed based on 1075 bp of mitochondrial DNA (mtDNA) sequence (partial cytochrome b and ND2) for 157 individuals from 27 sites (including three sites from neighbouring islets). Levels of sequence divergence were generally low, with the most distant haplotypes separated by only 14 mutational steps. MtDNA divergence appears to coincide with formation of the middle Pleistocene lowland that united formerly separate ancient islands to form the current island of Lanzarote, allowing rejection of a two-island model of phylogeographical structure. There was evidence of large-scale population expansion after island unification, consistent with the colonization of new areas. A nested clade phylogeographical analysis (NCPA) revealed significant phylogeographical structuring. Two-step and higher-level clades each had disjunct distributions, being found to the east and west of a common area with a north-south orientation that extends between coasts in the centre-east of the island (El Jable). Other clades were almost entirely restricted to the El Jable region alone. Bayesian Markov chain Monte Carlo analyses were used to separate ongoing gene flow from historical associations. These supported the NCPA by indicating recent (75,000-150,000 years ago) east-west vicariance across the El Jable region. Lava flows covered El Jable and other parts of the central lowland at this time and likely led to population extinctions and temporary dispersal barriers, although present-day evidence suggests some populations would have survived in small refugia. Expansion of the latter appears to explain the presence of a clade located between the eastern and western components of the disjunct clades. Direct relationships between mtDNA lineages and morphology were not found, although one of two morphological forms on the island has a disjunct distribution that is broadly concordant with east-west components of the phylogeographical pattern. This work demonstrates how recent volcanic activity can cause population fragmentation and thus shape genetic diversity on microgeographical scales.