EMPIRICAL COMPARISON OF G MATRIX TEST STATISTICS: FINDING BIOLOGICALLY RELEVANT CHANGE

A central assumption of quantitative genetic theory is that the breeder's equation ( R = GP ⁻¹ S ) accurately predicts the evolutionary response to selection. Recent studies highlight the fact that the additive genetic variance–covariance matrix ( G ) may change over time, rendering the breeder's equation incapable of predicting evolutionary change over more than a few generations. Although some consensus on whether G changes over time has been reached, multiple, often-incompatible methods for comparing G matrices are currently used. A major challenge of G matrix comparison is determining the biological relevance of observed change. Here, we develop a "selection skewers" G matrix comparison statistic that uses the breeder's equation to compare the response to selection given two G matrices while holding selection intensity constant. We present a bootstrap algorithm that determines the significance of G matrix differences using the selection skewers method, random skewers, Mantel's and Bartlett's tests, and eigenanalysis. We then compare these methods by applying the bootstrap to a dataset of laboratory populations of Tribolium castaneum . We find that the results of matrix comparison statistics are inconsistent based on differing a priori goals of each test, and that the selection skewers method is useful for identifying biologically relevant G matrix differences.     

POOR STATISTICAL PERFORMANCE OF THE MANTEL TEST IN PHYLOGENETIC COMPARATIVE ANALYSES

The Mantel test, based on comparisons of distance matrices, is commonly employed in comparative biology, but its statistical properties in this context are unknown. Here, we evaluate the performance of the Mantel test for two applications in comparative biology: testing for phylogenetic signal, and testing for an evolutionary correlation between two characters. We find that the Mantel test has poor performance compared to alternative methods, including low power and, under some circumstances, inflated type‐I error. We identify a remedy for the inflated type‐I error of three‐way Mantel tests using phylogenetic permutations; however, this test still has considerably lower power than independent contrasts. We recommend that use of the Mantel test should be restricted to cases in which data can only be expressed as pairwise distances among taxa.     

Using simulations to evaluate Mantel‐based methods for assessing landscape resistance to gene flow

Mantel‐based tests have been the primary analytical methods for understanding how landscape features influence observed spatial genetic structure. Simulation studies examining Mantel‐based approaches have highlighted major challenges associated with the use of such tests and fueled debate on when the Mantel test is appropriate for landscape genetics studies. We aim to provide some clarity in this debate using spatially explicit, individual‐based, genetic simulations to examine the effects of the following on the performance of Mantel‐based methods: (1) landscape configuration, (2) spatial genetic nonequilibrium, (3) nonlinear relationships between genetic and cost distances, and (4) correlation among cost distances derived from competing resistance models. Under most conditions, Mantel‐based methods performed poorly. Causal modeling identified the true model only 22% of the time. Using relative support and simple Mantel r values boosted performance to approximately 50%. Across all methods, performance increased when landscapes were more fragmented, spatial genetic equilibrium was reached, and the relationship between cost distance and genetic distance was linearized. Performance depended on cost distance correlations among resistance models rather than cell‐wise resistance correlations. Given these results, we suggest that the use of Mantel tests with linearized relationships is appropriate for discriminating among resistance models that have cost distance correlations <0.85 with each other for causal modeling, or <0.95 for relative support or simple Mantel r. Because most alternative parameterizations of resistance for the same landscape variable will result in highly correlated cost distances, the use of Mantel test‐based methods to fine‐tune resistance values will often not be effective.     

Performance of partial statistics in individual‐based landscape genetics

Individual‐based landscape genetic methods have become increasingly popular for quantifying fine‐scale landscape influences on gene flow. One complication for individual‐based methods is that gene flow and landscape variables are often correlated with geography. Partial statistics, particularly Mantel tests, are often employed to control for these inherent correlations by removing the effects of geography while simultaneously correlating measures of genetic differentiation and landscape variables of interest. Concerns about the reliability of Mantel tests prompted this study, in which we use simulated landscapes to evaluate the performance of partial Mantel tests and two ordination methods, distance‐based redundancy analysis (dbRDA) and redundancy analysis (RDA), for detecting isolation by distance (IBD) and isolation by landscape resistance (IBR). Specifically, we described the effects of suitable habitat amount, fragmentation and resistance strength on metrics of accuracy (frequency of correct results, type I/II errors and strength of IBR according to underlying landscape and resistance strength) for each test using realistic individual‐based gene flow simulations. Mantel tests were very effective for detecting IBD, but exhibited higher error rates when detecting IBR. Ordination methods were overall more accurate in detecting IBR, but had high type I errors compared to partial Mantel tests. Thus, no one test outperformed another completely. A combination of statistical tests, for example partial Mantel tests to detect IBD paired with appropriate ordination techniques for IBR detection, provides the best characterization of fine‐scale landscape genetic structure. Realistic simulations of empirical data sets will further increase power to distinguish among putative mechanisms of differentiation.     

Mantel test in population genetics

The comparison of genetic divergence or genetic distances, estimated by pairwise F_ST and related statistics, with geographical distances by Mantel test is one of the most popular approaches to evaluate spatial processes driving population structure. There have been, however, recent criticisms and discussions on the statistical performance of the Mantel test. Simultaneously, alternative frameworks for data analyses are being proposed. Here, we review the Mantel test and its variations, including Mantel correlograms and partial correlations and regressions. For illustrative purposes, we studied spatial genetic divergence among 25 populations of Dipteryx alata (“Baru”), a tree species endemic to the Cerrado, the Brazilian savannas, based on 8 microsatellite loci. We also applied alternative methods to analyze spatial patterns in this dataset, especially a multivariate generalization of Spatial Eigenfunction Analysis based on redundancy analysis. The different approaches resulted in similar estimates of the magnitude of spatial structure in the genetic data. Furthermore, the results were expected based on previous knowledge of the ecological and evolutionary processes underlying genetic variation in this species. Our review shows that a careful application and interpretation of Mantel tests, especially Mantel correlograms, can overcome some potential statistical problems and provide a simple and useful tool for multivariate analysis of spatial patterns of genetic divergence.     

Comparison of the Mantel test and alternative approaches for detecting complex multivariate relationships in the spatial analysis of genetic data

The Mantel test is widely used to test the linear or monotonic independence of the elements in two distance matrices. It is one of the few appropriate tests when the hypothesis under study can only be formulated in terms of distances; this is often the case with genetic data. In particular, the Mantel test has been widely used to test for spatial relationship between genetic data and spatial layout of the sampling locations. We describe the domain of application of the Mantel test and derived forms. Formula development demonstrates that the sum-of-squares (SS) partitioned in Mantel tests and regression on distance matrices differs from the SS partitioned in linear correlation, regression and canonical analysis. Numerical simulations show that in tests of significance of the relationship between simple variables and multivariate data tables, the power of linear correlation, regression and canonical analysis is far greater than that of the Mantel test and derived forms, meaning that the former methods are much more likely than the latter to detect a relationship when one is present in the data. Examples of difference in power are given for the detection of spatial gradients. Furthermore, the Mantel test does not correctly estimate the proportion of the original data variation explained by spatial structures. The Mantel test should not be used as a general method for the investigation of linear relationships or spatial structures in univariate or multivariate data. Its use should be restricted to tests of hypotheses that can only be formulated in terms of distances.     

An empirical assessment of saddlepoint approximations for testing a logistic regression parameter.

Saddlepoint methods provide quick and easy approximations to significance levels for conditional tests of logistic regression parameters. We evaluate the accuracies of saddlepoint approximations for three well-known conditional tests: Bartlett's test for no three-factor interaction in a 2 x 2 x 2 table, the test for trend in a series of probabilities, and the exact test of no association in stratified 2 x 2 tables with a common odds ratio. General recommendations are suggested regarding the use of saddlepoint approximations for exact conditional significance levels.     

Individual assignment test reveals differential restriction to dispersal between two salmonids despite no increase of genetic differences with distance

In many species genes move over limited distances, such that genetic differences among populations or individuals are expected to increase as a function of geographical distance. In other species, however, genes may move any distance over a single generation time, such that no increase of genetic differences is expected to occur with distance. Patterns of gene dispersal have been assessed typically using this theoretical property. In this study, this classical approach based on a Mantel test was compared to a new method using individual assignment to reveal contrasts in dispersal patterns between 15 populations of brook charr Salvelinus fontinalis and 10 populations of Atlantic salmon Salmo salar sampled in eastern Canada, where both species co-occur naturally. Based on the Mantel test, we found evidence for neither an increase of genetic differences with distance in either species nor a significant contrast between them. The individual-based method, in contrast, revealed that individual assignment in both species was non random, being significantly biased toward geographically proximate locations. Furthermore, brook charr were on average assigned to a closer river than were salmon, according to a priori expectations based on the dispersal behaviour of the two species. We thus propose that individual assignment methods might be a promising and more powerful alternative to Mantel tests when isolation by distance cannot be postulated a priori.     

Spatial distribution and environmental correlates of Australian snubfin and Indo-Pacific humpback dolphins

We present data on the spatial distribution of Australian snubfin and humpback dolphins using boat-based line transect surveys in three adjacent bays located in the Far Northern Section of the Great Barrier Reef Marine Park, northeast Queensland. We used Geographic Information Systems (GIS), and both randomization and Mantel tests to examine the relationship between the spatial distribution of the dolphins and three simple, readily quantified, environmental variables: distance to land, distance to river mouth, and water depth. Mantel tests allowed us to make clear inferences about the correlation of the species' distributions with environmental variables, while taking into account spatial autocorrelation and intercorrelation among variables. Randomization tests indicated snubfin and humpback dolphins occur closer to land than would be expected at random. Two-sample randomization tests indicated snubfin dolphins were found closer to river mouths than were humpback dolphins. Taking spatial autocorrelation into account, Mantel tests indicated all environmental variables were correlated with the spatial distribution of snubfin and humpback dolphins. Interspecific differences in spatial distribution appeared to be related to proximity to river mouths. Preference by snubfin and humpback dolphins for nearshore, estuarine waters is likely related to the productivity of these tropical coastal areas. This spatial analysis suggests that existing protected areas in this region may not include the most critical habitats for snubfin and humpback dolphins. The techniques used here shown relationships between the spatial distribution of the dolphins and environmental features that should facilitate their management and conservation.     

Applying randomization tests to cluster analyses

Abstract. In applying randomization tests to hierarchical cluster analyses, we have noted a potentially misleading result: within a significant group, linkages are often identified as significant even when species are randomly distributed among the group's sites. We demonstrate this through a cluster analysis of a constructed matrix with two groups of 20 sites that share no species, and within each group species are randomly distributed among sites. A randomization test identified both of the groups and all linkages within them as significant, while the same test found all linkages non‐significant in the cluster analysis of a matrix containing just one of the two groups of 20 sites. In general, a non‐random distribution of species within a data set shortens linkages relative to distances in null distributions derived from randomized versions of the data. This confounds efforts to identify significant sub‐groups within a significant group. However, the significance of sub‐groups possibly could be tested by comparing linkage distances to a null distribution derived from the randomization and clustering of a sub‐matrix containing only the sites within the larger group. In essence, this comparison tests the null hypothesis that within the significant group, sites represent random assemblages of species. When applied to actual data sets, an approach involving sequential randomization tests could allow the evaluation of all nodes in a classification, increasing the utility of randomization tests and strengthening the interpretation of groups produced by cluster analysis.     

Phylogenetic signal in bone microstructure of sauropsids

In spite of the fact that the potential usefulness of bone histology in systematics has been discussed for over one and a half centuries, the presence of a phylogenetic signal in the variation of histological characters has rarely been assessed. A quantitative assessment of phylogenetic signal in bone histological characters could provide a justification for performing optimizations of these traits onto independently generated phylogenetic trees (as has been done in recent years). Here we present an investigation on the quantification of the phylogenetic signal in the following bone histological, microanatomical, and morphological traits in a sample of femora of 35 species of sauropsids: vascular density, vascular orientation, index of Haversian remodeling, cortical thickness, and cross-sectional area (bone size). For this purpose, we use two methods, regressions on distance matrices tested for significance using permutations (a Mantel test) and random tree length distribution. Within sauropsids, these bone microstructural traits have an optimal systematic value in archosaurs. In this taxon, a Mantel test shows that the phylogeny explains 81.8% of the variation of bone size and 86.2% of the variation of cortical thickness. In contrast, a Mantel test suggests that the phylogenetic signal in histological traits is weak: although the phylogeny explains 18.7% of the variation of vascular density in archosaurs, the phylogenetic signal is not significant either for vascular orientation or for the index of Haversian remodeling. However, Mantel tests seem to underestimate the proportion of variance of the dependent character explained by the phylogeny, as suggested by a PVR (phylogenetic eigenvector) analysis. We also deal with some complementary questions. First, we evaluate the functional dependence of bone vascular density on bone size by using phylogenetically independent contrasts. Second, we perform a variation partitioning analysis and show that the phylogenetic signal in bone vascular density is not a by-product of phylogentic signal in bone size. Finally, we analyze the evolution of cortical thickness in diapsids by using an optimization by squared change parsimony and discuss the functional significance of this character in terms of decreased buoyancy in crocodiles and mass saving in birds. These results are placed in the framework of the constructional morphology model, according to which the variation of a character in a clade has a historical (phylogenetic) component, a functional (adaptive) component, and a structural (architectural) component.     

Habitat niche specialization in an understory species in a warm temperate forest

Relationships between microhabitat variables; understory light conditions in summer and winter, altitude, slope inclination and topographic categories (valley, ridge, and slope) and the distribution of Aucuba japonica Thunb. (Cornaceae), a common understory shrub species in Japan were examined using non-contagious 66, 20 × 20 m² quadrats. The Morishita’s I _δ suggested that A. japonica distributions were strongly heterogeneous among the quadrats. Therefore positive spatial autocorrelation of A. japonica at a within-quadrat level (≤20 m) was obvious. Moran’s I statistics showed a significant positive spatial autocorrelation in A. japonica abundance within the distance shorter than 60 m. But the partial Mantel tests suggested that the mass effect from neighboring quadrats would little explain A. japonica abundance in a quadrat. The partial Mantel tests also clearly showed that A. japonica distributions were strongly structured by topography and understory light conditions. Using Monte Carlo randomization tests, we found that A. japonica was aggregately distributed in quadrats in valley which were covered by deciduous canopies. Better understory light conditions in winter together with valley edaphic conditions may increase the abundance of A. japonica there. It is concluded that habitat niche specialization is important in structuring distribution of A. japonica in this forest community.     

Randomization Analyses: Mimicry, Geographic Variation and Cultural Evolution of Song in Brood-Parasitic Straw-Tailed Whydahs, Vidua fischeri

Although bird song has been an important model for investigating questions of behavior development, cultural evolution and population differentiation, the quantitative methods of analysis have been problematic. Here we develop and apply quantitative randomization methods to test hypotheses about these processes in a natural population of birds. Songs of the African brood-parasitic straw-tailed whydahs ( Vidua fischeri ) and songs of their host species, the purple grenadier ( Granatina ianthinogaster ), were compared in audiospectrograms for similarity to test the following hypotheses: Whydahs mimic the songs of their host species, they have local song dialects, neighboring males match their song themes, local males match the songs of local hosts, remote populations have different songs according to their geographic distance, and songs undergo cultural evolution over time across generations. Randomization analyses were completed using (1) Mantel matrix statistics and (2) tree-based measures employing Sankoff optimization of Manhattan matrices and approximate randomizations. Our results provide evidence for song mimicry, local song dialects, matching song themes between neighboring males, song matching of local whydah mimics and grenadier song models, correspondence of song differences and geographic distance, and cultural continuity with change in song traditions within a local population. These randomization methods may be useful in other studies of animal communication, and they are sufficiently general for use both with distance matrices derived either from naturalistic impressions of song similarity as in our example or from acoustic measurements.     

Assessing the robustness of randomization tests: examples from behavioural studies

Behavioural studies are commonly plagued with data that violate the assumptions of parametric statistics. Consequently, classic nonparametric methods (e.g. rank tests) and novel distribution-free methods (e.g. randomization tests) have been used to a great extent by behaviourists. However, the robustness of such methods in terms of statistical power and type I error have seldom been evaluated. This probably reflects the fact that empirical methods, such as Monte Carlo approaches, are required to assess these concerns. In this study we show that analytical methods cannot always be used to evaluate the robustness of statistical tests, but rather Monte Carlo approaches must be employed. We detail empirical protocols for estimating power and type I error rates for parametric, nonparametric and randomization methods, and demonstrate their application for an analysis of variance and a regression/correlation analysis design. Together, this study provides a framework from which behaviourists can compare the reliability of different methods for data analysis, serving as a basis for selecting the most appropriate statistical test given the characteristics of data at hand. Copyright 2001 The Association for the Study of Animal Behaviour.     

Tests for Monotonic Trend from Case‐Control Data: Cochran‐Armitage‐Mantel Trend Test,Isotonic Regression and Single and Multiple Contrast Tests

Tests for a monotonic trend between an ordered categorical exposure and disease status are routinely carried out from case‐control data using the Mantel‐extension trend test or the asymptotically equivalent Cochran‐Armitage test. In this study, we considered two alternative tests based on isotonic regression, namely an order‐restricted likelihood ratio test and an isotonic modification of the Mantel‐extension test extending the recent proposal by Mancuso, Ahn and Chen (2001) to case‐control data. Furthermore, we considered three tests based on contrasts, namely a single contrast (SC) test based on Schaafsma's coefficients, the Dosemeci and Benichou (DB) test, a multiple contrast (MC) test based on the Helmert, reverse‐Helmert and linear contrasts and we derived their case‐control versions. Using simulations, we compared the statistical properties of these five alternative tests to those of the Mantel‐extension test under various patterns including no relationship, as well as monotonic and non‐monotonic relationships between exposure and disease status. In the case of no relationship, all tests had close to nominal type I error except in situations combining a very unbalanced exposure distribution and small sample size, where the asymptotic versions of the three tests based on contrasts were highly anticonservative. The use of bootstrap instead of asymptotic versions corrected this anticonservatism. For monotonic patterns, all tests had close powers. For non monotonic patterns, the DB‐test showed the most favourable results as it was the least powerful test. The two tests based on isotonic regression were the most powerful tests and the Mantel‐extension test, the SC‐ and MC‐tests had in‐between powers. The six tests were applied to data from a case‐control study investigating the relationship between alcohol consumption and risk of laryngeal cancer in Turkey. In situations with no evidence of a monotonic relationship between exposure and disease status, the three tests based on contrasts did not conclude in favour of a significant trend whereas all the other tests did. (© 2004 WILEY‐VCH Verlag GmbH & Co. KGaA, Weinheim)     

Hypertonic Sabouraud Dextrose Agar as a Substrate for Differentiation of Candida dubliniensis

In this study, we aimed to detect the proportion of Candida dubliniensis among yeast strains previously identified as C. albicans by using several phenotypic methods and PCR. For this purpose, we screened 300 strains by using phenotypic tests suggested for the identification of C. dubliniensis in the literature, but we detected high proportion of false-positive reactions. Only two strains (0.6%) were detected as true C. dubliniensis by PCR and API ID 32C methods. Moreover, these two strains gave the expected results with all the phenotypic tests, including modified salt tolerance test for C. dubliniensis. In conclusion, none of the phenotypic methods, except for the modified salt tolerance test, revealed 100% successful results in discrimination of C. albicans and C. dubliniensis species. However, in the tobacco agar test, the rate of false positivity was as low as 0.6%. We suggest that in the case of absence of PCR and other automatized identification systems, these two phenotypic tests can be used in routine laboratories to obtain a presumptive result.     

Evolutionary Models and Phylogenetic Signal Assessment via Mantel Test

Phylogenetically closely related species tend to be more similar to each other than to more distantly related ones, a pattern called phylogenetic signal. Appropriate tests to evaluate the association between phylogenetic relatedness and trait variation among species are employed in a myriad of eco-evolutionary studies. However, most tests available to date are only suitable for datasets describing continuous traits, and are most often applicable only for single trait analysis. The Mantel test is a useful method to measure phylogenetic signal for multiple (continuous, binary and/or categorical) traits. However, the classical Mantel test does not incorporate any evolutionary model (EM) in the analysis. Here, we describe a new analytical procedure, which incorporates explicitly an evolutionary model in the standard Mantel test (EM-Mantel). We run numerical simulations to evaluate its statistical properties, under different combinations of species pool size, trait type and number. Our results showed that EM-Mantel test has appropriate type I error and acceptable power, which increases with the strength of phylogenetic signal and with species pool size but depended on trait type. EM-Mantel test is a good alternative for measuring phylogenetic signal in binary and categorical traits and for datasets with multiple traits.     

Significance Levels of Randomization Trend Tests in the Event of Rare Occurrences

In the statistical evaluation of data from a dose-response experiment, it is frequently of interest to test for dose-related trend: an increasing trend in response with increasing dose. The randomization trend test, a generalization of Fisher's exact test, has been recommended for animal tumorigenicity testing when the numbers of tumor occurrences are small. This paper examines the type I error of the randomization trend test, and the Cochran-Armitage and Mantel-Haenszel tests. Simulation results show that when the tumor incidence rates are less than 10%, the randomization test is conservative; the test becomes very conservative when the incidence rate is less than 5%. The Cochran-Armitage and Mantel-Haenszel tests are slightly anti-conservative (liberal) when the incidence rates are larger than 3%. Further, we propose a less conservatived method of calculating the p-value of the randomization trend test by excluding some permutations whose probabilities of occurrence are greater than the probability of the the observed outcome.     

Multicategory individualized treatment regime using outcome weighted learning

Individualized treatment regimes (ITRs) aim to recommend treatments based on patient‐specific characteristics in order to maximize the expected clinical outcome. Outcome weighted learning approaches have been proposed for this optimization problem with primary focus on the binary treatment case. Many require assumptions of the outcome value or the randomization mechanism. In this paper, we propose a general framework for multicategory ITRs using generic surrogate risk. The proposed method accommodates the situations when the outcome takes negative value and/or when the propensity score is unknown. Theoretical results about Fisher consistency, excess risk, and risk consistency are established. In practice, we recommend using differentiable convex loss for computational optimization. We demonstrate the superiority of the proposed method under multinomial deviance risk to some existing methods by simulation and application on data from a clinical trial.     

Pair-switching rerandomization

Rerandomization discards assignments with covariates unbalanced in the treatment and control groups to improve estimation and inference efficiency. However, the acceptance-rejection sampling method used in rerandomization is computationally inefficient. As a result, it is time-consuming for rerandomization to draw numerous independent assignments, which are necessary for performing Fisher randomization tests and constructing randomization-based confidence intervals. To address this problem, we propose a pair-switching rerandomization (PSRR) method to draw balanced assignments efficiently. We obtain the unbiasedness and variance reduction of the difference-in-means estimator and show that the Fisher randomization tests are valid under PSRR. Moreover, we propose an exact approach to invert Fisher randomization tests to confidence intervals, which is faster than the existing methods. In addition, our method is applicable to both nonsequentially and sequentially randomized experiments. We conduct comprehensive simulation studies to compare the finite-sample performance of the proposed method with that of classical rerandomization. Simulation results indicate that PSRR leads to comparable power of Fisher randomization tests and is 3–23 times faster than classical rerandomization. Finally, we apply the PSRR method to analyze two clinical trial datasets, both of which demonstrate the advantages of our method.