Is the weighted z‐test the best method for combining probabilities from independent tests?

Through simulation, Whitlock showed that when all the alternatives have the same effect size, the weighted z-test is superior to both unweighted z-test and Fisher's method when combining P-values from independent studies. In this paper, we show that under the same situation, the generalized Fisher method due to Lancaster outperforms the weighted z-test.     

Optimally weighted Z-test is a powerful method for combining probabilities in meta-analysis

The inverse normal and Fisher's methods are two common approaches for combining P-values. Whitlock demonstrated that a weighted version of the inverse normal method, or 'weighted Z-test', is superior to Fisher's method for combining P-values for one-sided T-tests. The problem with Fisher's method is that it does not take advantage of weighting and loses power to the weighted Z-test when studies are differently sized. This issue was recently revisited by Chen, who observed that Lancaster's variation of Fisher's method had higher power than the weighted Z-test. Nevertheless, the weighted Z-test has comparable power to Lancaster's method when its weights are set to square roots of sample sizes. Power can be further improved when additional information is available. Although there is no single approach that is the best in every situation, the weighted Z-test enjoys certain properties that make it an appealing choice as a combination method for meta-analysis.     

Genetic structure at three spatial scales is consistent with limited philopatry in Ricord's Rock Iguanas (Cyclura ricordii)

Cyclura ricordii is an endemic iguana from Hispaniola Island and is threatened on the IUCN Red List. The main threats are predation by introduced mammals, habitat destruction, and hunting pressure. The present study focused on two nesting sites from Pedernales Province in the Dominican Republic. The hypothesis that natal philopatry influences dispersal and nest‐site selection was tested. Monitoring and sampling took place in 2012 and 2013. Polymorphic markers were used to evaluate whether natal philopatry limits dispersal at multiple spatial scales. Ripley's K revealed that nests were significantly clustered at multiple scales, when both nesting sites were considered and within each nesting site. This suggests a patchy, nonrandom distribution of nests within nest sites. Hierarchical AMOVA revealed that nest‐site aggregations did not explain a significant portion of genetic variation within nesting sites. However, a small but positive correlation between geographic and genetic distance was detected using a Mantel's test. Hence, the relationship between geographic distance and genetic distance among hatchlings within nest sites, while detectable, was not strong enough to have a marked effect on fine‐scale genetic structure. Spatial and genetic data combined determined that the nesting sites included nesting females from multiple locations, and the hypothesis of “natal philopatry” was not supported because females nesting in the same cluster were no more closely related to each other than to other females from the same nesting site. These findings imply that nesting aggregations are more likely associated with cryptic habitat variables contributing to optimal nesting conditions.     

chifish: a computer program testing for genetic heterogeneity at multiple loci using chi‐square and Fisher's exact test

chifish is a 32‐bit Windows/DOS program evaluating divergence at multiple gene loci. It tests the hypothesis of no difference at any locus both by means of Pearson's traditional chi‐square and by using Fisher's method of combining P values obtained by Fisher's exact test. Input data are read from a file formatted for genepop . Commonly used population genetics software do not perform chi‐square tests, and the simultaneous application of both techniques aids in situations where poor power of the ‘exact approach’ may prevent detection of true differentiation (e.g. few populations and few alleles per locus).     

Test Procedures in Configural Frequency Analysis (CFA) Controlling the Local and Multiple Level

The test statistics used until now in the CFA have been developed under the assumption of the overall hypothesis of total independence. Therefore, the multiple test procedures based on these statistics are really only different tests of the overall hypothesis. If one likes to test a special cell hypothesis, one should only assume that this hypothesis is true and not the whole overall hypothesis. Such cell tests can then be used as elements of a multiple test procedure. In this paper it is shown that the usual test procedures can be very anticonservative (except of the two-dimensional, and, for some procedures, the three-dimensional case), and corrected test procedures are developed. Furthermore, for the construction of multiple tests controlling the multiple level, modifications of Holm's (1979) procedure are proposed which lead to sharper results than his general procedure and can also be performed very easily.     

An Adaptive Location‐Scale Test

Increasing locations are often accompanied by an increase in variability. In this case apparent heteroscedasticity can indicate that there are treatment effects and it is appropriate to consider an alternative involving differences in location as well as in scale. As a location‐scale test the sum of a location and a scale test statistic can be used. However, the power can be raised through weighting the sum. In order to select values for this weighting an adaptive design with an interim analysis is proposed: The data of the first stage are used to calculate the weights and with the second stage's data a weighted location‐scale test is carried out. The p‐values of the two stages are combined through Fisher's combination test. With a Lepage‐type location‐scale test it is illustrated that the resultant adaptive test can be more powerful than the ‘optimum’ test with no interim analysis. The principle to calculate weights, which cannot be reasonably chosen a priori, with the data of the first stage may be useful for other tests which utilize weighted statistics, too. Furthermore, the proposed test is illustrated with an example from experimental ecology.     

Statistical power when testing for genetic differentiation

A variety of statistical procedures are commonly employed when testing for genetic differentiation. In a typical situation two or more samples of individuals have been genotyped at several gene loci by molecular or biochemical means, and in a first step a statistical test for allele frequency homogeneity is performed at each locus separately, using, e.g. the contingency chi-square test, Fisher's exact test, or some modification thereof. In a second step the results from the separate tests are combined for evaluation of the joint null hypothesis that there is no allele frequency difference at any locus, corresponding to the important case where the samples would be regarded as drawn from the same statistical and, hence, biological population. Presently, there are two conceptually different strategies in use for testing the joint null hypothesis of no difference at any locus. One approach is based on the summation of chi-square statistics over loci. Another method is employed by investigators applying the Bonferroni technique (adjusting the P-value required for rejection to account for the elevated alpha errors when performing multiple tests simultaneously) to test if the heterogeneity observed at any particular locus can be regarded significant when considered separately. Under this approach the joint null hypothesis is rejected if one or more of the component single locus tests is considered significant under the Bonferroni criterion. We used computer simulations to evaluate the statistical power and realized alpha errors of these strategies when evaluating the joint hypothesis after scoring multiple loci. We find that the 'extended' Bonferroni approach generally is associated with low statistical power and should not be applied in the current setting. Further, and contrary to what might be expected, we find that 'exact' tests typically behave poorly when combined in existing procedures for joint hypothesis testing. Thus, while exact tests are generally to be preferred over approximate ones when testing each particular locus, approximate tests such as the traditional chi-square seem preferable when addressing the joint hypothesis.     

Tests for Homogeneity of Variances Using Robust Weighted Likelihood Estimates

The classical normal-theory tests for testing the null hypothesis of common variance and the classical estimates of scale have long been known to be quite nonrobust to even mild deviations from normality assumptions for moderate sample sizes. Levene (1960) suggested a one-way ANOVA type statistic as a robust test. Brown and Forsythe (1974) considered a modified version of Levene's test by replacing the sample means with sample medians as estimates of population locations, and their test is computationally the simplest among the three tests recommended by Conover , Johnson , and Johnson (1981) in terms of robustness and power. In this paper a new robust and powerful test for homogeneity of variances is proposed based on a modification of Levene's test using the weighted likelihood estimates (Markatou , Basu , and Lindsay , 1996) of the population means. For two and three populations the proposed test using the Hellinger distance based weighted likelihood estimates is observed to achieve better empirical level and power than Brown-Forsythe's test in symmetric distributions having a thicker tail than the normal, and higher empirical power in skew distributions under the use of F distribution critical values.     

A cautionary note on exact unconditional inference for a difference between two independent binomial proportions

Fisher's exact test for comparing response proportions in a randomized experiment can be overly conservative when the group sizes are small or when the response proportions are close to zero or one. This is primarily because the null distribution of the test statistic becomes too discrete, a partial consequence of the inference being conditional on the total number of responders. Accordingly, exact unconditional procedures have gained in popularity, on the premise that power will increase because the null distribution of the test statistic will presumably be less discrete. However, we caution researchers that a poor choice of test statistic for exact unconditional inference can actually result in a substantially less powerful analysis than Fisher's conditional test. To illustrate, we study a real example and provide exact test size and power results for several competing tests, for both balanced and unbalanced designs. Our results reveal that Fisher's test generally outperforms exact unconditional tests based on using as the test statistic either the observed difference in proportions, or the observed difference divided by its estimated standard error under the alternative hypothesis, the latter for unbalanced designs only. On the other hand, the exact unconditional test based on the observed difference divided by its estimated standard error under the null hypothesis (score statistic) outperforms Fisher's test, and is recommended. Boschloo's test, in which the p-value from Fisher's test is used as the test statistic in an exact unconditional test, is uniformly more powerful than Fisher's test, and is also recommended.     

MultiTest V.1.2, a program to binomially combine independent tests and performance comparison with other related methods on proportional data

Background  

Combining multiple independent tests, when all test the same hypothesis and in the same direction, has been the subject of several approaches. Besides the inappropriate (in this case) Bonferroni procedure, the Fisher's method has been widely used, in particular in population genetics. This last method has nevertheless been challenged by the SGM (symmetry around the geometric mean) and Stouffer's Z-transformed methods that are less sensitive to asymmetry and deviations from uniformity of the distribution of the partial P-values. Performances of these different procedures were never compared on proportional data such as those currently used in population genetics.     

Combining Global and Marginal Tests to Compare Two Treatments on Multiple Endpoints

We consider the problem of comparing two treatments on multiple endpoints where the goal is to identify the endpoints that have treatment effects, while controlling the familywise error rate. Two current approaches for this are (i) applying a global test within a closed testing procedure, and (ii) adjusting individual endpoint p‐values for multiplicity. We propose combining the two current methods. We compare the combined method with several competing methods in a simulation study. It is concluded that the combined approach maintains higher power under a variety of treatment effect configurations than the other methods and is thus more power‐robust.     

Statistical tests for clonality

Cancer investigators frequently conduct studies to examine tumor samples from pairs of apparently independent primary tumors with a view to determine whether they share a "clonal" origin. The genetic fingerprints of the tumors are compared using a panel of markers, often representing loss of heterozygosity (LOH) at distinct genetic loci. In this article we evaluate candidate significance tests for this purpose. The relevant information is derived from the observed correlation of the tumors with respect to the occurrence of LOH at individual loci, a phenomenon that can be evaluated using Fisher's exact test. Information is also available from the extent to which losses at the same locus occur on the same parental allele. Data from these combined sources of information can be evaluated using a simple adaptation of Fisher's exact test. The test statistic is the total number of loci at which concordant mutations occur on the same parental allele, with higher values providing more evidence in favor of a clonal origin for the two tumors. The test is shown to have high power for detecting clonality for plausible models of the alternative (clonal) hypothesis, and for reasonable numbers of informative loci, preferably located on distinct chromosomal arms. The method is illustrated using studies to identify clonality in contralateral breast cancer. Interpretation of the results of these tests requires caution due to simplifying assumptions regarding the possible variability in mutation probabilities between loci, and possible imbalances in the mutation probabilities between parental alleles. Nonetheless, we conclude that the method represents a simple, powerful strategy for distinguishing independent tumors from those of clonal origin.     

Incorporating Correlation for Multivariate Failure Time Data When Cluster Size Is Large

Summary We propose a new estimation method for multivariate failure time data using the quadratic inference function (QIF) approach. The proposed method efficiently incorporates within‐cluster correlations. Therefore, it is more efficient than those that ignore within‐cluster correlation. Furthermore, the proposed method is easy to implement. Unlike the weighted estimating equations in Cai and Prentice (1995, Biometrika 82 , 151–164), it is not necessary to explicitly estimate the correlation parameters. This simplification is particularly useful in analyzing data with large cluster size where it is difficult to estimate intracluster correlation. Under certain regularity conditions, we show the consistency and asymptotic normality of the proposed QIF estimators. A chi‐squared test is also developed for hypothesis testing. We conduct extensive Monte Carlo simulation studies to assess the finite sample performance of the proposed methods. We also illustrate the proposed methods by analyzing primary biliary cirrhosis (PBC) data.     

Power for detecting genetic divergence: differences between statistical methods and marker loci

Information on statistical power is critical when planning investigations and evaluating empirical data, but actual power estimates are rarely presented in population genetic studies. We used computer simulations to assess and evaluate power when testing for genetic differentiation at multiple loci through combining test statistics or P values obtained by four different statistical approaches, viz. Pearson's chi-square, the log-likelihood ratio G-test, Fisher's exact test, and an F(ST)-based permutation test. Factors considered in the comparisons include the number of samples, their size, and the number and type of genetic marker loci. It is shown that power for detecting divergence may be substantial for frequently used sample sizes and sets of markers, also at quite low levels of differentiation. The choice of statistical method may be critical, though. For multi-allelic loci such as microsatellites, combining exact P values using Fisher's method is robust and generally provides a high resolving power. In contrast, for few-allele loci (e.g. allozymes and single nucleotide polymorphisms) and when making pairwise sample comparisons, this approach may yield a remarkably low power. In such situations chi-square typically represents a better alternative. The G-test without Williams's correction frequently tends to provide an unduly high proportion of false significances, and results from this test should be interpreted with great care. Our results are not confined to population genetic analyses but applicable to contingency testing in general.     

A Robust Test for Multi-Sample Location Problems with Unequal Group Variances and Nonnormality

A robust test (to be referred to as M^* test) is proposed for testing equality of several group means without assuming normality and equality of variances. This test statistic is obtained by combining Tiku's MML robust procedure with the James statistic. Monte Carlo simulation studies indicate that the M^* test is more powerful than the Welch test, the James test, and the tests based on Huber's M-estimators over a wide range of nonnormal universes. It is also more powerful than the Brown and Forsythe test under most of nonnormal distributions and has substantially the same power as the Brown and Forsythe test under normal distribution. Comparing with Tan-Tabatabai test, M^* is almost as powerful as Tan-Tabatabai test.     

Relationship between functional diversity and productivity in meadow and marsh plant communities

功能多样性-生产力关系研究结果支持质量比假说和多样性假说, 但对于这两种假说的适用条件尚有争议。通过对吉林省西部草甸和沼泽植物群落的地上生物量、2个物种多样性指标(物种丰富度和Shannon-Weaver指数)、7种植物性状的两类功能多样性指标(群落权重均值和Rao二次熵), 以及土壤环境因子进行调查测量, 研究了群落功能多样性与生产力的关系。结果表明: 1)功能多样性与生产力的关系比物种多样性与生产力的关系更为密切; 2)功能群落权重均值解释生产力变异的能力好于Rao二次熵, 即优势物种对群落生产力的影响作用更大; 3)水淹条件影响着功能多样性与生产力的关系, 以群落权重均值为基础的质量比假说适于解释草甸群落功能多样性与生产力的关系, 而以Rao二次熵为基础的多样性假说适于解释有强烈环境筛(水淹)的沼泽群落功能多样性与生产力的关系。     

草甸和沼泽植物群落功能多样性与生产力的关系

功能多样性-生产力关系研究结果支持质量比假说和多样性假说, 但对于这两种假说的适用条件尚有争议。通过对吉林省西部草甸和沼泽植物群落的地上生物量、2个物种多样性指标(物种丰富度和Shannon-Weaver指数)、7种植物性状的两类功能多样性指标(群落权重均值和Rao二次熵), 以及土壤环境因子进行调查测量, 研究了群落功能多样性与生产力的关系。结果表明: 1)功能多样性与生产力的关系比物种多样性与生产力的关系更为密切; 2)功能群落权重均值解释生产力变异的能力好于Rao二次熵, 即优势物种对群落生产力的影响作用更大; 3)水淹条件影响着功能多样性与生产力的关系, 以群落权重均值为基础的质量比假说适于解释草甸群落功能多样性与生产力的关系, 而以Rao二次熵为基础的多样性假说适于解释有强烈环境筛(水淹)的沼泽群落功能多样性与生产力的关系。     

A comparison of eight methods for the dual-endpoint evaluation of efficacy in a proof-of-concept HIV vaccine trial

To support the design of the world's first proof-of-concept (POC) efficacy trial of a cell-mediated immunity-based HIV vaccine, we evaluate eight methods for testing the composite null hypothesis of no-vaccine effect on either the incidence of HIV infection or the viral load set point among those infected, relative to placebo. The first two methods use a single test applied to the actual values or ranks of a burden-of-illness (BOI) outcome that combines the infection and viral load endpoints. The other six methods combine separate tests for the two endpoints using unweighted or weighted versions of the two-part z, Simes', and Fisher's methods. Based on extensive simulations that were used to design the landmark POC trial, the BOI methods are shown to have generally low power for rejecting the composite null hypothesis (and hence advancing the vaccine to a subsequent large-scale efficacy trial). The unweighted Simes' and Fisher's combination methods perform best overall. Importantly, this conclusion holds even after the test for the viral load component is adjusted for bias that can be introduced by conditioning on a postrandomization event (HIV infection). The adjustment is derived using a selection bias model based on the principal stratification framework of causal inference.     

Statistical test for the comparison of samples from mutational spectra

The Monte Carlo estimate of the p value of the hypergeometric test is described and advocated for the testing of the hypothesis that different treatments induce the same mutational spectrum. The hypergeometric test is a generalization of Fisher's "exact" test for tables with more than two rows and two columns. Use of the test is demonstrated by the analysis of data from the characterization of nonsense mutations in the lacI gene of Escherichia coli. Unlike the chi-square test, the hypergeometric test remains valid when applied to sparse cross-classification tables. The hypergeometric test has the most discrimination power of any statistical test that could be employed routinely to compare samples from mutational spectra. Direct application of the hypergeometric test to large cross-classification tables is excessively computation intensive, but estimation of its p value via Monte Carlo techniques is practical.     

Statistical Methods for Integrating Randomized Clinical Studies for Evaluation of Combination Treatment Therapy

In regulatory applications, evaluation of a combination drug at a fixed dose is based on pairwise comparisons of the combination with its component drugs. These comparisons generate the least expected gain of the combination relative to its components as the key effect parameter for evaluation. Two test methods are developed for the evaluation to combine multiple clinical studies that are deemed combinable. One test method is based on a linear combination of the Min tests from individual studies. In the other test method, weighted estimators are first derived for the pairwise comparisons between the combination and the respective components by combining the studies. A Min test is then applied to these estimators. The latter test method tends to be more powerful than the former test method. A test-based confidence interval is constructed for the least expected gain of the combination relative to its components. A test for heterogeneity across studies is also developed.