A Comparison of the Three Conditional Exact Tests in Two‐way Contingency Tables Using the Unconditional Exact Power

The conditional exact tests of homogeneity of two binomial proportions are often used in small samples, because the exact tests guarantee to keep the size under the nominal level. The Fisher's exact test, the exact chi‐squared test and the exact likelihood ratio test are popular and can be implemented in software StatXact. In this paper we investigate which test is the best in small samples in terms of the unconditional exact power. In equal sample cases it is proved that the three tests produce the same unconditional exact power. A symmetry of the unconditional exact power is also found. In unequal sample cases the unconditional exact powers of the three tests are computed and compared. In most cases the Fisher's exact test turns out to be best, but we characterize some cases in which the exact likelihood ratio test has the highest unconditional exact power. (© 2004 WILEY‐VCH Verlag GmbH & Co. KGaA, Weinheim)     

Nonparametric Two-Sample Tests for General Clinical Data

Starting from the discussion of a practical example a unifying concept for the derivation of meaningfully interpretable nonparametric tests for the two-sample case is developed which may well be adapted for other designs, too. This methodology covers other well-known procedures, e.g. Fisher's exact test, the Wilcoxon-Mann-Whitney and Gehan's tests, and may furthermore be extended to all situations sharing the same fundamental structural property of the sample space, namely its strict order induced by the substantial problem under study. The resulting test procedure is discussed for a randomization argument, exact and approximate, as well as for the general specific test problem. A numerical example is provided.     

chifish: a computer program testing for genetic heterogeneity at multiple loci using chi‐square and Fisher's exact test

chifish is a 32‐bit Windows/DOS program evaluating divergence at multiple gene loci. It tests the hypothesis of no difference at any locus both by means of Pearson's traditional chi‐square and by using Fisher's method of combining P values obtained by Fisher's exact test. Input data are read from a file formatted for genepop . Commonly used population genetics software do not perform chi‐square tests, and the simultaneous application of both techniques aids in situations where poor power of the ‘exact approach’ may prevent detection of true differentiation (e.g. few populations and few alleles per locus).     

Exact Test Procedures for Generalized Kendall Correlation Coefficients

An exact test of KENDALL'S tau is presented, useful in small samples when the asymptotic test may not be adequate. The procedure is proposed especially for situations when ties and/or censored observations are present in the variables to be analyzed by KENDALL'S tau. The paper describes generalizations of the procedure, presents calculated examples and offers a computer program, which facilitates the usage of the proposed procedure.     

Power for detecting genetic divergence: differences between statistical methods and marker loci

Information on statistical power is critical when planning investigations and evaluating empirical data, but actual power estimates are rarely presented in population genetic studies. We used computer simulations to assess and evaluate power when testing for genetic differentiation at multiple loci through combining test statistics or P values obtained by four different statistical approaches, viz. Pearson's chi-square, the log-likelihood ratio G-test, Fisher's exact test, and an F(ST)-based permutation test. Factors considered in the comparisons include the number of samples, their size, and the number and type of genetic marker loci. It is shown that power for detecting divergence may be substantial for frequently used sample sizes and sets of markers, also at quite low levels of differentiation. The choice of statistical method may be critical, though. For multi-allelic loci such as microsatellites, combining exact P values using Fisher's method is robust and generally provides a high resolving power. In contrast, for few-allele loci (e.g. allozymes and single nucleotide polymorphisms) and when making pairwise sample comparisons, this approach may yield a remarkably low power. In such situations chi-square typically represents a better alternative. The G-test without Williams's correction frequently tends to provide an unduly high proportion of false significances, and results from this test should be interpreted with great care. Our results are not confined to population genetic analyses but applicable to contingency testing in general.     

A sequential test for assessing observed agreement between raters

Assessing the agreement between two or more raters is an important topic in medical practice. Existing techniques, which deal with categorical data, are based on contingency tables. This is often an obstacle in practice as we have to wait for a long time to collect the appropriate sample size of subjects to construct the contingency table. In this paper, we introduce a nonparametric sequential test for assessing agreement, which can be applied as data accrues, does not require a contingency table, facilitating a rapid assessment of the agreement. The proposed test is based on the cumulative sum of the number of disagreements between the two raters and a suitable statistic representing the waiting time until the cumulative sum exceeds a predefined threshold. We treat the cases of testing two raters' agreement with respect to one or more characteristics and using two or more classification categories, the case where the two raters extremely disagree, and finally the case of testing more than two raters' agreement. The numerical investigation shows that the proposed test has excellent performance. Compared to the existing methods, the proposed method appears to require significantly smaller sample size with equivalent power. Moreover, the proposed method is easily generalizable and brings the problem of assessing the agreement between two or more raters and one or more characteristics under a unified framework, thus providing an easy to use tool to medical practitioners.     

Statistical Versus Biological Hypothesis Testing: Response to Steidl

ABSTRACT In spite of the wide use and acceptance of information theoretic approaches in the wildlife sciences, debate continues on the correct use and interpretation of Akaike's Information Criterion as compared to frequentist methods. Misunderstandings as to the fundamental nature of such comparisons continue. Here we agree with Steidl's argument about situation-specific use of each approach. However, Steidl did not make clear the distinction between statistical and biological hypotheses. Certainly model selection is not statistical, or null, hypothesis testing; importantly, it represents a more effective means to test among competing biological, or research, hypotheses. Employed correctly, it leads to superior strength of inference and reduces the risk that favorite hypotheses are uncritically accepted.     

An Analogue of Jonckheere's Trend Test for Parametric and Dichotomous Data

The Jonckheere test is a widely used test for trend in the nonparametric location model. We present an analogue of Jonckheere's test which can be performed both for normally and binomially distributed endpoints. This test is a contrast test, therefore, we can also construct a reverse test. It is shown that in several situations the proposed tests are superior to the Helmert and the reverse-Helmert contrast tests in terms of size and power, especially for finite dichotomous data. The tests are applied to data of two preclinical studies.     

Nonparametric Tests for Trend: Jonckheere's Test,a Modification and a Maximum Test

Jonckheere's test is a frequently used nonparametric trend test for the evaluation of preclinical studies and clinical dose-finding trials. In this paper, a modification of Jonckheere's test is proposed. If the exact permutation distribution is used for inference, the modified test can fill out the level of the type I error in a much more complete way and is substantially more powerful than the common Jonckheere test. If the asymptotic normality is used for inference, the modified test is slightly more powerful. In addition, a maximum test is investigated which is more robust concerning an a priori unknown dose-response shape. The robustness is advantageous, especially in a closed testing procedure. The different tests are applied to two example data sets.     

Tests for Homogeneity of Variances Using Robust Weighted Likelihood Estimates

The classical normal-theory tests for testing the null hypothesis of common variance and the classical estimates of scale have long been known to be quite nonrobust to even mild deviations from normality assumptions for moderate sample sizes. Levene (1960) suggested a one-way ANOVA type statistic as a robust test. Brown and Forsythe (1974) considered a modified version of Levene's test by replacing the sample means with sample medians as estimates of population locations, and their test is computationally the simplest among the three tests recommended by Conover , Johnson , and Johnson (1981) in terms of robustness and power. In this paper a new robust and powerful test for homogeneity of variances is proposed based on a modification of Levene's test using the weighted likelihood estimates (Markatou , Basu , and Lindsay , 1996) of the population means. For two and three populations the proposed test using the Hellinger distance based weighted likelihood estimates is observed to achieve better empirical level and power than Brown-Forsythe's test in symmetric distributions having a thicker tail than the normal, and higher empirical power in skew distributions under the use of F distribution critical values.     

An Adaptive Location‐Scale Test

Increasing locations are often accompanied by an increase in variability. In this case apparent heteroscedasticity can indicate that there are treatment effects and it is appropriate to consider an alternative involving differences in location as well as in scale. As a location‐scale test the sum of a location and a scale test statistic can be used. However, the power can be raised through weighting the sum. In order to select values for this weighting an adaptive design with an interim analysis is proposed: The data of the first stage are used to calculate the weights and with the second stage's data a weighted location‐scale test is carried out. The p‐values of the two stages are combined through Fisher's combination test. With a Lepage‐type location‐scale test it is illustrated that the resultant adaptive test can be more powerful than the ‘optimum’ test with no interim analysis. The principle to calculate weights, which cannot be reasonably chosen a priori, with the data of the first stage may be useful for other tests which utilize weighted statistics, too. Furthermore, the proposed test is illustrated with an example from experimental ecology.     

Normal Approximation of a Discrete Distribution on Exact Test of Uniform Association in a 2 × 3 Table

For the analysis of 2 × 3 tables, TOMIZAWA (1993) considered an exact test of uniform association, which is an extension of independence, and then derived a discrete distribution. This paper gives a normal approximation of the discrete distribution and describes that the normalized statistic can test a one-sided hypothesis on the uniform association. Also it points out that the square of the normalized test statistic is equal to the Pearson's chi-squared statistic for testing the uniform association.     

Optimal Design of a Series of GSTR's Performing Reversible Reactions Catalyzed by Soluble Enzymes: A Theoretical Study

The condition for the minimum overall reactor volume of a given number of CSTR's in series is theoretically determined for a reversible, single reactant-single product (Uni-Uni) enzyme catalyzed reaction. The reactor network is assumed to operate in steady-state, isothermal conditions with a single phase and a constant activity of biocatalyst. The method is based on a mathematical analysis of the discrete substrate concentration profile along the CSTR's assuming complete micromixing. The algebraic equations describing the critical loci are obtained for the general case, the mathematical proof that these equations define a minimum is presented, and an exact solution arising from an asymptotic situation is found. An approximate analytical method of optimization based on the aforementioned critical behavior is reported and its validity and usefulness discussed. The formulae introduced can be used in more general situations as tools for getting the approximate range where the optimal overall volume of the series of CSTR's lies. Hence, the reasoning developed is important for the preliminary CSTR design and relevant in the initial steps of the more involved methods of numerical optimization. Finally, the enzymatic conversion of fumarate to L-malate is examined as a model system in order to assess the usefulness and applicability of the analysis developed.     

Small Vaccination Experiments with Binary Outcome: The Paradox of Increasing Power with Decreasing Sample Size and/or Increasing Imbalance

In animal vaccination experiments with binary outcome (diseased/non diseased), the comparison of the vaccinated and control group is often based on the Fisher exact test. A tool for the evaluation of different designs is proposed, based on the expected power of the Fisher exact test. The expected power can sometimes unexpectedly increase with decreasing sample size and/or increasing imbalance. The reasons for these peculiar results are explained and compared to the results of two other types of tests: the unconditional test and the randomisation test. In a vaccination experiment with a restricted number of animals it is shown to be important to consider expected power in order to choose the most appropriate design.     

POWSIM: a computer program for assessing statistical power when testing for genetic differentiation

Knowledge of statistical power is essential for sampling design and data evaluation when testing for genetic differentiation. Yet, such information is typically missing in studies of conservation and evolutionary genetics, most likely because of complex interactions between the many factors that affect power. powsim is a 32‐bit Windows/DOS simulation‐based computer program that estimates power (and α error) for chi‐square and Fisher's exact tests when evaluating the hypothesis of genetic homogeneity. Optional combinations include the number of samples, sample sizes, number of loci and alleles, allele frequencies, and degree of differentiation (quantified as F_ST). powsim is available at http://www.zoologi.su.se/~ryman .     

A note on the tests for clustered matched-pair binary data

McNemar's test is used to assess the difference between two different procedures (treatments) using independent matched-pair data. For matched-pair data collected in clusters, the tests proposed by Durkalski et al. and Obuchowski are popular and commonly used in practice since these tests do not require distributional assumptions or assumptions on the structure of the within-cluster correlation of the data. Motivated by these tests, this note proposes a modified Obuchowski test and illustrates comparisons of the proposed test with the extant methods. An extensive Monte Carlo simulation study suggests that the proposed test performs well with respect to the nominal size, and has higher power; Obuchowski's test is most conservative, and the performance of the Durkalski's test varies between the modified Obuchowski test and the original Obuchowski's test. These results form the basis for our recommendation that (i) for equal cluster size, the modified Obuchowski test is always preferred; (ii) for varying cluster size Durkalski's test can be used for a small number of clusters (e.g. K < 50), whereas for a large number of clusters (e.g. K ≥ 50) the modified Obuchowski test is preferred. Finally, to illustrate practical application of the competing tests, two real collections of clustered matched-pair data are analyzed.     

Tests Against Qualitative Interaction: Exact Critical Values and Robust Tests

Consider a study to evaluate treatment A with a placebo in two or more groups of patients. If treatment A is beneficial to one group of patients and harmful to another, then we say that there is qualitative interaction or crossover interaction between patient groups and the treatments. Gail and Simon (1985, Biometrics 41, 361-372) developed a large-sample procedure for this testing problem. Their test has received favorable coverage in the literature. In this article, we obtain corresponding exact finite sample results for normal error distribution and provide a table of critical values. The test statistic is similar to the familiar F-ratio, and its p-value is equal to a weighted sum of tail areas of F-distributions. The computations to implement this are simple. A simulation study shows that the exact critical values provided here for normal error distribution are preferable to the asymptotic critical values for a wide range of error distributions. We also develop tests that are power robust against long-tailed error distributions. Our robust test uses M-estimators instead of the least squares estimators. We show that the efficiency robustness of the M-estimator translates to power robustness of the corresponding test. Therefore, our robust tests are better if outliers are expected. A simulation study illustrates the substantial power advantages of our robust tests.     

The Statistical Significance of Mutation Frequencies

TRAUT has proposed a method for determining the statistical significance between a mutation frequency observed after treatment with a potential mutagen and a control frequency. The method is developed further by us for an easier practical use. A table for the minimal sample size in the region of interest for the sex-linked recessive lethal test in Drosophila melanogaster has been calculated. TRACT'S test is compared with FISHER'S exact test and the KASTENBAUM -BOWMAN test. TRAUT'S test turns out to be more sensitive than the other tests. This observation strongly supports TRAUT'S conclusion that his test should only be used if very accurate determinations of the spontaneous frequencies are available.     

The Probability of a Negative Linear Combination of Independent Mean Squares

SATTERTHWAITE'S (1941) approximation of the distribution of a linear combination, of independent mean squares is a commonly used technique in the analysis of variance. Confidence intervals and test statistics based on this approximation require that be positive. In this article, the probability that will be negative is considered in situations in which the mean squares are associated with a general balanced mixed model. Expressions are given for exact and approximate values of this probability in terms of the expected values and degrees of freedom of the mean squares. An example is presented to illustrate the implementation of the proposed methodology.     

Exact Test of Uniform Association in a 2 × 3 Table

In a similar approach to Fisher's exact test of independence (null association) for a 2 × 2 table, this note gives an exact test of uniform association for a 2 × 3 table.