Statistical power when testing for genetic differentiation

A variety of statistical procedures are commonly employed when testing for genetic differentiation. In a typical situation two or more samples of individuals have been genotyped at several gene loci by molecular or biochemical means, and in a first step a statistical test for allele frequency homogeneity is performed at each locus separately, using, e.g. the contingency chi-square test, Fisher's exact test, or some modification thereof. In a second step the results from the separate tests are combined for evaluation of the joint null hypothesis that there is no allele frequency difference at any locus, corresponding to the important case where the samples would be regarded as drawn from the same statistical and, hence, biological population. Presently, there are two conceptually different strategies in use for testing the joint null hypothesis of no difference at any locus. One approach is based on the summation of chi-square statistics over loci. Another method is employed by investigators applying the Bonferroni technique (adjusting the P-value required for rejection to account for the elevated alpha errors when performing multiple tests simultaneously) to test if the heterogeneity observed at any particular locus can be regarded significant when considered separately. Under this approach the joint null hypothesis is rejected if one or more of the component single locus tests is considered significant under the Bonferroni criterion. We used computer simulations to evaluate the statistical power and realized alpha errors of these strategies when evaluating the joint hypothesis after scoring multiple loci. We find that the 'extended' Bonferroni approach generally is associated with low statistical power and should not be applied in the current setting. Further, and contrary to what might be expected, we find that 'exact' tests typically behave poorly when combined in existing procedures for joint hypothesis testing. Thus, while exact tests are generally to be preferred over approximate ones when testing each particular locus, approximate tests such as the traditional chi-square seem preferable when addressing the joint hypothesis.     

Minimally selected p and other tests for a single abrupt changepoint in a binary sequence

A novel changepoint statistic based on the minimum value, over possible changepoint locations, of Fisher's Exact Test, is introduced. Specific points in the exact distribution of the minimally selected Fisher's value may be rapidly calculated as a lattice-path counting problem via known recurrence methods. The test is compared to the Kolmogorov-Smirnov two-sample test, the maximally selected chi-square, and a likelihood ratio test. The tests are applied to assessing recombination in genetic sequences of HIV.     

Beyond Bonferroni: less conservative analyses for conservation genetics

Studies in conservation genetics often attempt to determine genetic differentiation between two or more temporally or geographically distinct sample collections. Pairwise p-values from Fisher’s exact tests or contingency Chi-square tests are commonly reported with a Bonferroni correction for multiple tests. While the Bonferroni correction controls the experiment-wise α, this correction is very conservative and results in greatly diminished power to detect differentiation among pairs of sample collections. An alternative is to control the false discovery rate (FDR) that provides increased power, but this method only maintains experiment-wise α when none of the pairwise comparisons are significant. Recent modifications to the FDR method provide a moderate approach to determining significance level. Simulations reveal that critical values of multiple comparison tests with both the Bonferroni method and a modified FDR method approach a minimum asymptote very near zero as the number of tests gets large, but the Bonferroni method approaches zero much more rapidly than the modified FDR method. I compared pairwise significance from three published studies using three critical values corresponding to Bonferroni, FDR, and modified FDR methods. Results suggest that the modified FDR method may provide the most biologically important critical value for evaluating significance of population differentiation in conservation genetics.␣Ultimately, more thorough reporting of statistical significance is needed to allow interpretation of biological significance of genetic differentiation among populations.An erratum to this article can be found at     

Genetic diversity of 15 STR loci in a population of Montenegro

Genetic diversity and forensic parameters based on 15AmpFlSTR Identifiler short tandem repeat (STR) loci (D8S1179, D21S11, D7S820, CSF1PO, D3S1358, TH01, D13S317, D16S539, D2S1338, D19S433, vWA, TPOX, D18S51, D5S818 and FGA) were evaluated in a sample of 101 unrelated, autochthonous adults from Montenegro. After applying Bonferroni correction, the agreement with Hardy-Weinberg equilibrium (HWE) was confirmed for all loci with the exception of D5S818 (chi2 test) and D21S11 (exact test). The combined power of discrimination (PD) and the combined power of exclusion (PE) for the 15 studied loci were 0.9999999999999999844 and 0.99999382, respectively. According to measures of within-population genetic diversity, D2S1338, D18S51 and FGA may be considered as the most variable and most informative markers for forensic testing and population genetic analyses out of the 15 analysed loci in a population of Montenegro. D5S818 showed to be the least variable and together with TPOX, the least informative. Interpopulation comparisons were carried out and levels of genetic differentiation between population of Montenegro and five South-eastern European populations (Kosovo Albanians, Serbians from Vojvodina province, Macedonians, Bosnians and Croatians) were evaluated. The most differentiated population in relation to Montenegro is a population of Kosovo Albanians as suggested by both AMOVA and coefficients of genetic differentiation (F(ST) and R(ST)).     

POWSIM: a computer program for assessing statistical power when testing for genetic differentiation

Knowledge of statistical power is essential for sampling design and data evaluation when testing for genetic differentiation. Yet, such information is typically missing in studies of conservation and evolutionary genetics, most likely because of complex interactions between the many factors that affect power. powsim is a 32‐bit Windows/DOS simulation‐based computer program that estimates power (and α error) for chi‐square and Fisher's exact tests when evaluating the hypothesis of genetic homogeneity. Optional combinations include the number of samples, sample sizes, number of loci and alleles, allele frequencies, and degree of differentiation (quantified as F_ST). powsim is available at http://www.zoologi.su.se/~ryman .     

The Inheritance of Metabolic Flux: Expressions for the within-Sibship Mean and Variance Given the Parental Genotypes

Recent developments have related quantitative trait expression to metabolic flux. The present paper investigates some implications of this for statistical aspects of polygenic inheritance. Expressions are derived for the within-sibship genetic mean and genetic variance of metabolic flux given a pair of parental, diploid, n-locus genotypes. These are exact and hold for arbitrary numbers of gene loci, arbitrary allelic values at each locus, and for arbitrary recombination fractions between adjacent gene loci. The within-sibship, genetic variance is seen to be simply a measure of parental heterozygosity plus a measure of the degree of linkage coupling within the parental genotypes. Approximations are given for the within-sibship phenotypic mean and variance of metabolic flux. These results are applied to the problem of attaining adequate statistical power in a test of association between allozymic variation and inter-individual variation in metabolic flux. Simulations indicate that statistical power can be greatly increased by augmenting the data with predictions and observations on progeny statistics in relation to parental allozyme genotypes. Adequate power may thus be attainable at small sample sizes, and when allozymic variation is scored at a only small fraction of the total set of loci whose catalytic products determine the flux.     

Exact sample sizes needed to detect dependence in 2 x 3 tables

Many medical and biological studies entail classifying a number of observations according to two factors, where one has two and the other three possible categories. This is the case of, for example, genetic association studies of complex traits with single-nucleotide polymorphisms (SNPs), where the a priori statistical planning, analysis, and interpretation of results are of critical importance. Here, we present methodology to determine the minimum sample size required to detect dependence in 2 x 3 tables based on Fisher's exact test, assuming that neither of the two margins is fixed and only the grand total N is known in advance. We provide the numerical tools necessary to determine these sample sizes for desired power, significance level, and effect size, where only the computational time can be a limitation for extreme parameter values. These programs can be accessed at . This solution of the sample size problem for an exact test will permit experimentalists to plan efficient sampling designs, determine the extent of statistical support for their hypotheses, and gain insight into the repeatability of their results. We apply this solution to the sample size problem to three empirical studies, and discuss the results with specified power and nominal significance levels.     

Statistical test for the comparison of samples from mutational spectra

The Monte Carlo estimate of the p value of the hypergeometric test is described and advocated for the testing of the hypothesis that different treatments induce the same mutational spectrum. The hypergeometric test is a generalization of Fisher's "exact" test for tables with more than two rows and two columns. Use of the test is demonstrated by the analysis of data from the characterization of nonsense mutations in the lacI gene of Escherichia coli. Unlike the chi-square test, the hypergeometric test remains valid when applied to sparse cross-classification tables. The hypergeometric test has the most discrimination power of any statistical test that could be employed routinely to compare samples from mutational spectra. Direct application of the hypergeometric test to large cross-classification tables is excessively computation intensive, but estimation of its p value via Monte Carlo techniques is practical.     

Detecting differential and correlated protein expression in label-free shotgun proteomics

Recent studies have revealed a relationship between protein abundance and sampling statistics, such as sequence coverage, peptide count, and spectral count, in label-free liquid chromatography-tandem mass spectrometry (LC-MS/MS) shotgun proteomics. The use of sampling statistics offers a promising method of measuring relative protein abundance and detecting differentially expressed or coexpressed proteins. We performed a systematic analysis of various approaches to quantifying differential protein expression in eukaryotic Saccharomyces cerevisiae and prokaryotic Rhodopseudomonas palustris label-free LC-MS/MS data. First, we showed that, among three sampling statistics, the spectral count has the highest technical reproducibility, followed by the less-reproducible peptide count and relatively nonreproducible sequence coverage. Second, we used spectral count statistics to measure differential protein expression in pairwise experiments using five statistical tests: Fisher's exact test, G-test, AC test, t-test, and LPE test. Given the S. cerevisiae data set with spiked proteins as a benchmark and the false positive rate as a metric, our evaluation suggested that the Fisher's exact test, G-test, and AC test can be used when the number of replications is limited (one or two), whereas the t-test is useful with three or more replicates available. Third, we generalized the G-test to increase the sensitivity of detecting differential protein expression under multiple experimental conditions. Out of 1622 identified R. palustris proteins in the LC-MS/MS experiment, the generalized G-test detected 1119 differentially expressed proteins under six growth conditions. Finally, we studied correlated expression of these 1119 proteins by analyzing pairwise expression correlations and by delineating protein clusters according to expression patterns. Through pairwise expression correlation analysis, we demonstrated that proteins co-located in the same operon were much more strongly coexpressed than those from different operons. Combining cluster analysis with existing protein functional annotations, we identified six protein clusters with known biological significance. In summary, the proposed generalized G-test using spectral count sampling statistics is a viable methodology for robust quantification of relative protein abundance and for sensitive detection of biologically significant differential protein expression under multiple experimental conditions in label-free shotgun proteomics.     

A cautionary note on exact unconditional inference for a difference between two independent binomial proportions

Fisher's exact test for comparing response proportions in a randomized experiment can be overly conservative when the group sizes are small or when the response proportions are close to zero or one. This is primarily because the null distribution of the test statistic becomes too discrete, a partial consequence of the inference being conditional on the total number of responders. Accordingly, exact unconditional procedures have gained in popularity, on the premise that power will increase because the null distribution of the test statistic will presumably be less discrete. However, we caution researchers that a poor choice of test statistic for exact unconditional inference can actually result in a substantially less powerful analysis than Fisher's conditional test. To illustrate, we study a real example and provide exact test size and power results for several competing tests, for both balanced and unbalanced designs. Our results reveal that Fisher's test generally outperforms exact unconditional tests based on using as the test statistic either the observed difference in proportions, or the observed difference divided by its estimated standard error under the alternative hypothesis, the latter for unbalanced designs only. On the other hand, the exact unconditional test based on the observed difference divided by its estimated standard error under the null hypothesis (score statistic) outperforms Fisher's test, and is recommended. Boschloo's test, in which the p-value from Fisher's test is used as the test statistic in an exact unconditional test, is uniformly more powerful than Fisher's test, and is also recommended.