Rank Analysis of Block Designs Having Different Cell Frequencies

A distribution–free test is considered for testing the treatment effects in block designs with different cell frequencies. A test statistic which is a function of treatment ranks has been proposed which is distributed as chi-square for large samples. The null distribution of the test statistic has been obtained. The entire procedure has been explained by a numerical example.     

Selection of Judges for Sensory Evaluation

A test based on ordered observations for selection of consistent judges for sensory evaluation has been given. The test–statistic depends on the sum of ranks of different products under consideration. The probability distribution of the test-statistic has been worked out for small sample and it turns out to be chi-square distribution for large sample. The analytical procedure has been explained by a numerical example of a taste-testing experiment.     

Multi-Sample Slippage Test for Ordered Observations

A multi-sample slippage test based on ordered observations has been given. The test statistic is based on the sum of ranks of the sample. The probability distribution of the test statistic has been worked out for small sample and it turns out to be chi-square distribution for large sample. The analytical procedure has been explained by a numerical example.     

Modified nonparametric approaches to detecting differentially expressed genes in replicated microarray experiments

MOTIVATION: An important goal in analyzing microarray data is to determine which genes are differentially expressed across two kinds of tissue samples or samples obtained under two experimental conditions. Various parametric tests, such as the two-sample t-test, have been used, but their possibly too strong parametric assumptions or large sample justifications may not hold in practice. As alternatives, a class of three nonparametric statistical methods, including the empirical Bayes method of Efron et al. (2001), the significance analysis of microarray (SAM) method of Tusher et al. (2001) and the mixture model method (MMM) of Pan et al. (2001), have been proposed. All the three methods depend on constructing a test statistic and a so-called null statistic such that the null statistic's distribution can be used to approximate the null distribution of the test statistic. However, relatively little effort has been directed toward assessment of the performance or the underlying assumptions of the methods in constructing such test and null statistics. RESULTS: We point out a problem of a current method to construct the test and null statistics, which may lead to largely inflated Type I errors (i.e. false positives). We also propose two modifications that overcome the problem. In the context of MMM, the improved performance of the modified methods is demonstrated using simulated data. In addition, our numerical results also provide evidence to support the utility and effectiveness of MMM.     

Fractionation in Triad Comparisons

A statistical procedure for comparing the performance of a new product with the existing products on the basis of sensory characters has been developed. A test statistic was evolved for testing the null hypothesis of equality of treatment effects in the case of fractional triad comparisons. The null distribution of the test statistic has been obtained and it is found that it has a χ²-distribution for large number of observations. The procedure is quite simple and is based on a distribution-free test requiring only ordinal scale measurement. The method of analysis has been explained by a numerical example.     

The Nonparametric Behrens‐Fisher Problem: Asymptotic Theory and a Small‐Sample Approximation

A generalization of the Behrens‐Fisher problem for two samples is examined in a nonparametric model. It is not assumed that the underlying distribution functions are continuous so that data with arbitrary ties can be handled. A rank test is considered where the asymptotic variance is estimated consistently by using the ranks over all observations as well as the ranks within each sample. The consistency of the estimator is derived in the appendix. For small samples (n₁, n₂ ≥ 10), a simple approximation by a central t‐distribution is suggested where the degrees of freedom are taken from the Satterthwaite‐Smith‐Welch approximation in the parametric Behrens‐Fisher problem. It is demonstrated by means of a simulation study that the Wilcoxon‐Mann‐Whitney‐test may be conservative or liberal depending on the ratio of the sample sizes and the variances of the underlying distribution functions. For the suggested approximation, however, it turns out that the nominal level is maintained rather accurately. The suggested nonparametric procedure is applied to a data set from a clinical trial. Moreover, a confidence interval for the nonparametric treatment effect is given.     

A New Statistical Goodness-of-Fit Test Based on Graphical Representation

A statistical goodness-of-fit test, based on representing the sample observations by linked vectors, is developed. The direction and the length of the linked vectors are defined as functions of the expected values of the order statistics and sample order statistics, respectively. The underlying method can be used to test distributional assumptions for any location-scale family. A test statistic Q_n is introduced and some of its properties are studied. It is shown that the proposed test can be generalized to test if two or more independent samples come from the same distribution. The test procedure provides a graphical method of identifying the true distribution when the null hypothesis is rejected.     

On the use of permutation in and the performance of a class of nonparametric methods to detect differential gene expression

MOTIVATION: Recently a class of nonparametric statistical methods, including the empirical Bayes (EB) method, the significance analysis of microarray (SAM) method and the mixture model method (MMM), have been proposed to detect differential gene expression for replicated microarray experiments conducted under two conditions. All the methods depend on constructing a test statistic Z and a so-called null statistic z. The null statistic z is used to provide some reference distribution for Z such that statistical inference can be accomplished. A common way of constructing z is to apply Z to randomly permuted data. Here we point our that the distribution of z may not approximate the null distribution of Z well, leading to possibly too conservative inference. This observation may apply to other permutation-based nonparametric methods. We propose a new method of constructing a null statistic that aims to estimate the null distribution of a test statistic directly. RESULTS: Using simulated data and real data, we assess and compare the performance of the existing method and our new method when applied in EB, SAM and MMM. Some interesting findings on operating characteristics of EB, SAM and MMM are also reported. Finally, by combining the idea of SAM and MMM, we outline a simple nonparametric method based on the direct use of a test statistic and a null statistic.     

Estimating the null distribution to adjust observed confidence levels for genome-scale screening

In a novel approach to the multiple testing problem, Efron (2004, Journal of the American Statistical Association 99, 96-104; 2007a Journal of the American Statistical Association 102, 93-103; 2007b, Annals of Statistics 35, 1351-1377) formulated estimators of the distribution of test statistics or nominal p-values under a null distribution suitable for modeling the data of thousands of unaffected genes, nonassociated single-nucleotide polymorphisms, or other biological features. Estimators of the null distribution can improve not only the empirical Bayes procedure for which it was originally intended, but also many other multiple-comparison procedures. Such estimators in some cases improve the proposed multiple-comparison procedure (MCP) based on a recent non-Bayesian framework of minimizing expected loss with respect to a confidence posterior, a probability distribution of confidence levels. The flexibility of that MCP is illustrated with a nonadditive loss function designed for genomic screening rather than for validation. The merit of estimating the null distribution is examined from the vantage point of the confidence-posterior MCP (CPMCP). In a generic simulation study of genome-scale multiple testing, conditioning the observed confidence level on the estimated null distribution as an approximate ancillary statistic markedly improved conditional inference. Specifically simulating gene expression data, however, indicates that estimation of the null distribution tends to exacerbate the conservative bias that results from modeling heavy-tailed data distributions with the normal family. To enable researchers to determine whether to rely on a particular estimated null distribution for inference or decision making, an information-theoretic score is provided. As the sum of the degree of ancillarity and the degree of inferential relevance, the score reflects the balance conditioning would strike between the two conflicting terms. The CPMCP and other methods introduced are applied to gene expression microarray data.     

Beta diversity reconsidered

A simple analytical procedure to test for differences in beta diversity among sets of plots has been recently presented. Here, we describe an improved randomization procedure that replaces the one previously proposed. This procedure consists of shuffling within-group dissimilarities among groups and disregarding between-group dissimilarities. By repeating this operation many times, a distribution of the test statistics under the null hypothesis of no differences in the mean plot-to-plot dissimilarities within groups is obtained. This procedure ensures that the correct null model is selected. To describe this new procedure, we used plant and water beetle (Coleoptera) data collected from 45 permanent ponds. Beta diversity was compared between plant and water beetle (Coleoptera) assemblages.     

Computer analysis of two-dimensional gels.

This work identifies statistical algorithms which need to be included in analysis of two-dimensional gels for accurate determination of differential changes. Two-dimensional electrophoresis is a powerful tool for determining differential protein expression in complex mixtures, but the methodology, to date, is not producing expected results due to the degree of gel variability. The new DIGE procedure, comparing two samples in the same gel, does eliminate some of the variability introduced with gel-to-gel comparison, but still has variability due to differences in dye binding, charge, and fluorescence. Introducing quality-assurance statistical algorithms is necessary to extract meaningful data from the gels. A quality-control analysis of replicate gels needs to be performed prior to using the set in the final analysis. Increasing replicates to five from the usual three can only add greater variability. A statistical "replicate quality" gel test needs to be done on the computer gel scans, and replicates with greater than 20-30% variability should not be used. In addition, since spot intensity data are not normally distributed, spot differential analysis cannot be a t-test. The Studentized Range has been suggested as a more accurate method for calculating significant difference.     

Bacterial Density in Water Determined by Poisson or Negative Binomial Distributions

The question of how to characterize the bacterial density in a body of water when data are available as counts from a number of small-volume samples was examined for cases where either the Poisson or negative binomial probability distributions could be used to describe the bacteriological data. The suitability of the Poisson distribution when replicate analyses were performed under carefully controlled conditions and of the negative binomial distribution for samples collected from different locations and over time were illustrated by two examples. In cases where the negative binomial distribution was appropriate, a procedure was given for characterizing the variability by dividing the bacterial counts into homogeneous groups. The usefulness of this procedure was illustrated for the second example based on survey data for Lake Erie. A further illustration of the difference between results based on the Poisson and negative binomial distributions was given by calculating the probability of obtaining all samples sterile, assuming various bacterial densities and sample sizes.     

The Baumgartner-Weiss-Schindler test for the detection of differentially expressed genes in replicated microarray experiments

MOTIVATION: An important application of microarray experiments is to identify differentially expressed genes. Because microarray data are often not distributed according to a normal distribution nonparametric methods were suggested for their statistical analysis. Here, the Baumgartner-Weiss-Schindler test, a novel and powerful test based on ranks, is investigated and compared with the parametric t-test as well as with two other nonparametric tests (Wilcoxon rank sum test, Fisher-Pitman permutation test) recently recommended for the analysis of gene expression data. RESULTS: Simulation studies show that an exact permutation test based on the Baumgartner-Weiss-Schindler statistic B is preferable to the other three tests. It is less conservative than the Wilcoxon test and more powerful, in particular in case of asymmetric or heavily tailed distributions. When the underlying distribution is symmetric the differences in power between the tests are relatively small. Thus, the Baumgartner-Weiss-Schindler is recommended for the usual situation that the underlying distribution is a priori unknown. AVAILABILITY: SAS code available on request from the authors.     

Testing for differences in beta diversity from plot-to-plot dissimilarities

The central role of beta diversity in describing biodiversity patterns has been investigated in many fields of ecology and biogeography. While a variety of measures of beta diversity have been proposed over the past five decades, the question of how to test for differences in beta diversity among different sets of sampling plots has been addressed only rarely. Here, we describe a simple analytical procedure to test for differences in beta diversity among distinct sets of plots. The advantage of this approach compared to methods that have been previously proposed lies in its randomization procedure. Such a procedure creates a distribution of null values of the test statistic that is compatible with the null hypothesis of no significant difference in multivariate dispersion between the groups. The proposed test was illustrated using a large dataset of plant and water beetle (Coleoptera) assemblages collected from 45 farmland ponds in Ireland.     

An Exact Paired Rank Test

An exact rank test for two dependent samples based on overall mid‐ranks is discussed which can be applied to metric as well as to ordinal data. The exact conditional distribution of the test statistic given the observed vector of rank differences is determined. A recursion formula is given as well as a fast shift algorithm in SAS/IML code. Moreover, it is demonstrated that the paired rank test can be more powerful than other tests for paired samples by means of a simulation study. Finally, the test is applied to a psychiatric trial with longitudinal ordinal data.     

Construction of null statistics in permutation-based multiple testing for multi-factorial microarray experiments

MOTIVATION: The parametric F-test has been widely used in the analysis of factorial microarray experiments to assess treatment effects. However, the normality assumption is often untenable for microarray experiments with small replications. Therefore, permutation-based methods are called for help to assess the statistical significance. The distribution of the F-statistics across all the genes on the array can be regarded as a mixture distribution with a proportion of statistics generated from the null distribution of no differential gene expression whereas the other proportion of statistics generated from the alternative distribution of genes differentially expressed. This results in the fact that the permutation distribution of the F-statistics may not approximate well to the true null distribution of the F-statistics. Therefore, the construction of a proper null statistic to better approximate the null distribution of F-statistic is of great importance to the permutation-based multiple testing in microarray data analysis. RESULTS: In this paper, we extend the ideas of constructing null statistics based on pairwise differences to neglect the treatment effects from the two-sample comparison problem to the multifactorial balanced or unbalanced microarray experiments. A null statistic based on a subpartition method is proposed and its distribution is employed to approximate the null distribution of the F-statistic. The proposed null statistic is able to accommodate unbalance in the design and is also corrected for the undue correlation between its numerator and denominator. In the simulation studies and real biological data analysis, the number of true positives and the false discovery rate (FDR) of the proposed null statistic are compared with those of the permutated version of the F-statistic. It has been shown that our proposed method has a better control of the FDRs and a higher power than the standard permutation method to detect differentially expressed genes because of the better approximated tail probabilities.     

Estimating the proportion of true null hypotheses and adaptive false discovery rate control in discrete paradigm

Storey's estimator for the proportion of true null hypotheses, originally proposed under the continuous framework, has been modified in this work under the discrete framework. The modification results in improved estimation of the parameter of interest. The proposed estimator is used to formulate an adaptive version of the Benjamini–Hochberg procedure. Control over the false discovery rate by the proposed adaptive procedure has been proved analytically. The proposed estimate is also used to formulate an adaptive version of the Benjamini–Hochberg–Heyse procedure. Simulation experiments establish the conservative nature of this new adaptive procedure. Substantial amount of gain in power is observed for the new adaptive procedures over the standard procedures. For demonstration of the proposed method, two important real life gene expression data sets, one related to the study of HIV and the other related to methylation study, are used.     

Analyses of two-way chronic studies

This paper proposes a method for analyzing tumor data from chronic studies when the experimental design includes combinations of two factors, for example, sex and dose. Both main effects and combined-effect (interaction) hypotheses are considered. A stratified log-rank statistic is presented for tests of no column or row (main) effects. The paper shows that when the numbers of animals in the cells are unequal and disproportional, the null distribution of the unstratified log-rank statistic does not have a chi-square distribution. Two simple models, additive and multiplicative, for representing the combined effect of row and column are considered under the proportional hazards model. A simple conservative statistic is proposed for testing the additivity of the row and column effects. A simulation experiment to examine the behavior of the null distribution of the combined-effect test statistic under the additive model and the power of the test against the multiplicative model is reported. The procedure is illustrated by analyzing mammary tumors induced by 7,12-dimethylbenz[a]anthracene (DMBA) in yellow and agouti F1 female mice from a laboratory experiment.     

Improving data interpretation of fragmentary data-sets on invertebrate dispersal with permutation tests

Biological data often tend to have heterogeneous, discontinuous non-normal distributions. Statistical non-parametric tests, like the Mann–Whitney U-test or the extension for more than two samples, the Kruskal–Wallis test, are often used in these cases, although they assume certain preconditions which are often ignored. We developed a permutation test procedure that uses the ratio of the interquartile distances and the median differences of the original non-classified data to assess the properties of the real distribution more appropriately than the classical methods. We used this test on a heterogeneous, skewed biological data set on invertebrate dispersal and showed how different the reactions of the Kruskal–Wallis test and the permutation approach are. We then evaluated the new testing procedure with reproducible data that were generated from the normal distribution. Here, we tested the influence of four different experimental trials on the new testing procedure in comparison to the Kruskal–Wallis test. These trials showed the impact of data that were varying in terms of (a) negative correlation between variances and means of the samples, (b) changing variances that were not correlated with the means of the samples, (c) constant variances and means, but different sample sizes and in trials (d) we evaluated the testing power of the new procedure. Due to the different test statistics, the permutation test reacted more sensibly to the data presented in trials (a) and c) and non-uniformly in trial (b). In the evaluation of the testing power, no significant differences between the Kruskal–Wallis test and the new permutation testing procedure could be detected. We consider this test to be an alternative for working on heterogeneous data where the preconditions of the classical non-parametric tests are not met.     

On using Lehmann alternatives with nonresponders.

The problem of testing for treatment effect when some subjects in the treatment group may be unaffected by the treatment is considered. A form of the Lehmann alternative suggested by Conover and Salsburg is used that assumes that each control score has the same distribution as the minimum of the known number of responses in the treatment group. It is shown that the locally most powerful test leads to a test statistic that, under the hypothesis of no treatment effect, is the sum of independent pareto random variables whereas under the alternative hypothesis it is the sum of independent random variables from a mixture of two pareto distributions. The limiting distribution of the test statistic under both hypotheses is in the domain of attraction of a stable distribution whose indices are derived. The power of the test is given, and its properties are discussed. A set of data from clinical research involving development of a new drug is used to show application of the procedure and demonstrate its usefulness.