Simultaneous confidence regions corresponding to Holm's step-down procedure and other closed-testing procedures

Holm's (1979) step-down multiple-testing procedure (MTP) is appealing for its flexibility, transparency, and general validity, but the derivation of corresponding simultaneous confidence regions has remained an unsolved problem. This article provides such confidence regions. In fact, simultanenous confidence regions are provided for any MTP in the class of short-cut consonant closed-testing procedures based on marginal p -values and weighted Bonferroni tests for intersection hypotheses considered by Hommel, Bretz and Maurer (2007). In addition to Holm's MTP, this class includes the fixed-sequence MTP, recently proposed gatekeeping MTPs, and the fallback MTP. The simultaneous confidence regions are generally valid if underlying marginal p -values and corresponding marginal confidence regions (assumed to be available) are valid. The marginal confidence regions and estimated quantities are not assumed to be of any particular kinds/dimensions. Compared to the rejections made by the MTP for the family of null hypotheses H under consideration, the proposed confidence regions provide extra free information. In particular, with Holm's MTP, such extra information is provided: for all nonrejected H s, in case not all H s are rejected; or for certain (possibly all) H s, in case all H s are rejected. In case not all H s are rejected, no extra information is provided for rejected H s. This drawback seems however difficult to overcome. Illustrations concerning clinical studies are given.     

Simultaneous Randomization Tests

Randomization analyses have been developed for testing main effects and interactions in standard experimental designs. However, exact multiple comparisons procedures for these randomization analyses have received little attention. This article proposes a general procedure for constructing simultaneous randomization tests that have prescribed type I error rates. An application of the procedure does provide for multiple comparisons in the randomization analyses of designed experiments. This application is made to data collected in a biopharmaceutical experiment.     

Age-adjusted exact trend tests in the event of rare occurrences

Preclinical animal carcinogenicity studies are usually concerned with testing the statistical significance of a dose-response relationship. When the response consists of a rare event such as the development of a certain type of tumor, exact statistical methods are often employed. The exact randomization trend test based on the multivariate hypergeometric distribution is less powerful in the presence of treatment-related risks other than the specified response. Particularly, the loss of power becomes more pronounced when competing risks cause progressively higher mortality rates with increasing dose, which is usual in practice. An age-adjusted form of the randomization test is proposed to adjust for this effect. Permutational distribution for Peto's cause-of-death (COD) test is also explored and compared with its asymptotic counterpart by simulation. The use of COD information has been a controversial issue due to the subjectivity in the pathologists' determinations as well as for economic reasons. The proposed age-adjusted exact test does not require COD, and it is shown to compare favorably to the COD tests via an extensive Monte Carlo simulation. Applications of the methods to two real data sets are included.     

Simultaneous confidence intervals from randomization tests with application in testing bioequivalence with multiple endpoints

We propose a method to construct simultaneous confidence intervals for a parameter vector from inverting a series of randomization tests (RT). The randomization tests are facilitated by an efficient multivariate Robbins–Monro procedure that takes the correlation information of all components into account. The estimation method does not require any distributional assumption of the population other than the existence of the second moments. The resulting simultaneous confidence intervals are not necessarily symmetric about the point estimate of the parameter vector but possess the property of equal tails in all dimensions. In particular, we present the constructing the mean vector of one population and the difference between two mean vectors of two populations. Extensive simulation is conducted to show numerical comparison with four methods. We illustrate the application of the proposed method to test bioequivalence with multiple endpoints on some real data.     

On weighted Hochberg procedures

We consider different ways of constructing weighted Hochberg-typestep-up multiple test procedures including closed proceduresbased on weighted Simes tests and their conservative step-upshort-cuts, and step-up counterparts of two weighted Holm procedures.It is shown that the step-up counterparts have some seriouspitfalls such as lack of familywise error rate control and lackof monotonicity in rejection decisions in terms of p-values.Therefore an exact closed procedure appears to be the best alternative,its only drawback being lack of simple stepwise structure. Aconservative step-up short-cut to the closed procedure may beused instead, but with accompanying loss of power. Simulationsare used to study the familywise error rate and power propertiesof the competing procedures for independent and correlated p-values.Although many of the results of this paper are negative, theyare useful in highlighting the need for caution when procedureswith similar pitfalls may be used.     

生物多样性和均匀度显著性的随机化检验及计算软件

多样性指数和均匀度以其简单易用而被广泛应用于群落生物学和生物多样性等研究中,然而由于缺乏合适的统计检验方法等原因,其分析的可信性往往较低,因而限制了其应用。鉴于生物多样性研究中广泛应用主观和直接的比较不,有必要建立和使用较为严格的多样性统计检验。本研究建立和应用了如下随机化检验方法：单群落多样性指数和均匀度的显著性检验,单群落多样性指数和均匀度的置认区间,群落间多群样和均匀度的差异显著性检验。随机化方法已被成功地应用于群落生态学研究,其原理是：随机排序某一向量中的元素,或随机交换两向量中的对应元素。计算该随机化数据的多样性和均匀度,重复该过程多次,统计和计算显著性检验的p值,由向量中的对应元素。计算该随机化数据的多样性和均匀度,重复该过程多次,统计和显著性检验的p值。由此可确定多样性和差异的统计显著性。同时,研制了相应的Internet计算软件BiodiverisytTest。该软件由7个Java类和1个HTML文件组成,可运行于多种操作系统和网络浏览器上,可读取多种类型的ODBC数据库文件如Access,Excel,FoxPro,Dbase等。该软件中包括Shannon-Wiener多样性指数,Simpson多样性指数,McIntosh多样性指数,Berger-Parker多样性指数,Hrlbert多样性指数以及Brillouin多样性指数。基于Shannon-Wiener多样性指数和Berger-Parker多样性指数,用BiodiversityTest软件对水稻田节肢动物群落多样性（15个地点,17个功能群,125个节肢动物种）进行了比较和分析。结果显示,两组结果可较好地反映水稻节肢动物群落多样性的差异显著性,这些检验方法可有效地反映多样性指数和均匀度的变化。与水稻田节肢动物群落间多样性的直接比较法相比,该随机化检验方法获得更客观的结果。本算法与软件有助于改进生物多样性研究中使用的某些不甚严格的分析方法,为随机化检验方法在生物多样性研究中的进一步应用提供了一种可用的工具。     

Testing the ratio of two poisson rates

In this paper we compare the properties of four different general approaches for testing the ratio of two Poisson rates. Asymptotically normal tests, tests based on approximate p -values, exact conditional tests, and a likelihood ratio test are considered. The properties and power performance of these tests are studied by a Monte Carlo simulation experiment. Sample size calculation formulae are given for each of the test procedures and their validities are studied. Some recommendations favoring the likelihood ratio and certain asymptotic tests are based on these simulation results. Finally, all of the test procedures are illustrated with two real life medical examples.     

Significance Levels of Randomization Trend Tests in the Event of Rare Occurrences

In the statistical evaluation of data from a dose-response experiment, it is frequently of interest to test for dose-related trend: an increasing trend in response with increasing dose. The randomization trend test, a generalization of Fisher's exact test, has been recommended for animal tumorigenicity testing when the numbers of tumor occurrences are small. This paper examines the type I error of the randomization trend test, and the Cochran-Armitage and Mantel-Haenszel tests. Simulation results show that when the tumor incidence rates are less than 10%, the randomization test is conservative; the test becomes very conservative when the incidence rate is less than 5%. The Cochran-Armitage and Mantel-Haenszel tests are slightly anti-conservative (liberal) when the incidence rates are larger than 3%. Further, we propose a less conservatived method of calculating the p-value of the randomization trend test by excluding some permutations whose probabilities of occurrence are greater than the probability of the the observed outcome.     

A Two-Tailed P-Value for Fisher's Exact Test

A two-tailed P-value is presented for a significance test in two by two contingency tables. There is no extraneous quasi-observation such as is needed in the exact randomized uniformly most powerful unbiased (UMPU) test of the hypothesis of independence. The proposed P-value can never exceed unity and is always two-tailed, unlike other P-values proposed in the literature     

A note on P-values under group sequential testing and nonproportional hazards

Summary .   Group sequential designs are often used for periodically assessing treatment efficacy during the course of a clinical trial. Following a group sequential test, P -values computed under the assumption that the data were gathered according to a fixed sample design are no longer uniformly distributed under the null hypothesis of no treatment effect. Various sample space orderings have been proposed for computing proper P -values following a group sequential test. Although many of the proposed orderings have been compared in the setting of time-invariant treatment effects, little attention has been given to their performance when the effect of treatment within an individual varies over time. Our interest here is to compare two of the most commonly used methods for computing proper P -values following a group sequential test, based upon the analysis time (AT) and Z -statistic orderings, with respect to resulting power functions when treatment effects on survival are delayed. Power under the AT ordering is shown to be heavily influenced by the presence of a delayed treatment effect, while power functions corresponding to the Z -statistic ordering remain robust under time-varying treatment effects.     

Testing for latitudinal and other body-size gradients

Testing for a relationship between the body size of animals and a gradient such as latitude is complicated by the fact that typically there is a single average size for each species, and each species occurs at several sample stations over the gradient. This results in standard tests for statistical significance being invalid. This problem can be overcome by using a randomization test. A more difficult problem, however, is determining whether the relationship between size and latitude is the same for two subfamilies of species. In this paper a general method for relating body size to latitude and subfamily differences is proposed, with the significance of effects determined by randomization. A simulation study suggests that this procedure has good properties. This approach to data analysis has promise both for the particular situation considered and for other related problems in biogeography.     

Estimation of the Proportion of Differentially Expressed Genes Using Hellinger Distance

We consider the problem of estimating the proportion of differentially expressed genes from the distribution of P-values arising from statistical tests in a microarray experiment. A two-component mixture model is studied in which one component is uniform on [0,1] and corresponds to equally expressed genes and the other component corresponds to differentially expressed genes. Two different parametric families are explicitly investigated for modeling the nonuniform component—the Beta family and a mixture of uniform densities whose supports form a partition of [0,1]. The Minimum Hellinger discrepancy is used to estimate the mixture proportions which are then used to estimate the proportion of differentially expressed genes in the microarray study. The performance of the proposed methods are investigated using simulation studies in which they are also compared with selected other published procedures. The methods are illustrated through a case study involving data from a published microarray experiment.     

Avoiding erroneously high levels of detection in combinations of semi-independent tests

A randomization procedure is proposed which allows statistical tests to be combined into a single test to maintain specified and acceptable levels of false detection. This method was applied to the problem of detecting density dependence in 135 unpublished time-series (of 10 generations) from insect populations, and to simulated density-dependent and density-independent data, so that the correctness of observed levels of detection from the published data could be verified. To allow the application of the randomization procedure to Bulmer's (1975) tests and Varley and Gradwell's (1960) test, these were recast as randomization tests. The randomization procedure was tested with 39 combinations of tests for density dependence (and limitation/attraction); it generally producedcombined tests with levels of detection that were intermediate between detection levels of the constituent tests (and hence was limite by these). The specified rate of false detection (5%) was never exceeded (by more than 1%) when combined tests were applied to time-series from a random-walk model. Two different combinations of tests produced levels of detection from the published time-series which were slightly greater than their constituent tests when they were combined into single tests. These were the randomized form of Bulmer's (1975) first test with the tests of Pollard et al. (1987) and Reddingius and den Boer (1989) with the randomized form of Bulmer's second test. The combination of Bulmer's first and Pollard et al.'s test produced a greater level of detection (21.5%) than any other single test or combination of tests. These results were confirmed by the analysis of modelled density dependent data. Although the increase in power of combinations of tests over single tests is small with the data we used, the combined tests (listed above) had rates of detection that were less influenced by the form of data (of two forms of density-dependent data) than were their constituent tests. Hence, it appears that the combined tests are of greater generality than single test statistics. The method presented here for combining several statistical tests into a single randomization test is applicable in many other areas of ecology where we wish to apply several tests and take the most probable result of these; and if the tests being conducted are, or can be expressed as, randomization tests.     

Immunochemotherapy of breast cancer with Propionibacterium granulosum

Summary Eighty-two patients with metastatic breast cancer were treated with adriamycin, cyclophosphamide, and 5-fluorouracil (FAC). These patients were divided by randomization into two groups. One was receiving cytostatic drugs only, while another received the identical cytostatic medication plus additional therapy consisting of IV infusions of gamma-radiation-sterilized cells of Propionibacterium granulosum strain KP-45. Metastases were localized in six different sites and/or organs. Multiple parameters for monitoring changes in metastatic lesions and laboratory tests were performed serially in all patients. The addition of P. granulosum to standard (FAC) chemotherapy of metastatic breast cancer influenced metastates in the liver. It also resulted in increased survival time within the observation period. Possible mechanisms of immunomodulating activity of P. granulosum in this disease are discussed.     

A class of permutation tests for stratified survival data

We propose a class of permutation tests for stratified survival data. The tests are derived using the framework of Fay and Shih (1998, Journal of the American Statistical Association 93, 387-396), which creates tests by permuting scores based on a functional of estimated distribution functions. Here the estimated distribution function for each possibly right-, left-, or interval-censored observation is based on a shrinkage estimator similar to the nonparametric empirical estimator of Ghosh, Lahiri, and Tiwari (1989, Communications in Statistics--Theory and Methods 18, 121-146), and permutation is carried out within strata. The proposed test with a weighted Mann-Whitney functional is similar to the permutation form of the stratified log-rank test when there is a large strata effect or the sample size in each stratum is large and is similar to the permutation form of the ordinary log-rank test when there is little strata effect. Thus, the proposed test unifies the advantages of both the stratified and ordinary log-rank tests. By changing the functional, we may obtain a stratified Prentice-Wilcoxon test or a difference in means test with similar unifying properties. We show through simulations the advantage of the proposed test over existing tests for uncensored and right-censored data.     

Applying randomization tests to cluster analyses

Abstract. In applying randomization tests to hierarchical cluster analyses, we have noted a potentially misleading result: within a significant group, linkages are often identified as significant even when species are randomly distributed among the group's sites. We demonstrate this through a cluster analysis of a constructed matrix with two groups of 20 sites that share no species, and within each group species are randomly distributed among sites. A randomization test identified both of the groups and all linkages within them as significant, while the same test found all linkages non‐significant in the cluster analysis of a matrix containing just one of the two groups of 20 sites. In general, a non‐random distribution of species within a data set shortens linkages relative to distances in null distributions derived from randomized versions of the data. This confounds efforts to identify significant sub‐groups within a significant group. However, the significance of sub‐groups possibly could be tested by comparing linkage distances to a null distribution derived from the randomization and clustering of a sub‐matrix containing only the sites within the larger group. In essence, this comparison tests the null hypothesis that within the significant group, sites represent random assemblages of species. When applied to actual data sets, an approach involving sequential randomization tests could allow the evaluation of all nodes in a classification, increasing the utility of randomization tests and strengthening the interpretation of groups produced by cluster analysis.     

Rare Variant Association Testing by Adaptive Combination of P-values

With the development of next-generation sequencing technology, there is a great demand for powerful statistical methods to detect rare variants (minor allele frequencies (MAFs)<1%) associated with diseases. Testing for each variant site individually is known to be underpowered, and therefore many methods have been proposed to test for the association of a group of variants with phenotypes, by pooling signals of the variants in a chromosomal region. However, this pooling strategy inevitably leads to the inclusion of a large proportion of neutral variants, which may compromise the power of association tests. To address this issue, we extend the -MidP method (Cheung et al., 2012, Genet Epidemiol 36: 675–685) and propose an approach (named ‘adaptive combination of P-values for rare variant association testing’, abbreviated as ‘ADA’) that adaptively combines per-site P-values with the weights based on MAFs. Before combining P-values, we first imposed a truncation threshold upon the per-site P-values, to guard against the noise caused by the inclusion of neutral variants. This ADA method is shown to outperform popular burden tests and non-burden tests under many scenarios. ADA is recommended for next-generation sequencing data analysis where many neutral variants may be included in a functional region.     

Evaluation of Ecotoxicological Field Studies for Authorization of Plant Protection Products in Europe

An increasing number of ecotoxicological field studies are being submitted in the European Union procedure for authorization of pesticides. Although there is some guidance on how these studies can be used for risk assessment, not all aspects of field tests are covered and the guidance differs per type of test and per non-target group. To facilitate a more uniform approach by the regulatory authorities in the EU, a basic scheme is proposed with qualitative tools to: (i) assess the scientific reliability of individual field tests, and (ii) to assess the usefulness of field tests for regulatory risk assessment of the pesticide under registration. In this way, the treatment, evaluation, and the mutual comparability of field data for regulatory purposes is harmonized. It thereby provides a more consistent foundation for further risk assessment.     

Testing random effects in linear mixed‐effects models with serially correlated errors

In linear mixed‐effects models, random effects are used to capture the heterogeneity and variability between individuals due to unmeasured covariates or unknown biological differences. Testing for the need of random effects is a nonstandard problem because it requires testing on the boundary of parameter space where the asymptotic chi‐squared distribution of the classical tests such as likelihood ratio and score tests is incorrect. In the literature several tests have been proposed to overcome this difficulty, however all of these tests rely on the restrictive assumption of i.i.d. measurement errors. The presence of correlated errors, which often happens in practice, makes testing random effects much more difficult. In this paper, we propose a permutation test for random effects in the presence of serially correlated errors. The proposed test not only avoids issues with the boundary of parameter space, but also can be used for testing multiple random effects and any subset of them. Our permutation procedure includes the permutation procedure in Drikvandi, Verbeke, Khodadadi, and Partovi Nia (2013) as a special case when errors are i.i.d., though the test statistics are different. We use simulations and a real data analysis to evaluate the performance of the proposed permutation test. We have found that random slopes for linear and quadratic time effects may not be significant when measurement errors are serially correlated.     

Pair-switching rerandomization

Rerandomization discards assignments with covariates unbalanced in the treatment and control groups to improve estimation and inference efficiency. However, the acceptance-rejection sampling method used in rerandomization is computationally inefficient. As a result, it is time-consuming for rerandomization to draw numerous independent assignments, which are necessary for performing Fisher randomization tests and constructing randomization-based confidence intervals. To address this problem, we propose a pair-switching rerandomization (PSRR) method to draw balanced assignments efficiently. We obtain the unbiasedness and variance reduction of the difference-in-means estimator and show that the Fisher randomization tests are valid under PSRR. Moreover, we propose an exact approach to invert Fisher randomization tests to confidence intervals, which is faster than the existing methods. In addition, our method is applicable to both nonsequentially and sequentially randomized experiments. We conduct comprehensive simulation studies to compare the finite-sample performance of the proposed method with that of classical rerandomization. Simulation results indicate that PSRR leads to comparable power of Fisher randomization tests and is 3–23 times faster than classical rerandomization. Finally, we apply the PSRR method to analyze two clinical trial datasets, both of which demonstrate the advantages of our method.