On the Use of the Shapiro‐Wilk Test in Two‐Stage Adaptive Inference for Paired Data from Moderate to Very Heavy Tailed Distributions

Paired data arises in a wide variety of applications where often the underlying distribution of the paired differences is unknown. When the differences are normally distributed, the t‐test is optimum. On the other hand, if the differences are not normal, the t‐test can have substantially less power than the appropriate optimum test, which depends on the unknown distribution. In textbooks, when the normality of the differences is questionable, typically the non‐parametric Wilcoxon signed rank test is suggested. An adaptive procedure that uses the Shapiro‐Wilk test of normality to decide whether to use the t‐test or the Wilcoxon signed rank test has been employed in several studies. Faced with data from heavy tails, the U.S. Environmental Protection Agency (EPA) introduced another approach: it applies both the sign and t‐tests to the paired differences, the alternative hypothesis is accepted if either test is significant. This paper investigates the statistical properties of a currently used adaptive test, the EPA's method and suggests an alternative technique. The new procedure is easy to use and generally has higher empirical power, especially when the differences are heavy‐tailed, than currently used methods.     

On testing simultaneously non-inferiority in two multiple primary endpoints and superiority in at least one of them

In a clinical trial with an active treatment and a placebo the situation may occur that two (or even more) primary endpoints may be necessary to describe the active treatment's benefit. The focus of our interest is a more specific situation with two primary endpoints in which superiority in one of them would suffice given that non-inferiority is observed in the other. Several proposals exist in the literature for dealing with this or similar problems, but prove insufficient or inadequate at a closer look (e.g. Bloch et al. (2001, 2006) or Tamhane and Logan (2002, 2004)). For example, we were unable to find a good reason why a bootstrap p-value for superiority should depend on the initially selected non-inferiority margins or on the initially selected type I error alpha. We propose a hierarchical three step procedure, where non-inferiority in both variables must be proven in the first step, superiority has to be shown by a bivariate test (e.g. Holm (1979), O'Brien (1984), Hochberg (1988), a bootstrap (Wang (1998)), or L?uter (1996)) in the second step, and then superiority in at least one variable has to be verified in the third step by a corresponding univariate test. All statistical tests are performed at the same one-sided significance level alpha. From the above mentioned bivariate superiority tests we preferred L?uter's SS test and the Holm procedure for the reason that these have been proven to control the type I error strictly, irrespective of the correlation structure among the primary variables and the sample size applied. A simulation study reveals that the performance regarding power of the bivariate test depends to a considerable degree on the correlation and on the magnitude of the expected effects of the two primary endpoints. Therefore, the recommendation of which test to choose depends on knowledge of the possible correlation between the two primary endpoints. In general, L?uter's SS procedure in step 2 shows the best overall properties, whereas Holm's procedure shows an advantage if both a positive correlation between the two variables and a considerable difference between their standardized effect sizes can be expected.     

Distance-based tests for homogeneity of multivariate dispersions

Summary The traditional likelihood‐based test for differences in multivariate dispersions is known to be sensitive to nonnormality. It is also impossible to use when the number of variables exceeds the number of observations. Many biological and ecological data sets have many variables, are highly skewed, and are zero‐inflated. The traditional test and even some more robust alternatives are also unreasonable in many contexts where measures of dispersion based on a non‐Euclidean dissimilarity would be more appropriate. Distance‐based tests of homogeneity of multivariate dispersions, which can be based on any dissimilarity measure of choice, are proposed here. They rely on the rotational invariance of either the multivariate centroid or the spatial median to obtain measures of spread using principal coordinate axes. The tests are straightforward multivariate extensions of Levene's test, with P‐values obtained either using the traditional F‐distribution or using permutation of either least‐squares or LAD residuals. Examples illustrate the utility of the approach, including the analysis of stabilizing selection in sparrows, biodiversity of New Zealand fish assemblages, and the response of Indonesian reef corals to an El Niño. Monte Carlo simulations from the real data sets show that the distance‐based tests are robust and powerful for relevant alternative hypotheses of real differences in spread.     

Statistical Tests Based on New Composite Hypotheses in Clinical Trials Reflecting the Relative Clinical Importance of Multiple Endpoints Quantitatively

In clinical trials, several endpoints (EPs) are often evaluated to compare treatments in some therapeutic area. Suppose that there are two EPs in a clinical trial. We propose a new set of composite hypotheses for continuous variables, taking the relative clinical importance of the EPs into account. The main hypotheses were formulated to show that a treatment is so superior to the control treatment, which is not necessarily a placebo, in one EP, that the possible non‐inferiority of the treatment by at most a certain value in the other EP can be compensated sufficiently, taking the clinical point of view into account. The maximum non‐inferiority margin of one EP might not be a biologically unimportant difference in exchange for much superiority of the other EP. This formulation leads to a new composite EP and a very simple test statistic. The intersection‐union principle was employed to derive the proposed test.     

High‐dimensional data analysis: Selection of variables,data compression and graphics – Application to gene expression

The paper presents effective and mathematically exact procedures for selection of variables which are applicable in cases with a very high dimension as, for example, in gene expression analysis. Choosing sets of variables is an important method to increase the power of the statistical conclusions and to facilitate the biological interpretation. For the construction of sets, each single variable is considered as the centre of potential sets of variables. Testing for significance is carried out by means of the Westfall‐Young principle based on resampling or by the parametric method of spherical tests. The particular requirements for statistical stability are taken into account; each kind of overfitting is avoided. Thus, high power is attained and the familywise type I error can be kept in spite of the large dimension. To obtain graphical representations by heat maps and curves, a specific data compression technique is applied. Gene expression data from B‐cell lymphoma patients serve for the demonstration of the procedures.     

Application of Multiple Test Procedures to the Combination of Multivariate and Univariate Tests with Varying Variable Sets

The positive ascertainment of location differences in a multivariate comparison of two or more groups gives rise to the question for the contribution of the single variables or of subsets of variables to the multivariate difference. In this paper two methods are proposed to accomplish the original multivariate test by tests in variable subsets or in single variables using a closed test procedure and Holm's procedure, respectively. Both control the multiple level of the whole procedure.     

One‐Sided Test to Assess Correlation in Linear Logistic Models using Estimating Equations

A score‐type test is proposed for testing the hypothesis of independent binary random variables against positive correlation in linear logistic models with sparse data and cluster specific covariates. The test is developed for univariate and multivariate one‐sided alternatives. The main advantage of using score test is that it requires estimation of the model only under the null hypothesis, that in this case corresponds to the binomial maximum likelihood fit. The score‐type test is developed from a class of estimating equations with block‐diagonal structure in which the coefficients of the linear logistic model are estimated simultaneously with the correlation. The simplicity of the score test is illustrated in two particular examples.     

A simple test of association for contingency tables with multiple column responses

Loughin and Scherer (1998, Biometrics 54, 630-637) investigated tests of association in two-way tables when one of the categorical variables allows for multiple-category responses from individual respondents. Standard chi-squared tests are invalid in this case, and they developed a bootstrap test procedure that provides good control of test levels under the null hypothesis. This procedure and some others that have been proposed are computationally involved and are based on techniques that are relatively unfamiliar to many practitioners. In this paper, the methods introduced by Rao and Scott (1981, Journal of the American Statistical Association 76, 221-230) for analyzing complex survey data are used to develop a simple test based on a corrected chi-squared statistic.     

A note on tests for relevant differences with extremely large sample sizes

A well‐known problem in classical two‐tailed hypothesis testing is that P‐values go to zero when the sample size goes to infinity, irrespectively of the effect size. This pitfall can make the testing of data consisting of large sample sizes potentially unreliable. In this note, we propose to test for relevant differences to overcome this issue. We illustrate the proposed test on a real data set of about 40 million privately insured patients.     

Assessment of Optimal Selected Prognostic Factors

The identification and assessment of prognostic factors is one of the major tasks in clinical research. The assessment of one single prognostic factor can be done by recently established methods for using optimal cutpoints. Here, we suggest a method to consider an optimal selected prognostic factor from a set of prognostic factors of interest. This can be viewed as a variable selection method and is the underlying decision problem at each node of various tree building algorithms. We propose to use maximally selected statistics where the selection is defined over the set of prognostic factors and over all cutpoints in each prognostic factor. We demonstrate that it is feasible to compute the approximate null distribution. We illustrate the new variable selection test with data of the German Breast Cancer Study Group and of a small study on patients with diffuse large B‐cell lymphoma. Using the null distribution for a p‐value adjusted regression trees algorithm, we adjust for the number of variables analysed at each node as well. (© 2004 WILEY‐VCH Verlag GmbH & Co. KGaA, Weinheim)     

Testing Average Equivalence — Finding a Compromise Between Theory and Practice

Recently, Brown , Hwang , and Munk (1998) proposed and unbiased test for the average equivalence problem which improves noticeably in power on the standard two one‐sided tests procedure. Nevertheless, from a practical point of view there are some objections against the use of this test which are mainly adressed to the ‘unusual’ shape of the critical region. We show that every unbiased test has a critical region with such an ‘unusual’ shape. Therefore, we discuss three (biased) modifications of the unbiased test. We come to the conclusion that a suitable modification represents a good compromise between a most powerful test and a test with an appealing shape of its critical region. In order to perform these tests figures are given containing the rejection region. Finally, we compare all tests in an example from neurophysiology. This shows that it is beneficial to use these improved tests instead of the two one‐sided tests procedure.     

Testing for Genetic Association in the Presence of Linkage and Gene–Covariate Interactions

In order to study family‐based association in the presence of linkage, we extend a generalized linear mixed model proposed for genetic linkage analysis (Lebrec and van Houwelingen (2007), Human Heredity 64 , 5–15) by adding a genotypic effect to the mean. The corresponding score test is a weighted family‐based association tests statistic, where the weight depends on the linkage effect and on other genetic and shared environmental effects. For testing of genetic association in the presence of gene–covariate interaction, we propose a linear regression method where the family‐specific score statistic is regressed on family‐specific covariates. Both statistics are straightforward to compute. Simulation results show that adjusting the weight for the within‐family variance structure may be a powerful approach in the presence of environmental effects. The test statistic for genetic association in the presence of gene–covariate interaction improved the power for detecting association. For illustration, we analyze the rheumatoid arthritis data from GAW15. Adjusting for smoking and anti‐cyclic citrullinated peptide increased the significance of the association with the DR locus.     

Meta‐Analysis of Diagnostic Test Accuracy Studies with Multiple Thresholds using Survival Methods

Diagnostic tests play an important role in clinical practice. The objective of a diagnostic test accuracy study is to compare an experimental diagnostic test with a reference standard. The majority of these studies dichotomize test results into two categories: negative and positive. But often the underlying test results may be categorized into more than two, ordered, categories. This article concerns the situation where multiple studies have evaluated the same diagnostic test with the same multiple thresholds in a population of non‐diseased and diseased individuals. Recently, bivariate meta‐analysis has been proposed for the pooling of sensitivity and specificity, which are likely to be negatively correlated within studies. These ideas have been extended to the situation of diagnostic tests with multiple thresholds, leading to a multinomial model with multivariate normal between‐study variation. This approach is efficient, but computer‐intensive and its convergence is highly dependent on starting values. Moreover, monotonicity of the sensitivities/specificities for increasing thresholds is not guaranteed. Here, we propose a Poisson‐correlated gamma frailty model, previously applied to a seemingly quite different situation, meta‐analysis of paired survival curves. Since the approach is based on hazards, it guarantees monotonicity of the sensitivities/specificities for increasing thresholds. The approach is less efficient than the multinomial/normal approach. On the other hand, the Poisson‐correlated gamma frailty model makes no assumptions on the relationship between sensitivity and specificity, gives consistent results, appears to be quite robust against different between‐study variation models, and is computationally very fast and reliable with regard to the overall sensitivities/specificities.     

Nonparametric All‐Pairs Multiple Comparisons

Nonparametric all‐pairs multiple comparisons based on pairwise rankings can be performed in the one‐way design with the Steel‐Dwass procedure. To apply this test, Wilcoxon's rank sum statistic is calculated for all pairs of groups; the maximum of the rank sums is the test statistic. We provide exact calculations of the asymptotic critical values (and P‐values, respectively) even for unbalanced designs. We recommend this asymptotic method whenever large sample sizes are present. For small sample sizes we recommend the use of the new statistic according to Baumgartner , Weiss , and Schindler (1998, Biometrics 54 , 1129–1135) instead of Wilcoxon's rank sum for the multiple comparisons. We show that the resultant procedure can be less conservative and, according to simulation results, more powerful than the original Steel‐Dwass procedure. We illustrate the methods with a practical data set.     

Evaluation of Animal Carcinogenicity Studies: Cochran‐Armitage Trend Test vs. Multiple Contrast Tests

Typical animal carcinogenicity studies involve the comparison of several dose groups to a negative control. The uncorrected asymptotic Cochran‐Armitage trend test with equally spaced dose scores is the most frequently used test in such set‐ups. However, this test based on a weighted linear regression on proportions. It is well known that the Cochran‐Armitage test lacks in power for other shapes than the assumed linear one. Therefore, dichotomous multiple contrast tests are introduced. These build the maximum over several single contrasts, where each of them is chosen appropriately to cover a specific dose‐response shape. An extensive power study has been conducted to compare multiple contrast tests with the approaches used so far. Crucial results will be presented in this paper. Moreover, exact tests and continuity corrected versions are introduced and compared to the traditional uncorrected approaches regarding size and power behaviour. A trend test for any shape of the dose‐response relationship for either crude tumour rates or mortality‐ adjusted rates based on the simple Poly‐3 transformation is proposed for evaluation of carcinogenicity studies.     

A Penalized Likelihood Approach for Bivariate Conditional Normal Models for Dynamic Co‐expression Analysis

Summary Gene co‐expressions have been widely used in the analysis of microarray gene expression data. However, the co‐expression patterns between two genes can be mediated by cellular states, as reflected by expression of other genes, single nucleotide polymorphisms, and activity of protein kinases. In this article, we introduce a bivariate conditional normal model for identifying the variables that can mediate the co‐expression patterns between two genes. Based on this model, we introduce a likelihood ratio (LR) test and a penalized likelihood procedure for identifying the mediators that affect gene co‐expression patterns. We propose an efficient computational algorithm based on iterative reweighted least squares and cyclic coordinate descent and have shown that when the tuning parameter in the penalized likelihood is appropriately selected, such a procedure has the oracle property in selecting the variables. We present simulation results to compare with existing methods and show that the LR‐based approach can perform similarly or better than the existing method of liquid association and the penalized likelihood procedure can be quite effective in selecting the mediators. We apply the proposed method to yeast gene expression data in order to identify the kinases or single nucleotide polymorphisms that mediate the co‐expression patterns between genes.     

Misprescription and misuse of one‐tailed tests

One‐tailed statistical tests are often used in ecology, animal behaviour and in most other fields in the biological and social sciences. Here we review the frequency of their use in the 1989 and 2005 volumes of two journals (Animal Behaviour and Oecologia), their advantages and disadvantages, the extensive erroneous advice on them in both older and modern statistics texts and their utility in certain narrow areas of applied research. Of those articles with data sets susceptible to one‐tailed tests, at least 24% in Animal Behaviour and at least 13% in Oecologia used one‐tailed tests at least once. They were used 35% more frequently with nonparametric methods than with parametric ones and about twice as often in 1989 as in 2005. Debate in the psychological literature of the 1950s established the logical criterion that one‐tailed tests should be restricted to situations where there is interest only in results in one direction. ‘Interest’ should be defined; however, in terms of collective or societal interest and not by the individual investigator. By this ‘collective interest’ criterion, all uses of one‐tailed tests in the journals surveyed seem invalid. In his book Nonparametric Statistics, S. Siegel unrelentingly suggested the use of one‐tailed tests whenever the investigator predicts the direction of a result. That work has been a major proximate source of confusion on this issue, but so are most recent statistics textbooks. The utility of one‐tailed tests in research aimed at obtaining regulatory approval of new drugs and new pesticides is briefly described, to exemplify the narrow range of research situations where such tests can be appropriate. These situations are characterized by null hypotheses stating that the difference or effect size does not exceed, or is at least as great as, some ‘amount of practical interest’. One‐tailed tests rarely should be used for basic or applied research in ecology, animal behaviour or any other science.     

Biology and host specificity of Aulacobaris fallax (Coleoptera: Curculionidae), a potential biological control agent for dyer’s woad,Isatis tinctoria (Brassicaceae) in North America

Dyer’s woad, Isatis tinctoria, a plant of Eurasian origin is a problematic weed in western North America against which a classical biological weed control programme was initiated in 2004. Three European insect species were selected as candidate agents to control this invasive species, including the root‐mining weevil Aulacobaris fallax. To determine its suitability as an agent, the biology and host specificity of A. fallax were studied in outdoor plots and in the field between 2004 and 2006 in its native European range. Aulacobaris fallax is a univoltine species that lays its eggs from March to August into leaf stalks and roots of dyer’s woad. Larvae mine and pupate in the roots and adults emerge from August to October. Up to 62% of the dyer’s woad plants at the field sites investigated were attacked by this weevil. In no‐choice host‐specificity tests, A. fallax attacked 16 out of 39 species and varieties within the Family Brassicaceae. Twelve of these are native to North America. In subsequent multiple‐choice tests, seven species, all native to North America, suffered a similar level of attack as dyer’s woad, while none of the European species were attacked. Our results demonstrate the importance of including test plant species that have not co‐evolved with the respective candidate agent. In sum, we conclude that the risk of non‐target effects is too high for A. fallax to be considered as a biological control agent for dyer’s woad in the United States.