Many‐to‐one Comparisons in Stratified Designs

Cheung and Holland (1992) extended Dunnett's procedure for comparing all active treatments with a control simultaneously within each of r groups while maintaining the Type I error rate at some designated level α allowing different sample sizes for each of the group‐treatment categories. This paper shows that exact percentage points can be easily calculated with current available statistical software (SAS). This procedure is compared to resampling techniques and a Bonferroni corrected Dunnett‐within‐group procedure by means of a simulation study.     

Observations below multiple lower limits of quantification: How to estimate the mean and variance

Multiple lower limits of quantification (MLOQs) result if various laboratories are involved in the analysis of concentration data and some observations are too low to be quantified. For normally distributed data under MLOQs there exists only the multiple regression method of Helsel to estimate the mean and variance. We propose a simple imputation method and two new maximum likelihood estimation methods: the multiple truncated sample method and the multiple censored sample method. A simulation study is conducted to compare the performances of the newly introduced methods to Helsel's via the criteria root mean squared error (RMSE) and bias of the parameter estimates. Two and four lower limits of quantification (LLOQs), various amounts of unquantifiable observations and two sample sizes are studied. Furthermore, the robustness is investigated under model misspecification. The methods perform with decreasing accuracy for increasing rates of unquantified observations. Increasing sample sizes lead to smaller bias. There is almost no change in the performance between two and four LLOQs. The magnitude of the variance impairs the performance of all methods. For a smaller variance, the multiple censored sample method leads to superior estimates regarding the RMSE and bias, whereas Helsel's method is superior regarding the bias for a larger variance. Under model misspecification, Helsel's method was inferior to the other methods. Estimating the mean, the multiple censored sample method performed better, whereas the multiple truncated sample method performs best in estimating the variance. Summarizing, for a large sample size and normally distributed data we recommend to use Helsel's method. Otherwise, the multiple censored sample method should be used to obtain estimates of the mean and variance of data including MLOQs.     

Comparison of SANOVA Procedure of Simultaneous Testing Hypotheses to Others

In this paper a procedure SANOVA of simultaneous testing hypotheses is compared with others used in analysis of variance in a fixed linear model. The geometrical relation between SANOVA and Scheffé's confidence regions is discussed. It is shown that individual confidence intervals from SANOVA procedure are not longer than Scheffe's, Dunnett's and Tukey's ones. The cases, when they are the same are indicated. Theoretical considerations are illustrated by a practical example.     

Nonparametric All‐Pairs Multiple Comparisons

Nonparametric all‐pairs multiple comparisons based on pairwise rankings can be performed in the one‐way design with the Steel‐Dwass procedure. To apply this test, Wilcoxon's rank sum statistic is calculated for all pairs of groups; the maximum of the rank sums is the test statistic. We provide exact calculations of the asymptotic critical values (and P‐values, respectively) even for unbalanced designs. We recommend this asymptotic method whenever large sample sizes are present. For small sample sizes we recommend the use of the new statistic according to Baumgartner , Weiss , and Schindler (1998, Biometrics 54 , 1129–1135) instead of Wilcoxon's rank sum for the multiple comparisons. We show that the resultant procedure can be less conservative and, according to simulation results, more powerful than the original Steel‐Dwass procedure. We illustrate the methods with a practical data set.     

Harmonic Mean Approach to Regression Analysis Under Heteroscedasticity and Nonnormality

In this paper, by combining the harmonic mean approach with the Welch and the James procedure (see WELCH 1951, JAMES, 1951), we develop some robust procedures for testing parallelism in several straight lines under heteroscedasticity and nonnormality. Through Monte Carlo simulations it is shown that these new tests are quite robust with respect to departure from normality. For small sample sizes, however, the TAN-TABATABAI (1984) F_β and F*_β tests appear to be more powerful than the new tests, although when sample sizes are not small, there are hardly any differences between the Tan-Tabatabai F_β and F*_β tests and the new tests.     

Group Sequential Monitoring with Arbitrary Inspection Times

The classical group sequential test procedures that were proposed by Pocock (1977) and O'Brien and Fleming (1979) rest on the assumption of equal sample sizes between the interim analyses. Regarding this it is well known that for most situations there is not a great amount of additional Type I error if monitoring is performed for unequal sample sizes between the stages. In some cases, however, problems can arise resulting in an unacceptable liberal behavior of the test procedure. In this article worst case scenarios in sample size imbalancements between the inspection times are considered. Exact critical values for the Pocock and the O'Brien and Fleming group sequential designs are derived for arbitrary and for varying but bounded sample sizes. The approach represents a reasonable alternative to the flexible method that is based on the Type I error rate spending function. The SAS syntax for performing the calculations is provided. Using these procedures, the inspection times or the sample sizes in the consecutive stages need to be chosen independently of the data observed so far.     

A Robust Procedure for Comparing Multiple Means under Heteroscedasticity in Unbalanced Designs

Investigating differences between means of more than two groups or experimental conditions is a routine research question addressed in biology. In order to assess differences statistically, multiple comparison procedures are applied. The most prominent procedures of this type, the Dunnett and Tukey-Kramer test, control the probability of reporting at least one false positive result when the data are normally distributed and when the sample sizes and variances do not differ between groups. All three assumptions are non-realistic in biological research and any violation leads to an increased number of reported false positive results. Based on a general statistical framework for simultaneous inference and robust covariance estimators we propose a new statistical multiple comparison procedure for assessing multiple means. In contrast to the Dunnett or Tukey-Kramer tests, no assumptions regarding the distribution, sample sizes or variance homogeneity are necessary. The performance of the new procedure is assessed by means of its familywise error rate and power under different distributions. The practical merits are demonstrated by a reanalysis of fatty acid phenotypes of the bacterium Bacillus simplex from the “Evolution Canyons” I and II in Israel. The simulation results show that even under severely varying variances, the procedure controls the number of false positive findings very well. Thus, the here presented procedure works well under biologically realistic scenarios of unbalanced group sizes, non-normality and heteroscedasticity.     

Robust Univariate Two-Way Classification

We develop a robust classification procedure, based on TIKU'S (1967, 1980, 1982) MML (modified maximum likelihood) estimators, for classifying an observation in one of the two populations π₁ and π₂ and show that this procedure is superior to the classical and nonparametric procedures.     

Multiple comparisons for analyzing dichotomous response

Dichotomous response models are common in many experimental settings. Statistical parameters of interest are typically the probabilities, pi, that an experimental unit will respond at the various treatment levels. Herein, simultaneous procedures are considered for multiple comparisons among these probabilities, with attention directed at construction of simultaneous confidence intervals for various functions of the pi. The inferences are based on the asymptotic normality of the maximum likelihood estimator of pi. Specific applications include all pairwise comparisons and comparisons with a fixed (control) treatment. Monte Carlo evaluations are undertaken to examine the small-sample properties of the various procedures. It is seen that use of the usual estimates of variance consistently leads to less-than-nominal empirical coverage for most sample sizes examined. For very large samples (total size greater than about 300), nominal coverage is achieved. A reformulation of the pairwise comparisons using a construction noted by Beal (1987, Biometrics 43, 941-950) is shown to exhibit generally nominal empirical coverage characteristics, and is recommended for use with small-to-moderate sample sizes.     

A Booststrap Adaptive Test for Two-way Analysis of Variance

We propose a method to construct adaptive tests based on a bootstrap technique. The procedure leads to a nearly exact adaptive test depending on the size of the sample. With the use of the estimated Pitman's relative efficacy as selector statistic, we show that the adaptive test has a power that is asymptotically equal to the power of it's better component. We apply the idea to construct an adaptive test for two-way analysis of variance model. Finally, we use simulations to observe the behaviour of the method for small sample sizes.     

Nonparametric Two-Sample Tests for General Clinical Data

Starting from the discussion of a practical example a unifying concept for the derivation of meaningfully interpretable nonparametric tests for the two-sample case is developed which may well be adapted for other designs, too. This methodology covers other well-known procedures, e.g. Fisher's exact test, the Wilcoxon-Mann-Whitney and Gehan's tests, and may furthermore be extended to all situations sharing the same fundamental structural property of the sample space, namely its strict order induced by the substantial problem under study. The resulting test procedure is discussed for a randomization argument, exact and approximate, as well as for the general specific test problem. A numerical example is provided.     

Multiple Comparisons Among Dependent Groups Based on a Modified One‐Step M‐Estimator

Currently, among multiple comparison procedures for dependent groups, a bootstrap‐t with a 20% trimmed mean performs relatively well in terms of both Type I error probabilities and power. However, trimmed means suffer from two general concerns described in the paper. Robust M‐estimators address these concerns, but now no method has been found that gives good control over the probability of a Type I error when sample sizes are small. The paper suggests using instead a modified one‐step M‐estimator that retains the advantages of both trimmed means and robust M‐estimators. Yet another concern is that the more successful methods for trimmed means can be too conservative in terms of Type I errors. Two methods for performing all pairwise multiple comparisons are considered. In simulations, both methods avoid a familywise error (FWE) rate larger than the nominal level. The method based on comparing measures of location associated with the marginal distributions can have an actual FWE that is well below the nominal level when variables are highly correlated. However, the method based on difference scores performs reasonably well with very small sample sizes, and it generally performs better than any of the methods studied in Wilcox (1997b).     

Blinded and unblinded internal pilot study designs for clinical trials with count data

Internal pilot studies are a popular design feature to address uncertainties in the sample size calculations caused by vague information on nuisance parameters. Despite their popularity, only very recently blinded sample size reestimation procedures for trials with count data were proposed and their properties systematically investigated. Although blinded procedures are favored by regulatory authorities, practical application is somewhat limited by fears that blinded procedures are prone to bias if the treatment effect was misspecified in the planning. Here, we compare unblinded and blinded procedures with respect to bias, error rates, and sample size distribution. We find that both procedures maintain the desired power and that the unblinded procedure is slightly liberal whereas the actual significance level of the blinded procedure is close to the nominal level. Furthermore, we show that in situations where uncertainty about the assumed treatment effect exists, the blinded estimator of the control event rate is biased in contrast to the unblinded estimator, which results in differences in mean sample sizes in favor of the unblinded procedure. However, these differences are rather small compared to the deviations of the mean sample sizes from the sample size required to detect the true, but unknown effect. We demonstrate that the variation of the sample size resulting from the blinded procedure is in many practically relevant situations considerably smaller than the one of the unblinded procedures. The methods are extended to overdispersed counts using a quasi‐likelihood approach and are illustrated by trials in relapsing multiple sclerosis.     

Two-Stage Procedures for Comparing Treatments with a Control: Elimination at the First Stage and Estimation at the Second Stage

We consider the problem of comparing a set of p₁ test treatments with a control treatment. This is to be accomplished in two stages as follows: In the first stage, N₁ observations are allocated among the p₁ treatments and the control, and the subset selection procedure of Gupta and Sobel (1958) is employed to eliminate “inferior” treatments. In the second stage, N₂ observations are allocated among the (randomly) selected subset of p₂(≤p₁) treatments and the control, and joint confidence interval estimates of the treatment versus control differences are calculated using Dunnett's (1955) procedure. Here both N₁ and N₂ are assumed to be fixed in advance, and the so-called square root rule is used to allocate observations among the treatments and the control in each stage. Dunnett's procedure is applied using two different types of estimates of the treatment versus control mean differences: The unpooled estimates are based on only the data obtained in the second stage, while the pooled estimates are based on the data obtained in both stages. The procedure based on unpooled estimates uses the critical point from a p₂-variate Student t-distribution, while that based on pooled estimates uses the critical point from a p₁-variate Student t-distribution. The two procedures and a composite of the two are compared via Monte Carlo simulation. It is shown that the expected value of p₂ determines which procedure yields shorter confidence intervals on the average. Extensions of the procedures to the case of unequal sample sizes are given. Applicability of the proposed two-stage procedures to a drug screening problem is discussed.     

Multiple comparisons in the randomization analysis of designed experiments with growth curve responses

A randomization approach to multiple comparisons is developed for comparing several growth curves in randomized experiments. The exact Type I probability error rate for these comparisons may be prespecified, and a Type I error probability for each component test can be evaluated. These procedures are free of many of the standard assumptions for analyzing growth curves and for making multiple comparisons. An application of the procedure gives all pairwise comparisons among the mean growth curves associated with four treatments in an animal experiment using a Youden square design, where growth curves are obtained on monitoring hormone levels over time.     

A Unified Approach to Simultaneous Rank Test Procedures in the Unbalanced One‐way Layout

A general nonparametric approach to asymptotic multiple test procedures is proposed which is based on relative effects and which includes continuous as well as discontinuous distributions. The results can be applied to all relevant multiple testing problems in the one‐way layout and include the well known Steel tests as special cases. Moreover, a general estimator for the asymptotic covariance matrix is considered that is consistent even under alternative. This estimator is used to derive simultaneous confidence intervals for the relative effects as well as a test procedure for the multiple nonparametric Behrens‐Fisher problem.     

Imputation strategies for missing continuous outcomes in cluster randomized trials

In cluster randomized trials, intact social units such as schools, worksites or medical practices - rather than individuals themselves - are randomly allocated to intervention and control conditions, while the outcomes of interest are then observed on individuals within each cluster. Such trials are becoming increasingly common in the fields of health promotion and health services research. Attrition is a common occurrence in randomized trials, and a standard approach for dealing with the resulting missing values is imputation. We consider imputation strategies for missing continuous outcomes, focusing on trials with a completely randomized design in which fixed cohorts from each cluster are enrolled prior to random assignment. We compare five different imputation strategies with respect to Type I and Type II error rates of the adjusted two-sample t -test for the intervention effect. Cluster mean imputation is compared with multiple imputation, using either within-cluster data or data pooled across clusters in each intervention group. In the case of pooling across clusters, we distinguish between standard multiple imputation procedures which do not account for intracluster correlation and a specialized procedure which does account for intracluster correlation but is not yet available in standard statistical software packages. A simulation study is used to evaluate the influence of cluster size, number of clusters, degree of intracluster correlation, and variability among cluster follow-up rates. We show that cluster mean imputation yields valid inferences and given its simplicity, may be an attractive option in some large community intervention trials which are subject to individual-level attrition only; however, it may yield less powerful inferences than alternative procedures which pool across clusters especially when the cluster sizes are small and cluster follow-up rates are highly variable. When pooling across clusters, the imputation procedure should generally take intracluster correlation into account to obtain valid inferences; however, as long as the intracluster correlation coefficient is small, we show that standard multiple imputation procedures may yield acceptable type I error rates; moreover, these procedures may yield more powerful inferences than a specialized procedure, especially when the number of available clusters is small. Within-cluster multiple imputation is shown to be the least powerful among the procedures considered.     

Comparison of asymptotic population growth rates with heterogeneous variances

This report explores how the heterogeneity of variances affects randomization tests used to evaluate differences in the asymptotic population growth rate, λ. The probability of Type I error was calculated in four scenarios for populations with identical λ but different variance of λ: (1) Populations have different projection matrices: the same λ may be obtained from different sets of vital rates, which gives room for different variances of λ. (2) Populations have identical projection matrices but reproductive schemes differ and fecundity in one of the populations has a larger associated variance. The two other scenarios evaluate a sampling artifact as responsible for heterogeneity of variances. The same population is sampled twice, (3) with the same sampling design, or (4) with different sampling effort for different stages. Randomization tests were done with increasing differences in sample size between the two populations. This implies additional differences in the variance of λ. The probability of Type I error keeps at the nominal significance level (α = .05) in Scenario 3 and with identical sample sizes in the others. Tests were too liberal, or conservative, under a combination of variance heterogeneity and different sample sizes. Increased differences in sample size exacerbated the difference between observed Type I error and the nominal significance level. Type I error increases or decreases depending on which population has a larger sample size, the population with the smallest or the largest variance. However, by their own, sample size is not responsible for changes in Type I errors.     

Nonparametric Multiple Comparisons in Repeated Measures Designs for Data with Ties

We consider some multiple comparison problems in repeated measures designs for data with ties, particularly ordinal data; the methods are also applicable to continuous data, with or without ties. A unified asymptotic theory of rank tests of Brunner , Puri and Sen (1995) and Akritas and Brunner (1997) is utilized to derive large sample multiple comparison procedures (MCP's). First, we consider a single treatment and address the problem of comparing its time effects with respect to the baseline. Multiple sign tests and rank tests (and the corresponding simultaneous confidence intervals) are derived for this problem. Next, we consider two treatments and address the problem of testing for treatment × time interactions by comparing their time effects with respect to the baseline. Simulation studies are conducted to study the type I familywise error rates and powers of competing procedures under different distributional models. The data from a psychiatric study are analyzed using the above MCP's to answer the clinicians' questions.     

A Comparison of Population Size Estimators under the Truncated Count Model With and Without Allowance for Contaminations

The purpose of the study is to estimate the population size under a homogeneous truncated count model and under model contaminations via the Horvitz‐Thompson approach on the basis of a count capture‐recapture experiment. The proposed estimator is based on a mixture of zero‐truncated Poisson distributions. The benefit of using the proposed model is statistical inference of the long‐tailed or skewed distributions and the concavity of the likelihood function with strong results available on the nonparametric maximum likelihood estimator (NPMLE). The results of comparisons, for finding the appropriate estimator among McKendrick's, Mantel‐Haenszel's, Zelterman's, Chao's, the maximum likelihood, and the proposed methods in a simulation study, reveal that under model contaminations the proposed estimator provides the best choice according to its smallest bias and smallest mean square error for a situation of sufficiently large population sizes and the further results show that the proposed estimator performs well even for a homogeneous situation. The empirical examples, containing the cholera epidemic in India based on homogeneity and the heroin user data in Bangkok 2002 based on heterogeneity, are fitted with an excellent goodness‐of‐fit of the models and the confidence interval estimations may also be of considerable interest. (© 2008 WILEY‐VCH Verlag GmbH & Co. KGaA, Weinheim)