Multiple Comparisons Among Dependent Groups Based on a Modified One‐Step M‐Estimator

Currently, among multiple comparison procedures for dependent groups, a bootstrap‐t with a 20% trimmed mean performs relatively well in terms of both Type I error probabilities and power. However, trimmed means suffer from two general concerns described in the paper. Robust M‐estimators address these concerns, but now no method has been found that gives good control over the probability of a Type I error when sample sizes are small. The paper suggests using instead a modified one‐step M‐estimator that retains the advantages of both trimmed means and robust M‐estimators. Yet another concern is that the more successful methods for trimmed means can be too conservative in terms of Type I errors. Two methods for performing all pairwise multiple comparisons are considered. In simulations, both methods avoid a familywise error (FWE) rate larger than the nominal level. The method based on comparing measures of location associated with the marginal distributions can have an actual FWE that is well below the nominal level when variables are highly correlated. However, the method based on difference scores performs reasonably well with very small sample sizes, and it generally performs better than any of the methods studied in Wilcox (1997b).     

Confidence bands for logistic regression with restricted predictor variables

Confidence bands are constructed for the logistic response function when there is an interval restriction on each of the predictor variables. The construction involves application of a general fitting procedure using Scheffé's S-method, described by Casella and Strawderman (1980, Journal of the American Statistical Association 75, 862-868). Specific details are given for the case of one predictor variable, along with details for a fixed-width alternative to the S-method bands. In the one-predictor case, Monte Carlo results suggest that both bands are conservative for small sample sizes, such as N = 25. By N = 200 the S-method's coverage probabilities are seen to attain their nominal levels while the fixed-width bands remain conservative. The procedures are exemplified with data from a genetic toxicology experiment.     

A Bayesian approach for subgroup analysis

Several penalization approaches have been developed to identify homogeneous subgroups based on a regression model with subject-specific intercepts in subgroup analysis. These methods often apply concave penalty functions to pairwise comparisons of the intercepts, such that the subjects with similar intercept values are assigned to the same group, which is very similar to the procedure of the penalization approaches for variable selection. Since the Bayesian methods are commonly used in variable selection, it is worth considering the corresponding approaches to subgroup analysis in the Bayesian framework. In this paper, a Bayesian hierarchical model with appropriate prior structures is developed for the pairwise differences of intercepts based on a regression model with subject-specific intercepts, which can automatically detect and identify homogeneous subgroups. A Gibbs sampling algorithm is also provided to select the hyperparameter and estimate the intercepts and coefficients of the covariates simultaneously, which is computationally efficient for pairwise comparisons compared to the time-consuming procedures for parameter estimation of the penalization methods (e.g., alternating direction method of multiplier) in the case of large sample sizes. The effectiveness and usefulness of the proposed Bayesian method are evaluated through simulation studies and analysis of a Cleveland Heart Disease Dataset.     

Estimation of a parameter and its exact confidence interval following sequential sample size reestimation trials

For confirmatory trials of regulatory decision making, it is important that adaptive designs under consideration provide inference with the correct nominal level, as well as unbiased estimates, and confidence intervals for the treatment comparisons in the actual trials. However, naive point estimate and its confidence interval are often biased in adaptive sequential designs. We develop a new procedure for estimation following a test from a sample size reestimation design. The method for obtaining an exact confidence interval and point estimate is based on a general distribution property of a pivot function of the Self-designing group sequential clinical trial by Shen and Fisher (1999, Biometrics55, 190-197). A modified estimate is proposed to explicitly account for futility stopping boundary with reduced bias when block sizes are small. The proposed estimates are shown to be consistent. The computation of the estimates is straightforward. We also provide a modified weight function to improve the power of the test. Extensive simulation studies show that the exact confidence intervals have accurate nominal probability of coverage, and the proposed point estimates are nearly unbiased with practical sample sizes.     

Unbiased Confidence Intervals for the Odds Ratio of Two Independent Binomial Samples with Application to Case–Control Data

The problem of confidence interval construction for the odds ratio of two independent binomial samples is considered. Two methods of eliminating the nuisance parameter from the exact likelihood, conditioning and maximization, are described. A conditionally exact tail method exists by putting together upper and lower bounds. A shorter interval can be obtained by simultaneous consideration of both tails. We present here new methods that extend the tail and simultaneous approaches to the maximized likelihood. The methods are unbiased and applicable to case-control data, for which the odds ratio is important. The confidence interval procedures are compared unconditionally for small sample sizes in terms of their expected length and coverage probability. A Bayesian confidence interval method and a large-sample chi2 procedure are included in the comparisons.     

One-sided significance tests for generalized linear models under dichotomous response

Dichotomous response models are common in many experimental settings. Often, concomitant explanatory variables are recorded, and a generalized linear model, such as a logit model, is fit. In some cases, interest in specific model parameters is directed only at one-sided departures from some null effect. In these cases, procedures can be developed for testing the null effect against a one-sided alternative. These include Bonferroni-type adjustments of univariate Wald tests, and likelihood ratio tests that employ inequality-constrained multivariate theory. This paper examines such tests of significance. Monte Carlo evaluations are undertaken to examine the small-sample properties of the various procedures. The procedures are seen to perform fairly well, generally achieving their nominal sizes at total sample sizes near 100 experimental units. Extensions to the problem of one-sided tests against a control or standard are also considered.     

A Common Control Group - Optimising the Experiment Design to Maximise Sensitivity

Methods for choosing an appropriate sample size in animal experiments have received much attention in the statistical and biological literature. Due to ethical constraints the number of animals used is always reduced where possible. However, as the number of animals decreases so the risk of obtaining inconclusive results increases. By using a more efficient experimental design we can, for a given number of animals, reduce this risk. In this paper two popular cases are considered, where planned comparisons are made to compare treatments back to control and when researchers plan to make all pairwise comparisons. By using theoretical and empirical techniques we show that for studies where all pairwise comparisons are made the traditional balanced design, as suggested in the literature, maximises sensitivity. For studies that involve planned comparisons of the treatment groups back to the control group, which are inherently more sensitive due to the reduced multiple testing burden, the sensitivity is maximised by increasing the number of animals in the control group while decreasing the number in the treated groups.     

Approximate simultaneous confidence intervals for multiple contrasts of binomial proportions

Simultaneous confidence intervals for contrasts of means in a one-way layout with several independent samples are well established for Gaussian distributed data. Procedures addressing different hypotheses are available, such as all pairwise comparisons or comparisons to control, comparison with average, or different tests for order-restricted alternatives. However, if the distribution of the response is not Gaussian, corresponding methods are usually not available or not implemented in software. For the case of comparisons among several binomial proportions, we extended recently proposed confidence interval methods for the difference of two proportions or single contrasts to multiple contrasts by using quantiles of the multivariate normal distribution, taking the correlation into account. The small sample performance of the proposed methods was investigated in simulation studies. The simple adjustment of adding 2 pseudo-observations to each sample estimate leads to reasonable coverage probabilities. The methods are illustrated by the evaluation of real data examples of a clinical trial and a toxicological study. The proposed methods and examples are available in the R package MCPAN. ((c) 2008 WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim).     

Kernel estimates of dose response

A nonparametric method for analyzing quantal response data from an indirect bioassay experiment is proposed. Kernel estimates of the dose-response curve are used to develop approximate confidence intervals for (i) the optimal combination dose of a drug with therapeutic effects at low doses and toxic effects at high doses, and (ii) the lethal dose levels of a toxic chemical. This nonparametric procedure was implemented on real and simulated data. The confidence interval for problem (i) has high coverage probabilities when the dose-response curve is symmetric about the optima. However, the coverage probabilities are adversely affected by asymmetry about the optima and consequently are not reliable unless the sample sizes are large. The use of kernel estimators with higher-order kernels may alleviate this sensitivity to asymmetry. The confidence interval for problem (ii) has high coverage probabilities robust with respect to the shape or symmetry of the underlying dose-response curve.     

Improved exact confidence intervals for the odds ratio in two independent binomial samples

For two independent binomial samples, the usual exact confidence interval for the odds ratio based on the conditional approach can be very conservative. Recently, Agresti and Min (2002) showed that the unconditional intervals are preferable to conditional intervals with small sample sizes. We use the unconditional approach to obtain a modified interval, which has shorter length, and its coverage probability is closer to and at least the nominal confidence coefficient.     

Effects of cluster sampling on epidemiologic analysis in population-based case-control studies

We consider population-based case-control designs in which controls are selected by one of three cluster sampling plans from the entire population at risk. The effects of cluster sampling on classical epidemiologic procedures are investigated, and appropriately modified procedures are developed. In particular, modified procedures for testing the homogeneity of odds ratios across strata, and for estimating and testing a common odds ratio are presented. Simulations that use the data from the 1970 Health Interview Survey as a population suggest that classical procedures may be fairly robust in the presence of cluster sampling. A more extreme example based on a mixed multinomial model clearly demonstrates that the classical Mantel-Haenszel (1959, Journal of the National Cancer Institute 22, 719-748) and Woolf-Haldane tests of no exposure effect may have sizes exceeding nominal levels and confidence intervals with less than nominal coverage under an alternative hypothesis. Classical estimates of odds ratios may also be biased with non-self-weighting cluster samples. The modified procedures we propose remedy these defects.     

Analysis of covariance with pre-treatment measurements in randomized trials: comparison of equal and unequal slopes

In randomized trials, an analysis of covariance (ANCOVA) is often used to analyze post-treatment measurements with pre-treatment measurements as a covariate to compare two treatment groups. Random allocation guarantees only equal variances of pre-treatment measurements. We hence consider data with unequal covariances and variances of post-treatment measurements without assuming normality. Recently, we showed that the actual type I error rate of the usual ANCOVA assuming equal slopes and equal residual variances is asymptotically at a nominal level under equal sample sizes, and that of the ANCOVA with unequal variances is asymptotically at a nominal level, even under unequal sample sizes. In this paper, we investigated the asymptotic properties of the ANCOVA with unequal slopes for such data. The estimators of the treatment effect at the observed mean are identical between equal and unequal variance assumptions, and these are asymptotically normal estimators for the treatment effect at the true mean. However, the variances of these estimators based on standard formulas are biased, and the actual type I error rates are not at a nominal level, irrespective of variance assumptions. In equal sample sizes, the efficiency of the usual ANCOVA assuming equal slopes and equal variances is asymptotically the same as those of the ANCOVA with unequal slopes and higher than that of the ANCOVA with equal slopes and unequal variances. Therefore, the use of the usual ANCOVA is appropriate in equal sample sizes.     

Sharp simultaneous confidence intervals for the means of selected populations with application to microarray data analysis

Summary Simultaneous inference for a large number, N, of parameters is a challenge. In some situations, such as microarray experiments, researchers are only interested in making inference for the K parameters corresponding to the K most extreme estimates. Hence it seems important to construct simultaneous confidence intervals for these K parameters. The naïve simultaneous confidence intervals for the K means (applied directly without taking into account the selection) have low coverage probabilities. We take an empirical Bayes approach (or an approach based on the random effect model) to construct simultaneous confidence intervals with good coverage probabilities. For N= 10,000 and K= 100, typical for microarray data, our confidence intervals could be 77% shorter than the naïve K‐dimensional simultaneous intervals.     

A comparison of exact, mid-P, and score tests for matched case-control studies

A recently developed algorithm for generating the distribution of sufficient statistics for conditional logistic models can be put to a twofold use. First, it provides an avenue for performing inference for matched case-control studies that does not rely on the assumption of a large sample size. Second, joint distributions generated by this algorithm can be used to make comparisons of various inferential procedures that are free from Monte Carlo sampling errors. In this paper, these two features of the algorithm are utilized to compare small-sample properties of the exact, mid-P value, and score tests for a conditional logistic model with two unmatched binary covariates. Both uniparametric and multiparametric tests, performed at a nominal significance level of .05, were studied. It was found that the actual significance levels of the mid-P test tend to be closer to the nominal level when compared with those of the other two tests.     

Reliability of Confidence Interval Estimators under Various Nested Design Parental Sample Sizes

The coverage probabilities of several confidence limit estimators of genetic parameters, obtained from North Carolina I designs, were assessed by means of Monte Carlo simulations. The reliability of the estimators was compared under three different parental sample sizes. The coverage of confidence intervals set on the Normal distribution, and using standard errors either computed by the “delta” method or derived using an approximation for the variance of a variance component estimated by means of a linear combination of mean squares, was affected by the number of males and females included in the experiment. The “delta” method was found to provide reliable standard errors of the genetic parameters only when at least 48 males were each mated to six different females randomly selected from the reference population. Formulae are provided for obtaining “delta” method standard errors, and appropriate statistical software procedures are discussed. The error rates of confidence limits based on the Normal distribution and using standard errors obtained by an approximation for the variance of a variance component varied widely. The coverage of F-distribution confidence intervals for heritability estimates was not significantly affected by parental sample size and consistently provided a mean coverage near the stated coverage. For small parental sample sizes, confidence intervals for heritability estimates should be based on the F-distribution.     

Nonparametric All‐Pairs Multiple Comparisons

Nonparametric all‐pairs multiple comparisons based on pairwise rankings can be performed in the one‐way design with the Steel‐Dwass procedure. To apply this test, Wilcoxon's rank sum statistic is calculated for all pairs of groups; the maximum of the rank sums is the test statistic. We provide exact calculations of the asymptotic critical values (and P‐values, respectively) even for unbalanced designs. We recommend this asymptotic method whenever large sample sizes are present. For small sample sizes we recommend the use of the new statistic according to Baumgartner , Weiss , and Schindler (1998, Biometrics 54 , 1129–1135) instead of Wilcoxon's rank sum for the multiple comparisons. We show that the resultant procedure can be less conservative and, according to simulation results, more powerful than the original Steel‐Dwass procedure. We illustrate the methods with a practical data set.     

Small-sample confidence limits for parameters under inequality constraints with application to quantal bioassay

A family of methods is presented for constructing confidence limits for the parameters of a collection of distributions when a simple ordering is assumed among the parameters. The methods are shown to yield confidence limits that are exact or conservative for finite samples. For discrete distributions, one of the methods produces confidence limits that are at least as tight as the limits produced by a commonly used single-sample procedure. Confidence limits are demonstrated for a binomial quantal bioassay problem assuming a nondecreasing dose-response function. Results of a simulation study show that competing asymptotic methods can produce serious discrepancies between nominal and actual coverage probabilities for binomial samples of sizes up to 30, and demonstrate that the new approach can be competitive with the asymptotic methods when the latter maintain their nominal error rates.     

<Emphasis Type="Italic">Post hoc</Emphasis> pattern matching: assigning significance to statistically defined expression patterns in single channel microarray data

Background  

Researchers using RNA expression microarrays in experimental designs with more than two treatment groups often identify statistically significant genes with ANOVA approaches. However, the ANOVA test does not discriminate which of the multiple treatment groups differ from one another. Thus, post hoc tests, such as linear contrasts, template correlations, and pairwise comparisons are used. Linear contrasts and template correlations work extremely well, especially when the researcher has a priori information pointing to a particular pattern/template among the different treatment groups. Further, all pairwise comparisons can be used to identify particular, treatment group-dependent patterns of gene expression. However, these approaches are biased by the researcher's assumptions, and some treatment-based patterns may fail to be detected using these approaches. Finally, different patterns may have different probabilities of occurring by chance, importantly influencing researchers' conclusions about a pattern and its constituent genes.     

Simultaneous One-Sided Pairwise Comparisons in a Two-Way Design

Cheung and Chan (1996) provided a procedure for simultaneous two-sided pairwise comparisons of treatment means in a two-way design. Expanding on this earlier work, we develop a procedure where all such comparisons are instead one-sided. The new procedure is appropriate in situations where the treatment means are a priori ordered. The overall type I error rate for the simultaneous inferences is controlled at a designated level. Tables of upper percentage points of the test statistic are given to facilitate the implementation of our method. The application of the testing procedure is illustrated with an example from a medical study.     

Pairwise Comparisons in Each of Several Groups with Heterogenous Group Variances

Pairwise comparison procedures are frequently applied to analyze experimental results. In particular, practitioners in the area of medical researches often encounter situations which require these statistical techniques to compare various treatments. In this article, we focus on pairwise comparison procedures in a two‐factor design, where comparisons of one factor are made simultaneously for each level of another factor. For example, several new drugs to treat a certain cancer are being compared for both male and female patients. Previous research efforts were mainly devoted to models with homogeneous variances. The current paper is to address more common scenario where group variances are heterogeneous.