Sample size determination for the false discovery rate

MOTIVATION: There is not a widely applicable method to determine the sample size for experiments basing statistical significance on the false discovery rate (FDR). RESULTS: We propose and develop the anticipated FDR (aFDR) as a conceptual tool for determining sample size. We derive mathematical expressions for the aFDR and anticipated average statistical power. These expressions are used to develop a general algorithm to determine sample size. We provide specific details on how to implement the algorithm for a k-group (k > or = 2) comparisons. The algorithm performs well for k-group comparisons in a series of traditional simulations and in a real-data simulation conducted by resampling from a large, publicly available dataset. AVAILABILITY: Documented S-plus and R code libraries are freely available from www.stjuderesearch.org/depts/biostats.     

Sample size determination for classifiers based on single-nucleotide polymorphisms

Single-nucleotide polymorphisms (SNPs), believed to determine human differences, are widely used to predict risk of diseases. Typically, clinical samples are limited and/or the sampling cost is high. Thus, it is essential to determine an adequate sample size needed to build a classifier based on SNPs. Such a classifier would facilitate correct classifications, while keeping the sample size to a minimum, thereby making the studies cost-effective. For coded SNP data from 2 classes, an optimal classifier and an approximation to its probability of correct classification (PCC) are derived. A linear classifier is constructed and an approximation to its PCC is also derived. These approximations are validated through a variety of Monte Carlo simulations. A sample size determination algorithm based on the criterion, which ensures that the difference between the 2 approximate PCCs is below a threshold, is given and its effectiveness is illustrated via simulations. For the HapMap data on Chinese and Japanese populations, a linear classifier is built using 51 independent SNPs, and the required total sample sizes are determined using our algorithm, as the threshold varies. For example, when the threshold value is 0.05, our algorithm determines a total sample size of 166 (83 for Chinese and 83 for Japanese) that satisfies the criterion.     

Sample size determination for matched-pair equivalence trials using rate ratio

In this article, we compare Wald-type, logarithmic transformation, and Fieller-type statistics for the classical 2-sided equivalence testing of the rate ratio under matched-pair designs with a binary end point. These statistics can be implemented through sample-based, constrained least squares estimation and constrained maximum likelihood (CML) estimation methods. Sample size formulae based on the CML estimation method are developed. We consider formulae that control a prespecified power or confidence width. Our simulation studies show that statistics based on the CML estimation method generally outperform other statistics and methods with respect to actual type I error rate and average width of confidence intervals. Also, the corresponding sample size formulae are valid asymptotically in the sense that the exact power and actual coverage probability for the estimated sample size are generally close to their prespecified values. The methods are illustrated with a real example from a clinical laboratory study.     

Sample size determination for hierarchical longitudinal designs with differential attrition rates

Summary .   We consider the problem of sample size determination for three-level mixed-effects linear regression models for the analysis of clustered longitudinal data. Three-level designs are used in many areas, but in particular, multicenter randomized longitudinal clinical trials in medical or health-related research. In this case, level 1 represents measurement occasion, level 2 represents subject, and level 3 represents center. The model we consider involves random effects of the time trends at both the subject level and the center level. In the most common case, we have two random effects (constant and a single trend), at both subject and center levels. The approach presented here is general with respect to sampling proportions, number of groups, and attrition rates over time. In addition, we also develop a cost model, as an aid in selecting the most parsimonious of several possible competing models (i.e., different combinations of centers, subjects within centers, and measurement occasions). We derive sample size requirements (i.e., power characteristics) for a test of treatment-by-time interaction(s) for designs based on either subject-level or cluster-level randomization. The general methodology is illustrated using two characteristic examples.     

Sample size determination for testing whether an identified treatment is best

Laska and Meisner (1989, Biometrics 45, 1139-1151) dealt with the problem of testing whether an identified treatment belonging to a set of k + 1 treatments is better than each of the other k treatments. They calculated sample size tables for k = 2 when using multiple t-tests or Wilcoxon-Mann-Whitney tests, both under normality assumptions. In this paper, we provide sample size formulas as well as tables for sample size determination for k > or = 2 when t-tests under normality or Wilcoxon-Mann-Whitney tests under general distribution assumptions are used.     

Sample size determination in microarray experiments for class comparison and prognostic classification

Determining sample sizes for microarray experiments is important but the complexity of these experiments, and the large amounts of data they produce, can make the sample size issue seem daunting, and tempt researchers to use rules of thumb in place of formal calculations based on the goals of the experiment. Here we present formulae for determining sample sizes to achieve a variety of experimental goals, including class comparison and the development of prognostic markers. Results are derived which describe the impact of pooling, technical replicates and dye-swap arrays on sample size requirements. These results are shown to depend on the relative sizes of different sources of variability. A variety of common types of experimental situations and designs used with single-label and dual-label microarrays are considered. We discuss procedures for controlling the false discovery rate. Our calculations are based on relatively simple yet realistic statistical models for the data, and provide straightforward sample size calculation formulae.     

Sample size determination in clinical trials with an emphasis on exponentially distributed responses

Sample size determination in clinical trials (and other similar studies) depends on a number of factors including the distribution of patient survival (remission) times, available estimates of the requisite parameters of the distribution under the null and alternative hypotheses, sizes of the Type I and Type II errors, and the length of the clinical trial, which in turn determines whether there are many, few, or no censored observations with regard to patient survival (remission). A further consideration is the patient recruitment period, which is assumed to begin simultaneously with the clinical trial but whose length is less than the length of the clinical trial. The purpose of this article is to explore the optimum lengths of the clinical trial and the recruitment period on the basis of minimizing the expected cost of the trial. A specified cost function, patient entry distribution, and exponential survival distribution are all assumed, primarily for illustrative purposes.     

Sample size determination in clinical proteomic profiling experiments using mass spectrometry for class comparison

Mass spectrometric profiling approaches such as MALDI‐TOF and SELDI‐TOF are increasingly being used in disease marker discovery, particularly in the lower molecular weight proteome. However, little consideration has been given to the issue of sample size in experimental design. The aim of this study was to develop a protocol for the use of sample size calculations in proteomic profiling studies using MS. These sample size calculations can be based on a simple linear mixed model which allows the inclusion of estimates of biological and technical variation inherent in the experiment. The use of a pilot experiment to estimate these components of variance is investigated and is shown to work well when compared with larger studies. Examination of data from a number of studies using different sample types and different chromatographic surfaces shows the need for sample‐ and preparation‐specific sample size calculations.     

Sample size for beginners.

The common failure to include an estimation of sample size in grant proposals imposes a major handicap on applicants, particularly for those proposing work in any aspect of research in the health services. Members of research committees need evidence that a study is of adequate size for there to be a reasonable chance of a clear answer at the end. A simple illustrated explanation of the concepts in determining sample size should encourage the faint hearted to pay more attention to this increasingly important aspect of grantsmanship.     

Sample size determination for case-control studies and the comparison of stratified and unstratified analyses.

Woolson, Bean, and Rojas (1986, Biometrics 42, 927-932) present a simple approximation of sample size for Cochran's (1954, Biometrics 10, 417-451) test for detecting association between exposure and disease. It is useful in the design of case-control studies. We derive a sample size formula for Cochran's statistic with continuity correction which guarantees that the actual Type I error rate of the test does not exceed the nominal level. The corrected sample size is necessarily larger than the uncorrected one given by Woolson et al. and the relative difference between the two sample sizes is considerable. Allocation of equal number of cases and controls within each stratum is asymptotically optimal when the costs per case and control are the same. When any effect of stratification is absent, Cochran's stratified test, although valid, is less efficient than the unstratified one except for the important case of a balanced design.     

Sample size reestimation by Bayesian prediction

We review a Bayesian predictive approach for interim data monitoring and propose its application to interim sample size reestimation for clinical trials. Based on interim data, this approach predicts how the sample size of a clinical trial needs to be adjusted so as to claim a success at the conclusion of the trial with an expected probability. The method is compared with predictive power and conditional power approaches using clinical trial data. Advantages of this approach over the others are discussed.     

Sample size determination for establishing equivalence/noninferiority via ratio of two proportions in matched-pair design

In this article, we propose approximate sample size formulas for establishing equivalence or noninferiority of two treatments in match-pairs design. Using the ratio of two proportions as the equivalence measure, we derive sample size formulas based on a score statistic for two types of analyses: hypothesis testing and confidence interval estimation. Depending on the purpose of a study, these formulas can be used to provide a sample size estimate that guarantees a prespecified power of a hypothesis test at a certain significance level or controls the width of a confidence interval with a certain confidence level. Our empirical results confirm that these score methods are reliable in terms of true size, coverage probability, and skewness. A liver scan detection study is used to illustrate the proposed methods.     

Sample size for individually matched case-control studies

The standard formulas used to calculate sample size for an individually matched case-control study assume a constant probability of exposure throughout the pool of possible controls. We propose new formulas that allow for heterogeneity in the probability of exposure among controls in different matched sets. Since matching factors are suspected of being confounders, they are expected to divide the total population into subgroups with different proportions exposed. Thus, the assumption of homogeneity of exposure among controls, made by the currently used formulas, is inconsistent with the assumptions used to design a matched study. The proposed formulas avoid this inconsistency. We present an example to illustrate how heterogeneity can affect the required sample size.