Incomplete Split-Block Designs

The paper deals with modelling of experiments carried out in incomplete split-block designs. Moreover, the characterization of the design with respect to a general balance property is given. Also, some statistical properties of the design are examined. The problems considered are illustrated with examples.     

FACTORS INFLUENCING THE EFFICIENCY OF INCOMPLETE BLOCK DESIGNS IN SENSORY EVALUATION EXPERIMENTS

The efficiencies of incomplete block designs were investigated by comparing two hundred and twenty eight analyses from eleven trials using hedonic scales with corresponding randomized complete block analyses. Of the ten explanatory factors examined, only the panelist, the product type, the number of samples per session and the average score of the data had an effect on the efficiency of incomplete block designs. The effect of product type was attributed to influences of produce consumed outside the trial, and the effect of the data mean reflected decreased conscientiousness with products the panelists disliked. With three and four samples per session, incomplete block designs were 31 % and 2 % more efficient, respectively, than randomized complete block designs. When five or more samples were tested, the incomplete block designs were markedly less efficient. The practical implications of all these effects on experimental design are discussed.     

A comparison of experimental designs for selection in breeding trials with nested treatment structure

Plant breeders frequently evaluate large numbers of entries in field trials for selection. Generally, the tested entries are related by pedigree. The simplest case is a nested treatment structure, where entries fall into groups or families such that entries within groups are more closely related than between groups. We found that some plant breeders prefer to plant close relatives next to each other in the field. This contrasts with common experimental designs such as the α-design, where entries are fully randomized. A third design option is to randomize in such a way that entries of the same group are separated as much as possible. The present paper compares these design options by simulation. Another important consideration is the type of model used for analysis. Most of the common experimental designs were optimized assuming that the model used for analysis has fixed treatment effects. With many entries that are related by pedigree, analysis based on a model with random treatment effects becomes a competitive alternative. In simulations, we therefore study the properties of best linear unbiased predictions (BLUP) of genetic effects based on a nested treatment structure under these design options for a range of genetic parameters. It is concluded that BLUP provides efficient estimates of genetic effects and that resolvable incomplete block designs such as the α-design with restricted or unrestricted randomization can be recommended.     

Scheffy's Confidence Intervals in the Models of Analysis of Covariance

Scheffe's confidence intervals for linear functions of some subvectors of a vector of parameters are presented. The considered subvectors are such that covariance matrices of their estimators are known non-negative definite matrices multiplied by unknown positive constants. This property is characteristic of the least squares estimators of vectors of main and interaction effects in the analysis of covariance models of the following experimental designs: split-block, split-plot, completely randomized two-factor design and randomized complete block design. The formulas for confidence intervals for linear functions of vectors of main or interaction effects in the designs mentioned above are given in the paper. The practical example is given as an illustration.     

Nonparametric Methods in Crossover Trials

The crossover design is often used in biomedical trials since it eliminates between subject variability. This paper is concerned with the statistical analysis of data arising from such trials when assumptions like normality do not necessarily apply. Nonparametric analysis of the two-period, two-treatment design was first described by Koch in a paper 1972. The purpose of this paper is to study nonparametric methods in crossover designs with three or more treatments and an equal number of periods. The proposed test for direct treatment effects is based on within subject comparisons after removing a possible period effect. With only two treatments this test reduces to the twosided Wilcoxon signed rank test. By simulation experiments the validity of the significance level of the test when using the asymptotic distribution of the test statistic are manifested and the power against different alternatives illustrated. A test for first order carryover effects can be constructed by a straightforward generalization of the test proposed by Koch in 1972. However, since this test is based on between subject comparisons its power will be low. Our recommendation is to consider the crossover design rather than the parallel group design if the carryover effects are assumed to be neglible or positive and smaller then the direct treatment effects.     

On the PBB Designs for Multiresponse Experiments

We present the idea of using multiresponse incomplete block designs when not all responses can be observed in all experimental units. For a special class of such designs, in which partial designs are PBB designs, a method for estimating natural treatment contrast is given. We also consider the problem of testing the hypotheses concerning the natural and any estimable treatment contrasts. For testing this hypothesis the Wald statistics, being asymptotically chi-square distributed, is proposed.     

A Simple Procedure for Constructing Experiment Designs with Incomplete Blocks of Sizes 2 and 3

In order to maximize control of heterogeneity within complete blocks, an experimenter could use incomplete blocks of size k = 2 or 3. In certain situations, incomplete blocks of this nature would eliminate the need for such spatial types of analyses as nearest neighbor. The intrablock efficiency factors for such designs are relatively low. However, with recovery of interblock information, FEDERER and SPEED (1987) have presented measures of design efficiency factors which demonstrate that efficiency factors approach unity for certain ratios of the intrablock and interblock variance components. Hence with recovery of interblock information, even incomplete block designs with k = 2 or 3 have relatively high efficiency factors. The reduction in the intrablock error variance over the complete block error variance in many situations will provide designs with high efficiency. A simple procedure for constructing incomplete blocks of sizes 2 and 3 is presented. It is shown how to obtain additional zero-one association confounding arrangements when v = 4 t, t an integer, and for v = pk, k ≤ p. It is indicated how to do the statistical analysis for these designs.     

The devil lies in details: reply to Stuart Hurlbert

As pointed out in Stuart Hurlbert's recent article, ecologists still at times design their experiments sloppily, creating a situation where various forms of ‘non‐demonic intrusion’ could account for the documented contrasts between treatments and controls. If such contrasts are nevertheless presented to the reader as if they were statistically demonstrated treatment effects, then pseudoreplication is not a pseudoissue and the use of a stigmatizing label of is entirely warranted, as pointed out by Hurlbert. The problems with Hurlbert's concepts start in the context of studies, where the scope of the experiment is to provoke a chain of dramatic and a priori extremely unlikely events, which a given conjecture predicts to happen as a consequence of a given manipulation. As the essence of these experiments is to trigger large dynamical responses in a biological system, they often require much space and/or special conditions, allowing for efficient isolation of the experimental system from potential sources of contamination. These constraints can be incompatible with standard designs (randomization, replication and treatment‐control interspersion). In the context of experiments, where it has been necessary to sacrifice randomization, replication or treatment‐control interspersion, the logic of inferring treatment effects is the same as used when interpreting causes of spontaneous events or events triggered by manipulations with practical purposes. The observed contrasts can be reasonably interpreted as effects of the treatment if and only if their magnitudes and the timing of their emergence makes alternative explanations utterly implausible (which is up to the reader to judge). If the logic of inference is clearly explained and no claim of statistically demonstrated treatment effect is made, the use of stigmatizing labels like ‘pseudoreplication’ is unwarranted. However, it might clarify the literature if such imperfectly designed experiments are referred to as experimental events, to be distinguished from perfectly designed experiments, where mechanical interpretation of contrasts between treatments and controls as treatment effects can be regarded as socially acceptable.     

Design and Analysis of Crossover Trials for Absorbing Binary Endpoints

Summary The crossover is a popular and efficient trial design used in the context of patient heterogeneity to assess the effect of treatments that act relatively quickly and whose benefit disappears with discontinuation. Each patient can serve as her own control as within‐individual treatment and placebo responses are compared. Conventional wisdom is that these designs are not appropriate for absorbing binary endpoints, such as death or HIV infection. We explore the use of crossover designs in the context of these absorbing binary endpoints and show that they can be more efficient than the standard parallel group design when there is heterogeneity in individuals' risks. We also introduce a new two‐period design where first period “survivors” are rerandomized for the second period. This design combines the crossover design with the parallel design and achieves some of the efficiency advantages of the crossover design while ensuring that the second period groups are comparable by randomization. We discuss the validity of the new designs and evaluate both a mixture model and a modified Mantel–Haenszel test for inference. The mixture model assumes no carryover or period effects while the Mantel–Haenszel approach conditions out period effects. Simulations are used to compare the different designs and an example is provided to explore practical issues in implementation.     

Fractional simplex designs for interaction screening in complex mixtures

In mixture experiments, one may be interested in estimating not only main effects but also some interactions. Main effects and significant interactions in a mixture may be estimated through appropriate mixture experiments, such as simplex-centroid designs. However, for mixtures with a large number of factors, the run size for these designs becomes impractically large. A subset of a full simplex-centroid design may be used, but the problem remains regarding which factor-level settings should be selected. In this paper, we propose a solution that considers design points with either one or p individual nonzero factor-level settings. These fractional simplex designs provide a means of screening for interactions and of investigating the behavior of many-component mixtures as a whole while greatly reducing the run size compared with full simplex-centroid designs. The means of construction of the design arrays is described, and designs for < or = 31 factors are presented. Some of the proposed methodology is illustrated using generated data.     

Optimum experimental design for Free‐Air Carbon dioxide Enrichment (FACE) studies

This article presents the logical reasoning underlying the optimal design of an experiment. We used Free‐Air Carbon dioxide Enrichment (FACE) experiments to illustrate this trade‐off as such experiments are particularly costly. On a theoretical basis, two‐way nested designs and split‐plot designs have similar power in testing carbon dioxide (CO₂) main effects. If researchers have the choice of adding two replicate rings or two control plots to their experiment, our results show that both options provide a substantial gain in statistical power, with a slightly greater gain in the former case and at reduced financial cost in the latter. The former option, however, provides an insurance against possible ring failure. On an empirical basis, we analysed a preliminary FACE photosynthesis dataset collected at Duke University. The experiment was designed as a split‐plot design to test the effects of growth environment (GROWTH) and measurement CO₂ concentration (MEAS) on photosynthetic rates of loblolly pine. Although a significant effect of MEAS was observed, we failed to detect a significant main effect of GROWTH. Power analysis was used to understand why the GROWTH main effect was not significant. The minimum detectable difference between treatment means that we calculated for GROWTH in this experiment was 4.04 μmol CO₂ m^?2 s^?1 for a statistical power of 0.90, whereas the observed difference was 0.16 μmol CO₂ m^?2 s^?1. Our recommendations for the design of FACE experiments are: (i) consider a second treatment factor with many levels within each ring in order to obtain a split‐plot design that provides a powerful test of interaction between treatment factors; (ii) add control plots, unless financial constrictions disallow for necessary personnel; (3) pool the data of FACE experiments conducted in comparable ecosystems (e.g. forests or grasslands), with two rings per treatment level at each site.     

A note on P-values under group sequential testing and nonproportional hazards

Summary .   Group sequential designs are often used for periodically assessing treatment efficacy during the course of a clinical trial. Following a group sequential test, P -values computed under the assumption that the data were gathered according to a fixed sample design are no longer uniformly distributed under the null hypothesis of no treatment effect. Various sample space orderings have been proposed for computing proper P -values following a group sequential test. Although many of the proposed orderings have been compared in the setting of time-invariant treatment effects, little attention has been given to their performance when the effect of treatment within an individual varies over time. Our interest here is to compare two of the most commonly used methods for computing proper P -values following a group sequential test, based upon the analysis time (AT) and Z -statistic orderings, with respect to resulting power functions when treatment effects on survival are delayed. Power under the AT ordering is shown to be heavily influenced by the presence of a delayed treatment effect, while power functions corresponding to the Z -statistic ordering remain robust under time-varying treatment effects.     

On the Role of Baseline Measurements for Crossover Designs under the Self and Mixed Carryover Effects Model

Summary .  It is well known that optimal designs are strongly model dependent. In this article, we apply the Lagrange multiplier approach to the optimal design problem, using a recently proposed model for carryover effects. Generally, crossover designs are not recommended when carryover effects are present and when the primary goal is to obtain an unbiased estimate of the treatment effect. In some cases, baseline measurements are believed to improve design efficiency. This article examines the impact of baselines on optimal designs using two different assumptions about carryover effects during baseline periods and employing a nontraditional crossover design model. As anticipated, baseline observations improve design efficiency considerably for two-period designs, which use the data in the first period only to obtain unbiased estimates of treatment effects, while the improvement is rather modest for three- or four-period designs. Further, we find little additional benefits for measuring baselines at each treatment period as compared to measuring baselines only in the first period. Although our study of baselines did not change the results on optimal designs that are reported in the literature, the problem of strong model dependency problem is generally recognized. The advantage of using multiperiod designs is rather evident, as we found that extending two-period designs to three- or four-period designs significantly reduced variability in estimating the direct treatment effect contrast.     

Highly efficient stepped wedge designs for clusters of unequal size

The stepped wedge design (SWD) is a form of cluster randomized trial, usually comparing two treatments, which is divided into time periods and sequences, with clusters allocated to sequences. Typically all sequences start with the standard treatment and end with the new treatment, with the change happening at different times in the different sequences. The clusters will usually differ in size but this is overlooked in much of the existing literature. This paper considers the case when clusters have different sizes and determines how efficient designs can be found. The approach uses an approximation to the variance of the treatment effect, which is expressed in terms of the proportions of clusters and of individuals allocated to each sequence of the design. The roles of these sets of proportions in determining an efficient design are discussed and illustrated using two SWDs, one in the treatment of sexually transmitted diseases and one in renal replacement therapy. Cluster-balanced designs, which allocate equal numbers of clusters to each sequence, are shown to have excellent statistical and practical properties; suggestions are made about the practical application of the results for these designs. The paper concentrates on the cross-sectional case, where subjects are measured once, but it is briefly indicated how the methods can be extended to the closed-cohort design.     

Using real-world data to predict health outcomes—The prediction design: Application and sample size planning

The gold standard for investigating the efficacy of a new therapy is a (pragmatic) randomized controlled trial (RCT). This approach is costly, time-consuming, and not always practicable. At the same time, huge quantities of available patient-level control condition data in analyzable format of (former) RCTs or real-world data (RWD) are neglected. Therefore, alternative study designs are desirable. The design presented here consists of setting up a prediction model for determining treatment effects under the control condition for future patients. When a new treatment is intended to be tested against a control treatment, a single-arm trial for the new therapy is conducted. The treatment effect is then evaluated by comparing the outcomes of the single-arm trial against the predicted outcomes under the control condition. While there are obvious advantages of this design compared to classical RCTs (increased efficiency, lower cost, alleviating participants’ fear of being on control treatment), there are several sources of bias. Our aim is to investigate whether and how such a design—the prediction design—may be used to provide information on treatment effects by leveraging external data sources. For this purpose, we investigated under what assumptions linear prediction models could be used to predict the counterfactual of patients precisely enough to construct a test and an appropriate sample size formula for evaluating the average treatment effect in the population of a new study. A user-friendly R Shiny application (available at: https://web.imbi.uni-heidelberg.de/PredictionDesignR/ ) facilitates the application of the proposed methods, while a real-world application example illustrates them.     

Analysis of Partially Balanced Incomplete Block Designs with Standards

The paper presents a detailed analysis of variance of PBIB designs supplemented by one or more standard treatments. Comparisons were carried out of treatment effects within the group of standard and test treatments and between the two groups. The analysis is supplemented by an example.     

Conditional Markov Chain Design for Accrual Clinical Trials

Randomization in a comparative experiment has, as one aim, the control of bias in the initial selection of experimental units. When the experiment is a clinical trial employing the accrual of patients, two additional aims are the control of admission bias and control of chronologic bias. This can be accomplished by using a method of randomization, such as the “biased coin design” of Efron, which sequentially forces balance. As an extension of Efron's design, this paper develops a class of conditional Markov chain designs. The detailed randomization employed utilizes the sequential imbalances in the treatment allocation as states in a Markov process. Through the use of appropriate transition probabilities, a range of possible designs can be attained. An additional objective of physical randomization is to provide a model for data analysis. Such a randomization theoretic analysis is presented for the current designs. In addition, Monte Carlo sampling results are given to support the proposed normal theory approximation to the exact randomization distribution.     

Which Is Better: Holdout or Full-Sample Classifier Design?

Is it better to design a classifier and estimate its error on the full sample or to design a classifier on a training subset and estimate its error on the holdout test subset? Full-sample design provides the better classifier; nevertheless, one might choose holdout with the hope of better error estimation. A conservative criterion to decide the best course is to aim at a classifier whose error is less than a given bound. Then the choice between full-sample and holdout designs depends on which possesses the smaller expected bound. Using this criterion, we examine the choice between holdout and several full-sample error estimators using covariance models and a patient-data model. Full-sample design consistently outperforms holdout design. The relation between the two designs is revealed via a decomposition of the expected bound into the sum of the expected true error and the expected conditional standard deviation of the true error.     

Efficient two-sample designs for microarray experiments with biological replications

In the last years, biostatistical research has begun to apply linear models and design theory to develop efficient experimental designs and analysis tools for gene expression microarray data. With two-colour microarrays, direct comparisons of RNA-targets are possible and lead to incomplete block designs. In this setting, efficient designs for simple and factorial microarray experiments have mainly been proposed for technical replicates. But for biological replicates, which are crucial to obtain inference that can be generalised to a biological population, this question has only been discussed recently and is not fully solved yet. In this paper, we propose efficient designs for independent two-sample experiments using two-colour microarrays enabling biologists to measure their biological random samples in an efficient manner to draw generalisable conclusions. We give advice for experimental situations with differing group sizes and show the impact of different designs on the variance and degrees of freedom of the test statistics. The designs proposed in this paper can be evaluated using SAS PROC MIXED or S+/R lme.     

Heterogeneous treatment effects in stratified clinical trials with time‐to‐event endpoints

When analyzing clinical trials with a stratified population, homogeneity of treatment effects is a common assumption in survival analysis. However, in the context of recent developments in clinical trial design, which aim to test multiple targeted therapies in corresponding subpopulations simultaneously, the assumption that there is no treatment‐by‐stratum interaction seems inappropriate. It becomes an issue if the expected sample size of the strata makes it unfeasible to analyze the trial arms individually. Alternatively, one might choose as primary aim to prove efficacy of the overall (targeted) treatment strategy. When testing for the overall treatment effect, a violation of the no‐interaction assumption renders it necessary to deviate from standard methods that rely on this assumption. We investigate the performance of different methods for sample size calculation and data analysis under heterogeneous treatment effects. The commonly used sample size formula by Schoenfeld is compared to another formula by Lachin and Foulkes, and to an extension of Schoenfeld's formula allowing for stratification. Beyond the widely used (stratified) Cox model, we explore the lognormal shared frailty model, and a two‐step analysis approach as potential alternatives that attempt to adjust for interstrata heterogeneity. We carry out a simulation study for a trial with three strata and violations of the no‐interaction assumption. The extension of Schoenfeld's formula to heterogeneous strata effects provides the most reliable sample size with respect to desired versus actual power. The two‐step analysis and frailty model prove to be more robust against loss of power caused by heterogeneous treatment effects than the stratified Cox model and should be preferred in such situations.