Domain selection and familywise error rate for functional data: A unified framework

Functional data are smooth, often continuous, random curves, which can be seen as an extreme case of multivariate data with infinite dimensionality. Just as componentwise inference for multivariate data naturally performs feature selection, subsetwise inference for functional data performs domain selection. In this paper, we present a unified testing framework for domain selection on populations of functional data. In detail, p-values of hypothesis tests performed on pointwise evaluations of functional data are suitably adjusted for providing control of the familywise error rate (FWER) over a family of subsets of the domain. We show that several state-of-the-art domain selection methods fit within this framework and differ from each other by the choice of the family over which the control of the FWER is provided. In the existing literature, these families are always defined a priori. In this work, we also propose a novel approach, coined thresholdwise testing, in which the family of subsets is instead built in a data-driven fashion. The method seamlessly generalizes to multidimensional domains in contrast to methods based on a priori defined families. We provide theoretical results with respect to consistency and control of the FWER for the methods within the unified framework. We illustrate the performance of the methods within the unified framework on simulated and real data examples and compare their performance with other existing methods.     

Correct and logical inference on efficacy in subgroups and their mixture for binary outcomes

Targeted therapies are becoming more common. In targeted therapy development, suppose its companion diagnostic test divides patients into a marker‐positive subgroup and its complementary marker‐negative subgroup. To find the right patient population for the therapy to target, inference on efficacy in the marker‐positive and marker‐negative subgroups as well as efficacy in the overall mixture population are all of interest. Depending on the type of clinical endpoints, inference on mixture population can be nontrivial and commonly used efficacy measures may not be suitable for a mixture population. Correlations among estimates of efficacy in the marker‐positive, marker‐negative, and overall mixture population play a crucial role in using an earlier phase study to inform on the design of a confirmatory study (e.g., determination of sample size). This article first shows that when the clinical endpoint is binary (such as respond or not), odds ratio is inappropriate as an efficacy measure in this setting, but relative response (RR) is appropriate. We show a safe way of calculating estimated correlations is to consider mixing subgroup response probabilities within each treatment arm first, and then derive the joint distribution of RR estimates. We also show, if one calculates RR within each subgroup first, how wrong the correlations can be if the Delta method derivation fails to take randomness of estimating the mixing coefficient into account.     

Bayesian treatment screening and selection using subgroup-specific utilities of response and toxicity

A Bayesian design is proposed for randomized phase II clinical trials that screen multiple experimental treatments compared to an active control based on ordinal categorical toxicity and response. The underlying model and design account for patient heterogeneity characterized by ordered prognostic subgroups. All decision criteria are subgroup specific, including interim rules for dropping unsafe or ineffective treatments, and criteria for selecting optimal treatments at the end of the trial. The design requires an elicited utility function of the two outcomes that varies with the subgroups. Final treatment selections are based on posterior mean utilities. The methodology is illustrated by a trial of targeted agents for metastatic renal cancer, which motivated the design methodology. In the context of this application, the design is evaluated by computer simulation, including comparison to three designs that conduct separate trials within subgroups, or conduct one trial while ignoring subgroups, or base treatment selection on estimated response rates while ignoring toxicity.     

k‐FWER Control without p ‐value Adjustment,with Application to Detection of Genetic Determinants of Multiple Sclerosis in Italian Twins

Summary We show a novel approach for k‐FWER control which does not involve any correction, but only testing the hypotheses along a (possibly data‐driven) order until a suitable number of p‐values are found above the uncorrected α level. p‐values can arise from any linear model in a parametric or nonparametric setting. The approach is not only very simple and computationally undemanding, but also the data‐driven order enhances power when the sample size is small (and also when k and/or the number of tests is large). We illustrate the method on an original study about gene discovery in multiple sclerosis, in which were involved a small number of couples of twins, discordant by disease. The methods are implemented in an R package (someKfwer ), freely available on CRAN.     

A Bayesian platform trial design to simultaneously evaluate multiple drugs in multiple indications with mixed endpoints

In the era of targeted therapies and immunotherapies, the traditional drug development paradigm of testing one drug at a time in one indication has become increasingly inefficient. Motivated by a real-world application, we propose a master-protocol–based Bayesian platform trial design with mixed endpoints (PDME) to simultaneously evaluate multiple drugs in multiple indications, where different subsets of efficacy measures (eg, objective response and landmark progression-free survival) may be used by different indications as single or multiple endpoints. We propose a Bayesian hierarchical model to accommodate mixed endpoints and reflect the trial structure of indications that are nested within treatments. We develop a two-stage approach that first clusters the indications into homogeneous subgroups and then applies the Bayesian hierarchical model to each subgroup to achieve precision information borrowing. Patients are enrolled in a group-sequential way and adaptively assigned to treatments according to their efficacy estimates. At each interim analysis, the posterior probabilities that the treatment effect exceeds prespecified clinically relevant thresholds are used to drop ineffective treatments and “graduate” effective treatments. Simulations show that the PDME design has desirable operating characteristics compared to existing method.     

Conditionally unbiased and near unbiased estimation of the selected treatment mean for multistage drop‐the‐losers trials

The two‐stage drop‐the‐loser design provides a framework for selecting the most promising of K experimental treatments in stage one, in order to test it against a control in a confirmatory analysis at stage two. The multistage drop‐the‐losers design is both a natural extension of the original two‐stage design, and a special case of the more general framework of Stallard & Friede ( 2008 ) (Stat. Med. 27 , 6209–6227). It may be a useful strategy if deselecting all but the best performing treatment after one interim analysis is thought to pose an unacceptable risk of dropping the truly best treatment. However, estimation has yet to be considered for this design. Building on the work of Cohen & Sackrowitz ( 1989 ) (Stat. Prob. Lett. 8 , 273–278), we derive unbiased and near‐unbiased estimates in the multistage setting. Complications caused by the multistage selection process are shown to hinder a simple identification of the multistage uniform minimum variance conditionally unbiased estimate (UMVCUE); two separate but related estimators are therefore proposed, each containing some of the UMVCUEs theoretical characteristics. For a specific example of a three‐stage drop‐the‐losers trial, we compare their performance against several alternative estimators in terms of bias, mean squared error, confidence interval width and coverage.     

Estimating Litter Decomposition Rate in Single-Pool Models Using Nonlinear Beta Regression

Litter decomposition rate (k) is typically estimated from proportional litter mass loss data using models that assume constant, normally distributed errors. However, such data often show non-normal errors with reduced variance near bounds (0 or 1), potentially leading to biased k estimates. We compared the performance of nonlinear regression using the beta distribution, which is well-suited to bounded data and this type of heteroscedasticity, to standard nonlinear regression (normal errors) on simulated and real litter decomposition data. Although the beta model often provided better fits to the simulated data (based on the corrected Akaike Information Criterion, AIC_c), standard nonlinear regression was robust to violation of homoscedasticity and gave equally or more accurate k estimates as nonlinear beta regression. Our simulation results also suggest that k estimates will be most accurate when study length captures mid to late stage decomposition (50–80% mass loss) and the number of measurements through time is ≥5. Regression method and data transformation choices had the smallest impact on k estimates during mid and late stage decomposition. Estimates of k were more variable among methods and generally less accurate during early and end stage decomposition. With real data, neither model was predominately best; in most cases the models were indistinguishable based on AIC_c, and gave similar k estimates. However, when decomposition rates were high, normal and beta model k estimates often diverged substantially. Therefore, we recommend a pragmatic approach where both models are compared and the best is selected for a given data set. Alternatively, both models may be used via model averaging to develop weighted parameter estimates. We provide code to perform nonlinear beta regression with freely available software.     

Do island plant populations really have lower genetic variation than mainland populations? Effects of selection and distribution range on genetic diversity estimates

Ecological and evolutionary studies largely assume that island populations display low levels of neutral genetic variation. However, this notion has only been formally tested in a few cases involving plant taxa, and the confounding effect of selection on genetic diversity (GD) estimates based on putatively neutral markers has typically been overlooked. Here, we generated nuclear microsatellite and plastid DNA sequence data in Periploca laevigata, a plant taxon with an island–mainland distribution area, to (i) investigate whether selection affects GD estimates of populations across contrasting habitats; and (ii) test the long‐standing idea that island populations have lower GD than their mainland counterparts. Plastid data showed that colonization of the Canary Islands promoted strong lineage divergence within P. laevigata, which was accompanied by selective sweeps at several nuclear microsatellite loci. Inclusion of loci affected by strong divergent selection produced a significant downward bias in the GD estimates of the mainland lineage, but such underestimates were substantial (>14%) only when more than one loci under selection were included in the computations. When loci affected by selection were removed, we did not find evidence that insular Periploca populations have less GD than their mainland counterparts. The analysis of data obtained from a comprehensive literature survey reinforced this result, as overall comparisons of GD estimates between island and mainland populations were not significant across plant taxa (N = 66), with the only exception of island endemics with narrow distributions. This study suggests that identification and removal of markers potentially affected by selection should be routinely implemented in estimates of GD, particularly if different lineages are compared. Furthermore, it provides compelling evidence that the expectation of low GD cannot be generalized to island plant populations.     

Adjusting for publication bias in meta-analysis via inverse probability weighting using clinical trial registries

Publication bias is a major concern in conducting systematic reviews and meta-analyses. Various sensitivity analysis or bias-correction methods have been developed based on selection models, and they have some advantages over the widely used trim-and-fill bias-correction method. However, likelihood methods based on selection models may have difficulty in obtaining precise estimates and reasonable confidence intervals, or require a rather complicated sensitivity analysis process. Herein, we develop a simple publication bias adjustment method by utilizing the information on conducted but still unpublished trials from clinical trial registries. We introduce an estimating equation for parameter estimation in the selection function by regarding the publication bias issue as a missing data problem under the missing not at random assumption. With the estimated selection function, we introduce the inverse probability weighting (IPW) method to estimate the overall mean across studies. Furthermore, the IPW versions of heterogeneity measures such as the between-study variance and the I² measure are proposed. We propose methods to construct confidence intervals based on asymptotic normal approximation as well as on parametric bootstrap. Through numerical experiments, we observed that the estimators successfully eliminated bias, and the confidence intervals had empirical coverage probabilities close to the nominal level. On the other hand, the confidence interval based on asymptotic normal approximation is much wider in some scenarios than the bootstrap confidence interval. Therefore, the latter is recommended for practical use.     

How to obtain a precise and representative estimate for parameters in LCA

Background, Aim and Scope  Quite often there is need for precise and representative parameters in LCA studies. Probably the most relevant have direct influence on the functional unit, whose definition is crucial in the conduct of any LCA. Changes in the functional unit show directly in LCI and LCIA results. In comparative assertions, a bias in the functional unit may lead to a bias in the overall conclusions. Since quantitative data for the functional unit, such as geometric dimensions and specific weight, often vary, the question arises how to determine the functional unit, especially if a comparative assertion shall be representative for a region or market. Aim and scope of the study is to develop and apply methods for obtaining precise and representative estimates for the functional unit as one important parameter in an LCA study. Materials and Methods  Statistical sampling is applied in order to get empirical estimates for the weight of yoghurt cups, as a typical parameter for the functional unit. We used a two-stage sampling design, with stratified sampling in the first stage and three different sampling designs in the second stage, namely stratified, clustered, and a posteriori sampling. Sampling designs are motivated and described. In a case study, they are each used to determined a representative weight for 150 g yoghurt cups in Berlin, at the point of sale and within a specific time. In the first sampling stage, food markets are randomly selected, while in the second stage, yoghurt cups in these food markets are sampled. The sampling methods are applicable due to newly available internet data. These data sources and their shortcomings are described. Results  The random sampling procedure yields representative estimates, which are compared to figures for market leaders, i.e. yoghurt cups with very high occurrence in the supermarkets. While single types of yoghurt cups showed moderate uncertainty, representative estimates were highly precise. Discussion results show, for one, the performance of the applied statistical estimation procedures, and they show further that adding more information in the estimation procedure (on the shape of the cup, on the type of plastic, on the specific brand) helps reducing uncertainty. Conclusions  As conclusions, estimates and their uncertainty depend on the measurement procedure in a sensitive manner; any uncertainty information should be coupled with information on the measurement procedure, and it is recommended to use statistical sampling in order to reduce uncertainty for important parameters of an LCA study. Recommendations and Perspectives  Results for market leaders differed considerably from representative estimates. This implies to not use market leader data, or data with a high market share, as substitute for representative data in LCA studies. Statistical sampling has been barely used for Life Cycle Assessment. It turned out to be a feasible means for obtaining highly precise and representative estimates for the weight of yoghurt cups in the case study, based on empirical analysis. Further research is recommended in order to detect which parameters should best be investigated in LCA case studies; which data sources are available and recommended, and which sampling designs are appropriate for different application cases. ESS-Submission Editor: Seungdo Kim. PhD (kimseun@msu.edu)     

Evolution of the number of LRRs in plant disease resistance genes

The largest group of plant resistance (R) genes contain the regions that encode the nucleotide-binding site (NBS) and leucine-rich repeat (LRR) domains (NBS-LRR genes). To gain new resistance, amino acid substitutions and changes in number of the LRRs that recognize the presence of pathogens are considered important. In this study, we focus on the evolution of the number of LRRs and analyze the genome data of five plant species, Arabidopsis thaliana, Oryza sativa, Medicago truncatula, Lotus japonicus and Populus trichocarpa. We first categorized the NBS-LRR genes in each species into groups and subgroups based on the phylogenetic relationships of their NBS domain sequences. Then we estimated the evolutionary rate of the number of LRRs relative to the synonymous divergence in the NBS domain sequences by a maximum likelihood method assuming the single stepwise mutation model. The estimates ranged from 4.5 to 600 and differed between groups in the same species or between species. This indicated different roles played by different groups of the NBS-LRR genes within a species or the effects of various life history characteristics, such as generation time, of the species. We also tested the fit of the model to the data using the variance of number of LRRs in each subgroup. In some subgroups in some plants (16 out of 174 subgroups), the results of simulation using the estimated rates significantly deviated from the observed data. Those subgroups may have undergone different modes of selection from the other subgroups.     

Interim analysis incorporating short‐ and long‐term binary endpoints

Designs incorporating more than one endpoint have become popular in drug development. One of such designs allows for incorporation of short‐term information in an interim analysis if the long‐term primary endpoint has not been yet observed for some of the patients. At first we consider a two‐stage design with binary endpoints allowing for futility stopping only based on conditional power under both fixed and observed effects. Design characteristics of three estimators: using primary long‐term endpoint only, short‐term endpoint only, and combining data from both are compared. For each approach, equivalent cut‐off point values for fixed and observed effect conditional power calculations can be derived resulting in the same overall power. While in trials stopping for futility the type I error rate cannot get inflated (it usually decreases), there is loss of power. In this study, we consider different scenarios, including different thresholds for conditional power, different amount of information available at the interim, different correlations and probabilities of success. We further extend the methods to adaptive designs with unblinded sample size reassessments based on conditional power with inverse normal method as the combination function. Two different futility stopping rules are considered: one based on the conditional power, and one from P‐values based on Z‐statistics of the estimators. Average sample size, probability to stop for futility and overall power of the trial are compared and the influence of the choice of weights is investigated.     

Blinded sample size recalculation in adaptive enrichment designs

In the precision medicine era, (prespecified) subgroup analyses are an integral part of clinical trials. Incorporating multiple populations and hypotheses in the design and analysis plan, adaptive designs promise flexibility and efficiency in such trials. Adaptations include (unblinded) interim analyses (IAs) or blinded sample size reviews. An IA offers the possibility to select promising subgroups and reallocate sample size in further stages. Trials with these features are known as adaptive enrichment designs. Such complex designs comprise many nuisance parameters, such as prevalences of the subgroups and variances of the outcomes in the subgroups. Additionally, a number of design options including the timepoint of the sample size review and timepoint of the IA have to be selected. Here, for normally distributed endpoints, we propose a strategy combining blinded sample size recalculation and adaptive enrichment at an IA, that is, at an early timepoint nuisance parameters are reestimated and the sample size is adjusted while subgroup selection and enrichment is performed later. We discuss implications of different scenarios concerning the variances as well as the timepoints of blinded review and IA and investigate the design characteristics in simulations. The proposed method maintains the desired power if planning assumptions were inaccurate and reduces the sample size and variability of the final sample size when an enrichment is performed. Having two separate timepoints for blinded sample size review and IA improves the timing of the latter and increases the probability to correctly enrich a subgroup.     

The patterns of population differentiation in a <Emphasis Type="Italic">Brassica rapa</Emphasis> core collection

With the recent advances in high throughput profiling techniques the amount of genetic and phenotypic data available has increased dramatically. Although many genetic diversity studies combine morphological and genetic data, metabolite profiling has yet to be integrated into these studies. For our study we selected 168 accessions representing the different morphotypes and geographic origins of Brassica rapa. Metabolite profiling was performed on all plants of this collection in the youngest expanded leaves, 5 weeks after transplanting and the same material was used for molecular marker profiling. During the same season a year later, 26 morphological characteristics were measured on plants that had been vernalized in the seedling stage. The number of groups and composition following a hierarchical clustering with molecular markers was highly correlated to the groups based on morphological traits (r = 0.420) and metabolic profiles (r = 0.476). To reveal the admixture levels in B. rapa, comparison with the results of the programme STRUCTURE was needed to obtain information on population substructure. To analyze 5546 metabolite (LC–MS) signals the groups identified with STRUCTURE were used for random forests classification. When comparing the random forests and STRUCTURE membership probabilities 86% of the accessions were allocated into the same subgroup. Our findings indicate that if extensive phenotypic data (metabolites) are available, classification based on this type of data is very comparable to genetic classification. These multivariate types of data and methodological approaches are valuable for the selection of accessions to study the genetics of selected traits and for genetic improvement programs, and additionally provide information on the evolution of the different morphotypes in B. rapa.     

Testing a Primary and a Secondary Endpoint in a Group Sequential Design

Summary We consider a clinical trial with a primary and a secondary endpoint where the secondary endpoint is tested only if the primary endpoint is significant. The trial uses a group sequential procedure with two stages. The familywise error rate (FWER) of falsely concluding significance on either endpoint is to be controlled at a nominal level α. The type I error rate for the primary endpoint is controlled by choosing any α‐level stopping boundary, e.g., the standard O'Brien–Fleming or the Pocock boundary. Given any particular α‐level boundary for the primary endpoint, we study the problem of determining the boundary for the secondary endpoint to control the FWER. We study this FWER analytically and numerically and find that it is maximized when the correlation coefficient ρ between the two endpoints equals 1. For the four combinations consisting of O'Brien–Fleming and Pocock boundaries for the primary and secondary endpoints, the critical constants required to control the FWER are computed for different values of ρ. An ad hoc boundary is proposed for the secondary endpoint to address a practical concern that may be at issue in some applications. Numerical studies indicate that the O'Brien–Fleming boundary for the primary endpoint and the Pocock boundary for the secondary endpoint generally gives the best primary as well as secondary power performance. The Pocock boundary may be replaced by the ad hoc boundary for the secondary endpoint with a very little loss of secondary power if the practical concern is at issue. A clinical trial example is given to illustrate the methods.     

A Bootstrap Procedure for Adaptive Selection of the Test Statistic in Flexible Two‐Stage Designs

Adaptive two‐stage designs allow a data‐driven change of design characteristics during the ongoing trial. One of the available options is an adaptive choice of the test statistic for the second stage of the trial based on the results of the interim analysis. Since there is often only a vague knowledge of the distribution shape of the primary endpoint in the planning phase of a study, a change of the test statistic may then be considered if the data indicate that the assumptions underlying the initial choice of the test are not correct. Collings and Hamilton proposed a bootstrap method for the estimation of the power of the two‐sample Wilcoxon test for shift alternatives. We use this approach for the selection of the test statistic. By means of a simulation study, we show that the gain in terms of power may be considerable when the initial assumption about the underlying distribution was wrong, whereas the loss is relatively small when in the first instance the optimal test statistic was chosen. The results also hold true for comparison with a one‐stage design. Application of the method is illustrated by a clinical trial example.     

Exploratory subgroup analysis in clinical trials by model selection

The interest in individualized medicines and upcoming or renewed regulatory requests to assess treatment effects in subgroups of confirmatory trials requires statistical methods that account for selection uncertainty and selection bias after having performed the search for meaningful subgroups. The challenge is to judge the strength of the apparent findings after mining the same data to discover them. In this paper, we describe a resampling approach that allows to replicate the subgroup finding process many times. The replicates are used to adjust the effect estimates for selection bias and to provide variance estimators that account for selection uncertainty. A simulation study provides some evidence of the performance of the method and an example from oncology illustrates its use.     

Multiple hypothesis testing to detect lineages under positive selection that affects only a few sites

Detection of positive Darwinian selection has become ever more important with the rapid growth of genomic data sets. Recent branch-site models of codon substitution account for variation of selective pressure over branches on the tree and across sites in the sequence and provide a means to detect short episodes of molecular adaptation affecting just a few sites. In likelihood ratio tests based on such models, the branches to be tested for positive selection have to be specified a priori. In the absence of a biological hypothesis to designate so-called foreground branches, one may test many branches, but a correction for multiple testing becomes necessary. In this paper, we employ computer simulation to evaluate the performance of 6 multiple test correction procedures when the branch-site models are used to test every branch on the phylogeny for positive selection. Four of the methods control the familywise error rates (FWERs), whereas the other 2 control the false discovery rate (FDR). We found that all correction procedures achieved acceptable FWER except for extremely divergent sequences and serious model violations, when the test may become unreliable. The power of the test to detect positive selection is influenced by the strength of selection and the sequence divergence, with the highest power observed at intermediate divergences. The 4 correction procedures that control the FWER had similar power. We recommend Rom's procedure for its slightly higher power, but the simple Bonferroni correction is useable as well. The 2 correction procedures that control the FDR had slightly more power and also higher FWER. We demonstrate the multiple test procedures by analyzing gene sequences from the extracellular domain of the cluster of differentiation 2 (CD2) gene from 10 mammalian species. Both our simulation and real data analysis suggest that the multiple test procedures are useful when multiple branches have to be tested on the same data set.     

Information fraction estimation based on the number of events within the standard treatment regimen

For a Phase III randomized trial that compares survival outcomes between an experimental treatment versus a standard therapy, interim monitoring analysis is used to potentially terminate the study early based on efficacy. To preserve the nominal Type I error rate, alpha spending methods and information fractions are used to compute appropriate rejection boundaries in studies with planned interim analyses. For a one-sided trial design applied to a scenario in which the experimental therapy is superior to the standard therapy, interim monitoring should provide the opportunity to stop the trial prior to full follow-up and conclude that the experimental therapy is superior. This paper proposes a method called total control only (TCO) for estimating the information fraction based on the number of events within the standard treatment regimen. Based on theoretical derivations and simulation studies, for a maximum duration superiority design, the TCO method is not influenced by departure from the designed hazard ratio, is sensitive to detecting treatment differences, and preserves the Type I error rate compared to information fraction estimation methods that are based on total observed events. The TCO method is simple to apply, provides unbiased estimates of the information fraction, and does not rely on statistical assumptions that are impossible to verify at the design stage. For these reasons, the TCO method is a good approach when designing a maximum duration superiority trial with planned interim monitoring analyses.     

Adjustment for exploratory cut-off selection in randomized clinical trials with survival endpoint

Defining the target population based on predictive biomarkers plays an important role during clinical development. After establishing a relationship between a biomarker candidate and response to treatment in exploratory phases, a subsequent confirmatory trial ideally involves only subjects with high potential of benefiting from the new compound. In order to identify those subjects in case of a continuous biomarker, a cut-off is needed. Usually, a cut-off is chosen that resulted in a subgroup with a large observed treatment effect in an exploratory trial. However, such a data-driven selection may lead to overoptimistic expectations for the subsequent confirmatory trial. Treatment effect estimates, probability of success, and posterior probabilities are useful measures for deciding whether or not to conduct a confirmatory trial enrolling the biomarker-defined population. These measures need to be adjusted for selection bias. We extend previously introduced Approximate Bayesian Computation techniques for adjustment of subgroup selection bias to a time-to-event setting with cut-off selection. Challenges in this setting are that treatment effects become time-dependent and that subsets are defined by the biomarker distribution. Simulation studies show that the proposed method provides adjusted statistical measures which are superior to naïve Maximum Likelihood estimators as well as simple shrinkage estimators.