Biomarker-Defined Subgroup Selection Adaptive Design for Phase III Confirmatory Trial with Time-to-Event Data: Comparing Group Sequential and Various Adaptive Enrichment Designs

Predictive and prognostic biomarkers play an important role in personalized medicine to determine strategies for drug evaluation and treatment selection. In the context of continuous biomarkers, identification of an optimal cutoff for patient selection can be challenging due to limited information on biomarker predictive value, the biomarker’s distribution in the intended use population, and the complexity of the biomarker relationship to clinical outcomes. As a result, prespecified candidate cutoffs may be rationalized based on biological and practical considerations. In this context, adaptive enrichment designs have been proposed with interim decision rules to select a biomarker-defined subpopulation to optimize study performance. With a group sequential design as a reference, the performance of several proposed adaptive designs are evaluated and compared under various scenarios (e.g., sample size, study power, enrichment effects) where type I error rates are well controlled through closed testing procedures and where subpopulation selections are based upon the predictive probability of trial success. It is found that when the treatment is more effective in a subpopulation, these adaptive designs can improve study power substantially. Furthermore, we identified one adaptive design to have generally higher study power than the other designs under various scenarios.     

Inference on Multiple Endpoints in Clinical Trials with Adaptive Interim Analyses

Planned interim analyses which permit early stopping or sample size adaption of a trial are desirable for ethical and scientific reasons. Multiple test procedures allow inference about several hypotheses within a single clinical trial. In this paper, a method which combines multiple testing with adaptive interim analyses whilst controlling the experimentwise error rate is proposed. The general closed testing principle, the situation of a priori ordered hypotheses, and application of the Bonferroni-Holm method are considered. The practical application of the method is demonstrated by an example.     

Subgroup-Based Adaptive (SUBA) Designs for Multi-arm Biomarker Trials

Targeted therapies based on biomarker profiling are becoming a mainstream direction of cancer research and treatment. Depending on the expression of specific prognostic biomarkers, targeted therapies assign different cancer drugs to subgroups of patients even if they are diagnosed with the same type of cancer by traditional means, such as tumor location. For example, Herceptin is only indicated for the subgroup of patients with HER2+ breast cancer, but not other types of breast cancer. However, subgroups like HER2+ breast cancer with effective targeted therapies are rare, and most cancer drugs are still being applied to large patient populations that include many patients who might not respond or benefit. Also, the response to targeted agents in humans is usually unpredictable. To address these issues, we propose subgroup-based adaptive (SUBA), designs that simultaneously search for prognostic subgroups and allocate patients adaptively to the best subgroup-specific treatments throughout the course of the trial. The main features of SUBA include the continuous reclassification of patient subgroups based on a random partition model and the adaptive allocation of patients to the best treatment arm based on posterior predictive probabilities. We compare the SUBA design with three alternative designs including equal randomization, outcome-adaptive randomization, and a design based on a probit regression. In simulation studies, we find that SUBA compares favorably against the alternatives.     

Interphase Clinical Trials: Phase III/Phase IV

In this paper we investigate ways in which the results of a controlled Phase III clinical trial can be used in subsequent Phase IV, and possibly further Phase III studies. Specifically we are interested in; 1) developing particular hypothesis relating to a modified study population, 2) studying how changes in the particularities of the Phase III study group can influence certain outcome variables of interest and 3) using the results of the Phase III study applied to specific target groups, having particular characteristics, to updating observations from the Phase III study with information obtained at a later stage. These issues are all concerned with the way in which we can exploit information from a Phase III trial, information that is of high quality but not necessarily directly related to the way in which many post Phase III studies, focussing on different patient population groups, are carried out. Since it is often these post Phase III studies that have the strongest influence on clinical practice we aim to develop a framework around which the post Phase III studies might be structured.     

Bayesian Adaptive Randomization and Trial Monitoring with Predictive Probability for Time-to-Event Endpoint

There has been much development in Bayesian adaptive designs in clinical trials. In the Bayesian paradigm, the posterior predictive distribution characterizes the future possible outcomes given the currently observed data. Based on the interim time-to-event data, we develop a new phase II trial design by combining the strength of both Bayesian adaptive randomization and the predictive probability. By comparing the mean survival times between patients assigned to two treatment arms, more patients are assigned to the better treatment on the basis of adaptive randomization. We continuously monitor the trial using the predictive probability for early termination in the case of superiority or futility. We conduct extensive simulation studies to examine the operating characteristics of four designs: the proposed predictive probability adaptive randomization design, the predictive probability equal randomization design, the posterior probability adaptive randomization design, and the group sequential design. Adaptive randomization designs using predictive probability and posterior probability yield a longer overall median survival time than the group sequential design, but at the cost of a slightly larger sample size. The average sample size using the predictive probability method is generally smaller than that of the posterior probability design.     

Likelihood-Based Methods for Assessing Principal Surrogate Endpoints in Vaccine Trials

Statistics in Biosciences - When evaluating principal surrogate biomarkers in vaccine trials, missingness in potential outcomes requires prediction using auxiliary variables and/or augmented study...     

Treatment-Related Death in Patients with Small-Cell Lung Cancer in Phase III Trials over the Last Two Decades

Introduction

Treatment-related death (TRD) remains a serious problem in small-cell lung cancer (SCLC), despite recent improvements in supportive care. However, few studies have formally assessed time trends in the proportion of TRD over the past two decades. The aim of this study was to determine the frequency and pattern of TRD over time.

Methods

We examined phase 3 trials conducted between 1990 and 2010 to address the role of systemic treatment for SCLC. The time trend was assessed using linear regression analysis.

Results

In total, 97 trials including nearly 25,000 enrolled patients were analyzed. The overall TRD proportion was 2.95%. Regarding the time trend, while it was not statistically significant, it tended to decrease, with a 0.138% decrease per year and 2.76% decrease per two decades. The most common cause of death was febrile neutropenia without any significant time trend in its incidence over the years examined (p = 0.139). However, deaths due to febrile neutropenia as well as all causes in patients treated with non-platinum chemotherapy increased significantly (p = 0.033).

Conclusions

The overall TRD rate has been low, but not negligible, in phase III trials for SCLC over the past two decades.     

Analysis of Group Randomized Trials with Multiple Binary Endpoints and Small Number of Groups

The group randomized trial (GRT) is a common study design to assess the effect of an intervention program aimed at health promotion or disease prevention. In GRTs, groups rather than individuals are randomized into intervention or control arms. Then, responses are measured on individuals within those groups. A number of analytical problems beset GRT designs. The major problem emerges from the likely positive intraclass correlation among observations of individuals within a group. This paper provides an overview of the analytical method for GRT data and applies this method to a randomized cancer prevention trial, where multiple binary primary endpoints were obtained. We develop an index of extra variability to investigate group-specific effects on response. The purpose of the index is to understand the influence of individual groups on evaluating the intervention effect, especially, when a GRT study involves a small number of groups. The multiple endpoints from the GRT design are analyzed using a generalized linear mixed model and the stepdown Bonferroni method of Holm.     

A New Overall-Subgroup Simultaneous Test for Optimal Inference in Biomarker-Targeted Confirmatory Trials

We propose a joint hypothesis test for simultaneous confirmatory inference in the overall population and a pre-defined marker-positive subgroup under the assumption that the treatment effect in the marker-positive subgroup is larger than that in the overall population. The proposed confirmatory overall-subgroup simultaneous test (COSST) is based on partitioning the sample space of the test statistics in the marker-positive and marker-negative subgroups. We define two rejection regions in the joint sample space of the two test statistics: (1) efficacy in the marker-positive subgroup only; (2) efficacy in the overall population. COSST achieves higher statistical power to detect the overall and subgroup efficacy than most sequential procedures while controlling the family-wise type I error rate. COSST also takes into account the potentially harmful effect in the subgroups in the decision. The optimal rejection regions depend on the specific alternative hypothesis and the sample size. COSST can be useful for Phase III clinical trials with tailoring objectives.     

Twenty-Seven Years of Phase III Trials for Patients with Extensive Disease Small-Cell Lung Cancer: Disappointing Results

Background

Few studies have formally assessed whether treatment outcomes have improved substantially over the years for patients with extensive disease small-cell lung cancer (ED-SCLC) enrolled in phase III trials. The objective of the current investigation was to determine the time trends in outcomes for the patients in those trials.

Methods and Findings

We searched for trials that were reported between January 1981 and August 2008. Phase III randomized controlled trials were eligible if they compared first-line, systemic chemotherapy for ED-SCLC. Data were evaluated by using a linear regression analysis. Results: In total, 52 trials were identified that had been initiated between 1980 and 2006; these studies involved 10,262 patients with 110 chemotherapy arms. The number of randomized patients and the proportion of patients with good performance status (PS) increased over time. Cisplatin-based regimens, especially cisplatin and etoposide (PE) regimen, have increasingly been studied, whereas cyclophosphamide, doxorubicin, and vincristine–based regimens have been less investigated. Multiple regression analysis showed no significant improvement in survival over the years. Additionally, the use of a PE regimen did not affect survival, whereas the proportion of patients with good PS and the trial design of assigning prophylactic cranial irradiation were significantly associated with favorable outcome.

Conclusions and Significance

The survival of patients with ED-SCLC enrolled in phase III trials did not improve significantly over the years, suggesting the need for further development of novel targets, newer agents, and comprehensive patient care.     

Subcutaneous Progesterone Is Effective and Safe for Luteal Phase Support in IVF: An Individual Patient Data Meta-Analysis of the Phase III Trials

Objective

To summarize efficacy and safety data on a new progesterone compound which is available for subcutaneous administration as compared to vaginally administered progesterone for luteal phase support in patients undergoing IVF treatment.

Design

Data from two randomized phase III trials (07EU/Prg06 and 07USA/Prg05) performed according to GCP standards with a total sample size of 1435 per-protocol patients were meta-analyzed on an individual patient data level.

Setting

University affiliated reproductive medicine unit.

Patients

Subcutaneous progesterone was administered to a total of 714 subjects and vaginal progesterone was administered to a total of 721 subjects who underwent fresh embryo transfer after ovarian stimulation followed by IVF or ICSI. The subjects were between 18 and 42 years old and had a BMI <30kg/m².

Interventions

Subcutaneous progesterone 25 mg daily vs. either progesterone vaginal gel 90 mg daily (07EU/Prg06) or 100 mg intravaginal twice a day (07USA/Prg05) for luteal phase support in IVF patients.

Main outcome measures

Ongoing pregnancy rate beyond 10 gestational weeks, live birth rate and OHSS risk.

Results

The administration of subcutaneous progesterone versus intra-vaginal progesterone had no impact on ongoing pregnancy likelihood (OR = 0.865, 95% CI 0.694 to 1.077; P = n.s.), live birth likelihood (OR = 0.889, 95% CI 0.714 to 1.106; P = n.s.) or OHSS risk (OR = 0.995, 95% CI 0.565 to 1.754; P = n.s.) in regression analyses accounting for clustering of patients within trials, while adjusting for important confounders. Only female age and number of oocytes retrieved were significant predictors of live birth likelihood and OHSS risk.

Conclusion

No statistical significant or clinical significant differences exist between subcutaneous and vaginal progesterone for luteal phase support.     

Sample Size Reassessment and Hypothesis Testing in Adaptive Survival Trials

Mid-study design modifications are becoming increasingly accepted in confirmatory clinical trials, so long as appropriate methods are applied such that error rates are controlled. It is therefore unfortunate that the important case of time-to-event endpoints is not easily handled by the standard theory. We analyze current methods that allow design modifications to be based on the full interim data, i.e., not only the observed event times but also secondary endpoint and safety data from patients who are yet to have an event. We show that the final test statistic may ignore a substantial subset of the observed event times. An alternative test incorporating all event times is found, where a conservative assumption must be made in order to guarantee type I error control. We examine the power of this approach using the example of a clinical trial comparing two cancer therapies.     

Phthalates Biomarker Identification and Exposure Estimates in a Population of Pregnant Women

Phthalates are known reproductive and developmental toxicants in experimental animals. However, in humans, there are few data on the exposure of pregnant women that can be used to assess the potential developmental exposure experienced by the fetus. We measured several phthalate metabolites in maternal urine, maternal serum, and cord serum samples collected at the time of delivery from 150 pregnant women from central New Jersey. The urinary concentrations of most metabolites were comparable to or less than among the U.S. general population, except for mono(2-ethylhexyl) phthalate (MEHP), mono(2-ethyl-5-hydroxyhexyl) phthalate (MEHHP), and mono(2-ethyl-5-oxohexyl) phthalate (MEOHP), three metabolites of di(2-ethylhexyl) phthalate (DEHP). The median urinary concentrations of MEHHP (109 μ g/l) and MEOHP (95.1 μ g/l) were more than 5 times their population-based concentrations, whereas the median urinary concentration of MEHP was more than 20 times higher. High concentration of MEHP may indicate a recent exposure to the parent chemical DEHP in the hospital shortly before the collection of the samples. Calculation of daily intakes using the urinary biomarker data reveals that none of the pregnant women tested had integrated exposures to DEHP greater than the Agency for Toxic Substances and Disease Registry's minimal risk levels (MRLs chronic 60, intermediate 100 μ g/kg/day). No abnormal birth outcomes (e.g., birth weight, Apgar Score, and gestational age) were noted in those newborns whose mothers had relatively greater exposure to DEHP during the perinatal period than others in this study. Significantly greater concentrations and detection frequencies in maternal urine than in maternal serum and cord serum suggest that the urinary concentrations of the phthalate metabolites may be more reliable biomarkers of exposure than their concentrations in other biological specimens.     

Reconstructing Source-Sink Dynamics in a Population with a Pelagic Dispersal Phase

For many organisms, the reconstruction of source-sink dynamics is hampered by limited knowledge of the spatial assemblage of either the source or sink components or lack of information on the strength of the linkage for any source-sink pair. In the case of marine species with a pelagic dispersal phase, these problems may be mitigated through the use of particle drift simulations based on an ocean circulation model. However, when simulated particle trajectories do not intersect sampling sites, the corroboration of model drift simulations with field data is hampered. Here, we apply a new statistical approach for reconstructing source-sink dynamics that overcomes the aforementioned problems. Our research is motivated by the need for understanding observed changes in jellyfish distributions in the eastern Bering Sea since 1990. By contrasting the source-sink dynamics reconstructed with data from the pre-1990 period with that from the post-1990 period, it appears that changes in jellyfish distribution resulted from the combined effects of higher jellyfish productivity and longer dispersal of jellyfish resulting from a shift in the ocean circulation starting in 1991. A sensitivity analysis suggests that the source-sink reconstruction is robust to typical systematic and random errors in the ocean circulation model driving the particle drift simulations. The jellyfish analysis illustrates that new insights can be gained by studying structural changes in source-sink dynamics. The proposed approach is applicable for the spatial source-sink reconstruction of other species and even abiotic processes, such as sediment transport.     

Selection in Finite Populations with Multiple Alleles. III. Genetic Divergence with Centripetal Selection and Mutation

Natural selection for an intermediate level of gene or enzyme activity has been shown to lead to a high frequency of heterotic polymorphisms in populations subject to mutation and random genetic drift. The model assumes a symmetrical spectrum of mutational variation, with the majority of variants having only minor effects on the probability of survival. Each mutational event produces a variant which is novel to the population. Allelic effects are assumed to be additive on the scale of enzyme activity, heterosis arising whenever a heterozygote has a mean level of activity closer to optimal than that of other genotypes in the population.-A new measure of genetic divergence between populations is proposed, which is readily interpreted genetically, and increases approximately linearly with time under centripetal selection, drift and mutation. The parameter is closely related to the rate of accumulation of mutational changes in a cistron over an evolutionary time span.-A survey of published data concerning polymorphic loci in man and Drosophila suggests than an alternative model, based on the superiority of hybrid molecules, is not of general importance. Thirteen loci giving rise to hybrid zones on electrophoresis have a mean heterozygote frequency of 0.22 +/-.06, compared with a value of 0.23 +/-.04 for 16 loci classified as producing no hybrid enzyme.     

Confirmatory seamless phase II/III clinical trials with hypotheses selection at interim: opportunities and limitations

This is a discussion of the following two papers in this special issue on adaptive designs: 'Confirmatory seamless phase II/III clinical trials with hypotheses selection at interim: General concepts' by Frank Bretz, Heinz Schmidli, Franz K?nig, Amy Racine and Willi Maurer, and 'Confirmatory seamless phase II/III clinical trials with hypotheses selection at interim: Applications and practical considerations' by Heinz Schmidli, Frank Bretz, Amy Racine and Willi Maurer.     

A Critical Assessment of Feature Selection Methods for Biomarker Discovery in Clinical Proteomics

In this paper, we compare the performance of six different feature selection methods for LC-MS-based proteomics and metabolomics biomarker discovery—t test, the Mann–Whitney–Wilcoxon test (mww test), nearest shrunken centroid (NSC), linear support vector machine–recursive features elimination (SVM-RFE), principal component discriminant analysis (PCDA), and partial least squares discriminant analysis (PLSDA)—using human urine and porcine cerebrospinal fluid samples that were spiked with a range of peptides at different concentration levels. The ideal feature selection method should select the complete list of discriminating features that are related to the spiked peptides without selecting unrelated features. Whereas many studies have to rely on classification error to judge the reliability of the selected biomarker candidates, we assessed the accuracy of selection directly from the list of spiked peptides. The feature selection methods were applied to data sets with different sample sizes and extents of sample class separation determined by the concentration level of spiked compounds. For each feature selection method and data set, the performance for selecting a set of features related to spiked compounds was assessed using the harmonic mean of the recall and the precision (f-score) and the geometric mean of the recall and the true negative rate (g-score). We conclude that the univariate t test and the mww test with multiple testing corrections are not applicable to data sets with small sample sizes (n = 6), but their performance improves markedly with increasing sample size up to a point (n > 12) at which they outperform the other methods. PCDA and PLSDA select small feature sets with high precision but miss many true positive features related to the spiked peptides. NSC strikes a reasonable compromise between recall and precision for all data sets independent of spiking level and number of samples. Linear SVM-RFE performs poorly for selecting features related to the spiked compounds, even though the classification error is relatively low.Biomarkers play an important role in advancing medical research through the early diagnosis of disease and prognosis of treatment interventions (1, 2). Biomarkers may be proteins, peptides, or metabolites, as well as mRNAs or other kinds of nucleic acids (e.g. microRNAs) whose levels change in relation to the stage of a given disease and which may be used to accurately assign the disease stage of a patient. The accurate selection of biomarker candidates is crucial, because it determines the outcome of further validation studies and the ultimate success of efforts to develop diagnostic and prognostic assays with high specificity and sensitivity. The success of biomarker discovery depends on several factors: consistent and reproducible phenotyping of the individuals from whom biological samples are obtained; the quality of the analytical methodology, which in turn determines the quality of the collected data; the accuracy of the computational methods used to extract quantitative and molecular identity information to define the biomarker candidates from raw analytical data; and finally the performance of the applied statistical methods in the selection of a limited list of compounds with the potential to discriminate between predefined classes of samples. De novo biomarker research consists of a biomarker discovery part and a biomarker validation part (3). Biomarker discovery uses analytical techniques that try to measure as many compounds as possible in a relatively low number of samples. The goal of subsequent data preprocessing and statistical analysis is to select a limited number of candidates, which are subsequently subjected to targeted analyses in large number of samples for validation.Advanced technology, such as high-performance liquid chromatography–mass spectrometry (LC-MS),¹ is increasingly applied in biomarker discovery research. Such analyses detect tens of thousands of compounds, as well as background-related signals, in a single biological sample, generating enormous amounts of multivariate data. Data preprocessing workflows reduce data complexity considerably by trying to extract only the information related to compounds resulting in a quantitative feature matrix, in which rows and columns correspond to samples and extracted features, respectively, or vice versa. Features may also be related to data preprocessing artifacts, and the ratio of such erroneous features to compound-related features depends on the performance of the data preprocessing workflow (4). Preprocessed LC-MS data sets contain a large number of features relative to the sample size. These features are characterized by their m/z value and retention time, and in the ideal case they can be combined and linked to compound identities such as metabolites, peptides, and proteins. In LC-MS-based proteomics and metabolomics studies, sample analysis is so time consuming that it is practically impossible to increase the number of samples to a level that balances the number of features in a data set. Therefore, the success of biomarker discovery depends on powerful feature selection methods that can deal with a low sample size and a high number of features. Because of the unfavorable statistical situation and the risk of overfitting the data, it is ultimately pivotal to validate the selected biomarker candidates in a larger set of independent samples, preferably in a double-blinded fashion, using targeted analytical methods (1).Biomarker selection is often based on classification methods that are preceded by feature selection methods (filters) or which have built-in feature selection modules (wrappers and embedded methods) that can be used to select a list of compounds/peaks/features that provide the best classification performance for predefined sample groups (e.g. healthy versus diseased) (5). Classification methods are able to classify an unknown sample into a predefined sample class. Univariate feature selection methods such as filters (t test or Wilcoxon–Mann–Whitney tests) cannot be used for sample classification. Other classification methods such as the nearest shrunken centroid method have intrinsic feature selection ability, whereas other classification methods such as principal component discriminant analysis (PCDA) and partial least squares regression coupled with discriminant analysis (PLSDA) should be augmented with a feature selection method. There are classifiers having no feature selection option that perform the classification using all variables, such as support vector machines that use non-linear kernels (6). Classification methods without the ability to select features cannot be used for biomarker discovery, because these methods aim to classify samples into predefined classes but cannot identify the limited number of variables (features or compounds) that form the basis of the classification (6, 7). Different statistical methods with feature selection have been developed according to the complexity of the analyzed data, and these have been extensively reviewed (5, 6, 8, 9). Ways of optimizing such methods to improve sensitivity and specificity are a major topic in current biomarker discovery research and in the many “omics-related” research areas (6, 10, 11). Comparisons of classification methods with respect to their classification and learning performance have been initiated. Van der Walt et al. (12) focused on finding the most accurate classifiers for simulated data sets with sample sizes ranging from 20 to 100. Rubingh et al. (13) compared the influence of sample size in an LC-MS metabolomics data set on the performance of three different statistical validation tools: cross validation, jack-knifing model parameters, and a permutation test. That study concluded that for small sample sets, the outcome of these validation methods is influenced strongly by individual samples and therefore cannot be trusted, and the validation tool cannot be used to indicate problems due to sample size or the representativeness of sampling. This implies that reducing the dimensionality of the feature space is critical when approaching a classification problem in which the number of features exceeds the number of samples by a large margin. Dimensionality reduction retains a smaller set of features to bring the feature space in line with the sample size and thus allow the application of classification methods that perform with acceptable accuracy only when the sample size and the feature size are similar.In this study we compared different classification methods focusing on feature selection in two types of spiked LC-MS data sets that mimic the situation of a biomarker discovery study. Our results provide guidelines for researchers who will engage in biomarker discovery or other differential profiling “omics” studies with respect to sample size and selecting the most appropriate feature selection method for a given data set. We evaluated the following approaches: univariate t test and Mann–Whitney–Wilcoxon test (mww test) with multiple testing correction (14), nearest shrunken centroid (NSC) (15, 16), support vector machine–recursive features elimination (SVM-RFE) (17), PLSDA (18), and PCDA (19). PCDA and PLSDA were combined with the rank-product as a feature selection criterion (20). These methods were evaluated with data sets having three characteristics: different biological background, varying sample size, and varying within- and between-class variability of the added compounds. Data were acquired via LC-MS from human urine and porcine cerebrospinal fluid (CSF) samples that were spiked with a set of known peptides (true positives) at different concentration levels. These samples were then combined in two classes containing peptides spiked at low and high concentration levels. The performance of the classification methods with feature selection was measured based on their ability to select features that were related to the spiked peptides. Because true positives were known in our data set, we compared performance based on the f-score (the harmonic mean of precision and recall) and the g-score (the geometric mean of accuracy).