A note on one-sided tests with multiple endpoints

Testing problems with multivariate one-sided alternative hypotheses are common in clinical trials with multiple endpoints. In the case of comparing two treatments, treatment 1 is often preferred if it is superior for at least one of the endpoints and not biologically inferior for the remaining endpoints. Bloch et al. (2001, Biometrics57, 1039-1047) propose an intersection-union test (IUT) for this testing problem, but their test does not utilize the appropriate multivariate one-sided test. In this note we modify their test by an alternative IUT that does utilize the appropriate one-sided test. Empirical and graphical evidence show that the proposed test is more appropriate for this testing problem.     

Testing for Treatment Effects on Subsets of Endpoints

Multiple endpoints are tested to assess an overall treatment effect and also to identify which endpoints or subsets of endpoints contributed to treatment differences. The conventional p‐value adjustment methods, such as single‐step, step‐up, or step‐down procedures, sequentially identify each significant individual endpoint. Closed test procedures can also detect individual endpoints that have effects via a step‐by‐step closed strategy. This paper proposes a global‐based statistic for testing an a priori number, say, r of the k endpoints, as opposed to the conventional approach of testing one (r = 1) endpoint. The proposed test statistic is an extension of the single‐step p‐value‐based statistic based on the distribution of the smallest p‐value. The test maintains strong control of the FamilyWise Error (FWE) rate under the null hypothesis of no difference in any (sub)set of r endpoints among all possible combinations of the k endpoints. After rejecting the null hypothesis, the individual endpoints in the sets that are rejected can be tested further, using a univariate test statistic in a second step, if desired. However, the second step test only weakly controls the FWE. The proposed method is illustrated by application to a psychosis data set.     

Procedures for two-sample comparisons with multiple endpoints controlling the experimentwise error rate

Clinical trials are often concerned with the comparison of two treatment groups with multiple endpoints. As alternatives to the commonly used methods, the T2 test and the Bonferroni method, O'Brien (1984, Biometrics 40, 1079-1087) proposes tests based on statistics that are simple or weighted sums of the single endpoints. This approach turns out to be powerful if all treatment differences are in the same direction [compare Pocock, Geller, and Tsiatis (1987, Biometrics 43, 487-498)]. The disadvantage of these multivariate methods is that they are suitable only for demonstrating a global difference, whereas the clinician is further interested in which specific endpoints or sets of endpoints actually caused this difference. It is shown here that all tests are suitable for the construction of a closed multiple test procedure where, after the rejection of the global hypothesis, all lower-dimensional marginal hypotheses and finally the single hypotheses are tested step by step. This procedure controls the experimentwise error rate. It is just as powerful as the multivariate test and, in addition, it is possible to detect significant differences between the endpoints or sets of endpoints.     

Sample size determination in clinical trials with multiple co‐primary endpoints including mixed continuous and binary variables

In the field of pharmaceutical drug development, there have been extensive discussions on the establishment of statistically significant results that demonstrate the efficacy of a new treatment with multiple co‐primary endpoints. When designing a clinical trial with such multiple co‐primary endpoints, it is critical to determine the appropriate sample size for indicating the statistical significance of all the co‐primary endpoints with preserving the desired overall power because the type II error rate increases with the number of co‐primary endpoints. We consider overall power functions and sample size determinations with multiple co‐primary endpoints that consist of mixed continuous and binary variables, and provide numerical examples to illustrate the behavior of the overall power functions and sample sizes. In formulating the problem, we assume that response variables follow a multivariate normal distribution, where binary variables are observed in a dichotomized normal distribution with a certain point of dichotomy. Numerical examples show that the sample size decreases as the correlation increases when the individual powers of each endpoint are approximately and mutually equal.     

Trimmed Weighted Simes' Test for Two One‐Sided Hypotheses With Arbitrarily Correlated Test Statistics

The two‐sided Simes test is known to control the type I error rate with bivariate normal test statistics. For one‐sided hypotheses, control of the type I error rate requires that the correlation between the bivariate normal test statistics is non‐negative. In this article, we introduce a trimmed version of the one‐sided weighted Simes test for two hypotheses which rejects if (i) the one‐sided weighted Simes test rejects and (ii) both p‐values are below one minus the respective weighted Bonferroni adjusted level. We show that the trimmed version controls the type I error rate at nominal significance level α if (i) the common distribution of test statistics is point symmetric and (ii) the two‐sided weighted Simes test at level 2α controls the level. These assumptions apply, for instance, to bivariate normal test statistics with arbitrary correlation. In a simulation study, we compare the power of the trimmed weighted Simes test with the power of the weighted Bonferroni test and the untrimmed weighted Simes test. An additional result of this article ensures type I error rate control of the usual weighted Simes test under a weak version of the positive regression dependence condition for the case of two hypotheses. This condition is shown to apply to the two‐sided p‐values of one‐ or two‐sample t‐tests for bivariate normal endpoints with arbitrary correlation and to the corresponding one‐sided p‐values if the correlation is non‐negative. The Simes test for such types of bivariate t‐tests has not been considered before. According to our main result, the trimmed version of the weighted Simes test then also applies to the one‐sided bivariate t‐test with arbitrary correlation.     

Bayesian Variable Selection with Joint Modeling of Categorical and Survival Outcomes: An Application to Individualizing Chemotherapy Treatment in Advanced Colorectal Cancer

Summary Colorectal cancer is the second leading cause of cancer related deaths in the United States, with more than 130,000 new cases of colorectal cancer diagnosed each year. Clinical studies have shown that genetic alterations lead to different responses to the same treatment, despite the morphologic similarities of tumors. A molecular test prior to treatment could help in determining an optimal treatment for a patient with regard to both toxicity and efficacy. This article introduces a statistical method appropriate for predicting and comparing multiple endpoints given different treatment options and molecular profiles of an individual. A latent variable‐based multivariate regression model with structured variance covariance matrix is considered here. The latent variables account for the correlated nature of multiple endpoints and accommodate the fact that some clinical endpoints are categorical variables and others are censored variables. The mixture normal hierarchical structure admits a natural variable selection rule. Inference was conducted using the posterior distribution sampling Markov chain Monte Carlo method. We analyzed the finite‐sample properties of the proposed method using simulation studies. The application to the advanced colorectal cancer study revealed associations between multiple endpoints and particular biomarkers, demonstrating the potential of individualizing treatment based on genetic profiles.     

Properties of a nonparametric test for early comparison of treatments in clinical trials in the presence of surrogate endpoints

A nonparametric test is derived for comparing treatments with respect to the final endpoint in clinical trials in which the final endpoint has been observed for a random subset of patients, but results are available for a surrogate endpoint for a larger sample of patients. The test is an adaptation of the Wilcoxon-Mann-Whitney two-sample test, with an adjustment that involves a comparison of the ranks of the surrogate endpoints between patients with and without final endpoints. The validity of the test depends on the assumption that the patients with final endpoints represent a random sample of the patients registered in the study. This assumption is viable in trials in which the final endpoint is evaluated at a "landmark" timepoint in the patients' natural history. A small sample simulation study demonstrates that the test has a size that is close to the nominal value for all configurations evaluated. When compared with the conventional test based only on the final endpoints, the new test delivers substantial increases in power only when the surrogate endpoint is highly correlated with the true endpoint. Our research indicates that, in the absence of modeling assumptions, auxiliary information derived from surrogate endpoints can provide significant additional information only under special circumstances.     

Blinded sample size recalculation in multiple composite population designs with normal data and baseline adjustments

The increasing interest in subpopulation analysis has led to the development of various new trial designs and analysis methods in the fields of personalized medicine and targeted therapies. In this paper, subpopulations are defined in terms of an accumulation of disjoint population subsets and will therefore be called composite populations. The proposed trial design is applicable to any set of composite populations, considering normally distributed endpoints and random baseline covariates. Treatment effects for composite populations are tested by combining p-values, calculated on the subset levels, using the inverse normal combination function to generate test statistics for those composite populations while the closed testing procedure accounts for multiple testing. Critical boundaries for intersection hypothesis tests are derived using multivariate normal distributions, reflecting the joint distribution of composite population test statistics given no treatment effect exists. For sample size calculation and sample size, recalculation multivariate normal distributions are derived which describe the joint distribution of composite population test statistics under an assumed alternative hypothesis. Simulations demonstrate the absence of any practical relevant inflation of the type I error rate. The target power after sample size recalculation is typically met or close to being met.     

Accurate critical constants for the one-sided approximate likelihood ratio test of a normal mean vector when the covariance matrix is estimated

Tang, Gnecco, and Geller (1989, Biometrika 76, 577-583) proposed an approximate likelihood ratio (ALR) test of the null hypothesis that a normal mean vector equals a null vector against the alternative that all of its components are nonnegative with at least one strictly positive. This test is useful for comparing a treatment group with a control group on multiple endpoints, and the data from the two groups are assumed to follow multivariate normal distributions with different mean vectors and a common covariance matrix (the homoscedastic case). Tang et al. derived the test statistic and its null distribution assuming a known covariance matrix. In practice, when the covariance matrix is estimated, the critical constants tabulated by Tang et al. result in a highly liberal test. To deal with this problem, we derive an accurate small-sample approximation to the null distribution of the ALR test statistic by using the moment matching method. The proposed approximation is then extended to the heteroscedastic case. The accuracy of both the approximations is verified by simulations. A real data example is given to illustrate the use of the approximations.     

An Evaluation of Beta-Binomial and Dirichlet-Trinomial Models for the Analysis of Reproductive and Developmental Effects

The common endpoints for the evaluation of reproductive and developmental toxic effects are the number of dead/resorbed fetuses, the number of malformed fetuses, and the number of normal fetuses for each litter. The joint distribution of the three endpoints could be modelled by a Dirichlettrinomial distribution or by a product of two-beta-binomial distributions. A simulation experiment is used to investigate the biases of the maximum likelihood estimate (MLE) for the probability of adverse effects under the Dirichlet-trinomial model and the beta-binomial model. Also, the type I errors and powers of the likelihood ratio test for comparing the difference between treatment and control are evaluated for the two underlying models. In estimation, the two MLE's are comparable, the bias estimates are small. In testing, the likelihood ratio test is generally more powerful under the Dirichlet-trinomial model than the beta-binomial model. The type I error rate is greater than the nominal level using the Dirichlet-trinomial model in some cases, when the data are generated from the two-beta-binomial model, and it is less than the nominal level using the beta-binomial model in other cases, when the data are generated from the Dirichlet-trinomial model.     

Endpoints in the design of clinical trials for primary sclerosing cholangitis

Primary sclerosing cholangitis is an enigmatic disease affecting the bile ducts, eventually leading to liver failure necessitating liver transplantation in many cases. There is currently no therapy that has proven to halt disease progression. One of the reasons for this is the lack of proper endpoints to measure the effect of medical intervention on the course of the disease. Relevant clinical endpoints such as death or liver transplantation occur too infrequently in this orphan disease to be used as endpoints in phase 2 or 3 trials. It is therefore of utmost importance to identify appropriate surrogate endpoints that are reasonably likely to measure true clinical benefit. This article will discuss a number of surrogate endpoints that are likely candidates to serve this role. This article is part of a Special Issue entitled: Cholangiocytes in Health and Diseaseedited by Jesus Banales, Marco Marzioni, Nicholas LaRusso and Peter Jansen.     

Analysis of trinomial responses from reproductive and developmental toxicity experiments.

This paper presents a Dirichlet-trinomial distribution for modelling data obtained from reproductive and developmental studies. The common endpoints for the evaluation of reproductive and developmental toxic effects are the number of dead fetuses, the number of malformed fetuses, and the number of normal fetuses for each litter. With current statistical methods for the evaluation of reproductive and developmental effects, the effect on the number of deaths and the effect on the number of malformations are analyzed separately. The Dirichlet-trinomial model provides a procedure for the analysis of multiple endpoints simultaneously. This proposed Dirichlet-trinomial model is a generalization of the beta-binomial model that has been used for handling the litter effect in reproductive and developmental experiments. Likelihood ratio tests for differences in the number of deaths, the number of malformations, and the number of normals among dosed and control groups are derived. The proposed test procedure based on the Dirichlet-trinomial model is compared with that based on the beta-binomial model with an application to a real data set.     

Circadian rhythm of the PRL response to TRH in healthy subjects

The PRL response to TRH constitutes an important clinical tool for diagnosing forms of hyperprolactinemic syndrome. Hence it is important to establish the characteristics of the circadian variation in the response of PRL to TRH to improve the diagnostic value of the test. Six male subjects, ranging in age from 23 to 24 years, participated in this study. All were considered healthy on the basis of clinical examination, biochemical and hormonal tests. Six TRH tests were performed on each subject, one test every other day during a total span of 12 days. Each test was performed at a different clock hour: 0000, 0400, 0800, 1200, 1600, 2000. For the test, subjects received 200 microgram TRH intravenously. Blood samples were drawn from a catheterized arm vein before the TRH injection (basal value) and 20, 30, 60 and 120 min after injection. At each timepoint 5 endpoints were determined for PRL on each subject. The population mean cosinor, according to Halberg, was used to investigate the circadian rhythm in each of the endpoints. All the 5 endpoints for PRL are consistent on showing p values near 0.5 and acrophase estimates before midnight (while basal value displays acrophase at 0400). Further investigations are necessary to clarify these circadian rhythms and the shift of the acrophases.     

Tests for differentiation in gene expression using a data-driven order or weights for hypotheses

In the analysis of gene expression by microarrays there are usually few subjects, but high-dimensional data. By means of techniques, such as the theory of spherical tests or with suitable permutation tests, it is possible to sort the endpoints or to give weights to them according to specific criteria determined by the data while controlling the multiple type I error rate. The procedures developed so far are based on a sequential analysis of weighted p-values (corresponding to the endpoints), including the most extreme situation of weighting leading to a complete order of p-values. When the data for the endpoints have approximately equal variances, these procedures show good power properties. In this paper, we consider an alternative procedure, which is based on completely sorting the endpoints, but smoothed in the sense that some perturbations in the sequence of the p-values are allowed. The procedure is relatively easy to perform, but has high power under the same restrictions as for the weight-based procedures.     

Gatekeeping testing via adaptive alpha allocation

In a typical clinical trial, there are one or two primary endpoints, and a few secondary endpoints. When at least one primary endpoint achieves statistical significance, there is considerable interest in using results for the secondary endpoints to enhance characterization of the treatment effect. Because multiple endpoints are involved, regulators may require that the familywise type I error rate be controlled at a pre-set level. This requirement can be achieved by using "gatekeeping" methods. However, existing methods suffer from logical oddities such as allowing results for secondary endpoint(s) to impact the likelihood of success for the primary endpoint(s). We propose a novel and easy-to-implement gatekeeping procedure that is devoid of such deficiencies. A real data example and simulation results are used to illustrate efficiency gains of our method relative to existing methods.     

Quantitation of human mammary epithelial stem cells with in vivo regenerative properties using a subrenal capsule xenotransplantation assay

Methods to identify and enumerate primitive, and typically rare, undifferentiated cells in normal tissue using functional endpoints are powerful tools for acquiring insights into the mechanisms that regulate normal tissue stem cell turnover and differentiation. In this paper, we describe a xenotransplantation-based protocol that allows mammary stem cells with in vivo tissue regenerative properties to be specifically detected and quantified among the heterogeneous cell populations obtained from dissociated normal human mammary tissue. This methodology involves implanting a collagen gel containing the test cells in combination with supportive fibroblasts under the kidney capsule of highly immune-deficient, hormone-supplemented mice and then, 4 weeks later, searching for regenerated human cells with in vitro clonogenic activity. Quantification of the input human mammary stem cells is achieved using standard limiting dilution transplant approaches. This approach circumvents the need to modify the mouse mammary fat pad, and is objective, rapid (～5 weeks) and economical to perform.     

Identification of the critical timing of sex determination in <Emphasis Type="Italic">Daphnia magna</Emphasis> (Crustacea,Branchiopoda) for use in toxicological studies

Daphnia are often used as test organisms to determine toxicity of chemicals found in the environment. Fecundity and mortality are the classic endpoints in such tests; however, there is increased interest in other sub-lethal endpoints. Most Daphnia reproduce by cyclic parthenogenesis, typically reproducing asexually to produce genetically identical females. Environmental cues trigger the production of males allowing for sexual reproduction. This ability to switch reproductive strategies has been increasingly used as a bioassay endpoint in screens for sub-lethal effects of contaminants. In order to optimize such test, it is necessary to know when sex is determined during Daphnia development. A previous study using the insect juvenile hormone methyl farnesoate has shown that sex determination occurred during the egg maturation period in Daphnia. Our study exposed female Daphnia magna carrying eggs and embryos at different stages of maturation, to the insecticide toxaphene to determine if the timing of sex determination is similarly influenced by a pesticide. Our results suggest that in response to toxaphene exposure sex is determined in the immature oocyte, before the final maturation cleavage, and before the developing egg is extruded into the brood chamber. Thus, sex determination is transgenerational, being determined while the egg is still immature and in the mother’s ovary.     

A generalized calibrated Bayesian hierarchical modeling approach to basket trials with multiple endpoints

A basket trial simultaneously evaluates a treatment in multiple cancer subtypes, offering an effective way to accelerate drug development in multiple indications. Many basket trials are designed and monitored based on a single efficacy endpoint, primarily the tumor response. For molecular targeted or immunotherapy agents, however, a single efficacy endpoint cannot adequately characterize the treatment effect. It is increasingly important to use more complex endpoints to comprehensively assess the risk–benefit profile of such targeted therapies. We extend the calibrated Bayesian hierarchical modeling approach to monitor phase II basket trials with multiple endpoints. We propose two generalizations, one based on the latent variable approach and the other based on the multinomial–normal hierarchical model, to accommodate different types of endpoints and dependence assumptions regarding information sharing. We introduce shrinkage parameters as functions of statistics measuring homogeneity among subgroups and propose a general calibration approach to determine the functional forms. Theoretical properties of the generalized hierarchical models are investigated. Simulation studies demonstrate that the monitoring procedure based on the generalized approach yields desirable operating characteristics.     

K‐Sample comparisons using propensity analysis

In this paper, we investigate K‐group comparisons on survival endpoints for observational studies. In clinical databases for observational studies, treatment for patients are chosen with probabilities varying depending on their baseline characteristics. This often results in noncomparable treatment groups because of imbalance in baseline characteristics of patients among treatment groups. In order to overcome this issue, we conduct propensity analysis and match the subjects with similar propensity scores across treatment groups or compare weighted group means (or weighted survival curves for censored outcome variables) using the inverse probability weighting (IPW). To this end, multinomial logistic regression has been a popular propensity analysis method to estimate the weights. We propose to use decision tree method as an alternative propensity analysis due to its simplicity and robustness. We also propose IPW rank statistics, called Dunnett‐type test and ANOVA‐type test, to compare 3 or more treatment groups on survival endpoints. Using simulations, we evaluate the finite sample performance of the weighted rank statistics combined with these propensity analysis methods. We demonstrate these methods with a real data example. The IPW method also allows us for unbiased estimation of population parameters of each treatment group. In this paper, we limit our discussions to survival outcomes, but all the methods can be easily modified for any type of outcomes, such as binary or continuous variables.     

Combining Global and Marginal Tests to Compare Two Treatments on Multiple Endpoints

We consider the problem of comparing two treatments on multiple endpoints where the goal is to identify the endpoints that have treatment effects, while controlling the familywise error rate. Two current approaches for this are (i) applying a global test within a closed testing procedure, and (ii) adjusting individual endpoint p‐values for multiplicity. We propose combining the two current methods. We compare the combined method with several competing methods in a simulation study. It is concluded that the combined approach maintains higher power under a variety of treatment effect configurations than the other methods and is thus more power‐robust.