A note on familywise error rate for a primary and secondary endpoint

Hung et al. (2007) considered the problem of controlling the type I error rate for a primary and secondary endpoint in a clinical trial using a gatekeeping approach in which the secondary endpoint is tested only if the primary endpoint crosses its monitoring boundary. They considered a two-look trial and showed by simulation that the naive method of testing the secondary endpoint at full level α at the time the primary endpoint reaches statistical significance does not control the familywise error rate at level α. Tamhane et al. (2010) derived analytic expressions for familywise error rate and power and confirmed the inflated error rate of the naive approach. Nonetheless, many people mistakenly believe that the closure principle can be used to prove that the naive procedure controls the familywise error rate. The purpose of this note is to explain in greater detail why there is a problem with the naive approach and show that the degree of alpha inflation can be as high as that of unadjusted monitoring of a single endpoint.     

Testing a Primary and a Secondary Endpoint in a Group Sequential Design

Summary We consider a clinical trial with a primary and a secondary endpoint where the secondary endpoint is tested only if the primary endpoint is significant. The trial uses a group sequential procedure with two stages. The familywise error rate (FWER) of falsely concluding significance on either endpoint is to be controlled at a nominal level α. The type I error rate for the primary endpoint is controlled by choosing any α‐level stopping boundary, e.g., the standard O'Brien–Fleming or the Pocock boundary. Given any particular α‐level boundary for the primary endpoint, we study the problem of determining the boundary for the secondary endpoint to control the FWER. We study this FWER analytically and numerically and find that it is maximized when the correlation coefficient ρ between the two endpoints equals 1. For the four combinations consisting of O'Brien–Fleming and Pocock boundaries for the primary and secondary endpoints, the critical constants required to control the FWER are computed for different values of ρ. An ad hoc boundary is proposed for the secondary endpoint to address a practical concern that may be at issue in some applications. Numerical studies indicate that the O'Brien–Fleming boundary for the primary endpoint and the Pocock boundary for the secondary endpoint generally gives the best primary as well as secondary power performance. The Pocock boundary may be replaced by the ad hoc boundary for the secondary endpoint with a very little loss of secondary power if the practical concern is at issue. A clinical trial example is given to illustrate the methods.     

Development of gatekeeping strategies in confirmatory clinical trials

This paper discusses multiplicity issues arising in confirmatory clinical trials with hierarchically ordered multiple objectives. In order to protect the overall type I error rate, multiple objectives are analyzed using multiple testing procedures. When the objectives are ordered and grouped in multiple families (e.g. families of primary and secondary endpoints), gatekeeping procedures are employed to account for this hierarchical structure. We discuss considerations arising in the process of building gatekeeping procedures, including proper use of relevant trial-specific information and criteria for selecting gatekeeping procedures. The methods and principles discussed in this paper are illustrated using a clinical trial in patients with type II diabetes mellitus.     

Interim analysis incorporating short‐ and long‐term binary endpoints

Designs incorporating more than one endpoint have become popular in drug development. One of such designs allows for incorporation of short‐term information in an interim analysis if the long‐term primary endpoint has not been yet observed for some of the patients. At first we consider a two‐stage design with binary endpoints allowing for futility stopping only based on conditional power under both fixed and observed effects. Design characteristics of three estimators: using primary long‐term endpoint only, short‐term endpoint only, and combining data from both are compared. For each approach, equivalent cut‐off point values for fixed and observed effect conditional power calculations can be derived resulting in the same overall power. While in trials stopping for futility the type I error rate cannot get inflated (it usually decreases), there is loss of power. In this study, we consider different scenarios, including different thresholds for conditional power, different amount of information available at the interim, different correlations and probabilities of success. We further extend the methods to adaptive designs with unblinded sample size reassessments based on conditional power with inverse normal method as the combination function. Two different futility stopping rules are considered: one based on the conditional power, and one from P‐values based on Z‐statistics of the estimators. Average sample size, probability to stop for futility and overall power of the trial are compared and the influence of the choice of weights is investigated.     

Improved two‐stage group sequential procedures for testing a secondary endpoint after the primary endpoint achieves significance

In two‐stage group sequential trials with a primary and a secondary endpoint, the overall type I error rate for the primary endpoint is often controlled by an α‐level boundary, such as an O'Brien‐Fleming or Pocock boundary. Following a hierarchical testing sequence, the secondary endpoint is tested only if the primary endpoint achieves statistical significance either at an interim analysis or at the final analysis. To control the type I error rate for the secondary endpoint, this is tested using a Bonferroni procedure or any α‐level group sequential method. In comparison with marginal testing, there is an overall power loss for the test of the secondary endpoint since a claim of a positive result depends on the significance of the primary endpoint in the hierarchical testing sequence. We propose two group sequential testing procedures with improved secondary power: the improved Bonferroni procedure and the improved Pocock procedure. The proposed procedures use the correlation between the interim and final statistics for the secondary endpoint while applying graphical approaches to transfer the significance level from the primary endpoint to the secondary endpoint. The procedures control the familywise error rate (FWER) strongly by construction and this is confirmed via simulation. We also compare the proposed procedures with other commonly used group sequential procedures in terms of control of the FWER and the power of rejecting the secondary hypothesis. An example is provided to illustrate the procedures.     

Loss of consciousness as criterion of +Gz tolerance at Institute of Aviation Medicine MMA during +Gz acceleration selective test

+Gz induced loss of consciousness (G-LOC) is one of the most serious threats to aircrews flying high performance fighter aircraft. From the early beginning of use of our Centrifuge, use in selection was primary task. As a functional "endpoints" we use criteria: loss of peripheral vision, extreme pulse rate (above 180 b.p.m.), arrhythmias and loss of consciousness. The key-method in selection the candidate who tolerates +Gz stress on the best way is selection by common selective centrifuge "Test of linear increasing of acceleration" (TOLIA). We used gradual onset rate (GOR--0.1 G/s) and maximum/peak value: +5.5 Gz, +6.0 Gz and 7.0 Gz. Applied peak value depends on the goal of the test. The lowest peak value is for candidates planned for Air Academy, higher peak value is for those pilots planned for training to supersonic combat aircrafts and the highest peak value is for pilots who are planned to fly High performance combat aircrafts. We examined 2192 candidates in the last 20 years. Eleven subjects experienced G-LOC episodes. All episodes of G-LOC had occurred occasionally and without warning symptoms (loss of peripheral vision, gray out, blackout). The percentage of subjects having G-LOC episodes was 0.50%. Nine subjects experienced G-LOC during primary selection (+5.5 Gz), one G-LOC were observed at secondary selection (+6.0 Gz) and one G-LOC was observed during tertiary selection (+7 Gz). G-LOC is the only "endpoint" in the centrifuge selection which disqualifies the candidate at once and forever for planned flying duties. The other "endpoints" (loss of peripheral vision, heart rate above 180 b.p.m., arrhythmias) allow one more testing, not less than seven days later.     

In vitro activity of the echinocandins. How should it be evaluated?

Echinocandins are a novel class of antifungal drugs. They have good activity against Candida spp and Aspergillus spp. Their low selective toxicity allows their administration at high doses with few secondary side effects. We have reviewed the available data on the endpoints for these drugs in their in vitro susceptibility testing on yeasts and moulds. The microdilution broth method is the most commonly used technique and MIC-1 (80% of growth inhibition) seems to be the most reliable endpoint when yeasts are tested. This endpoint also seems to be the most appropriate for the different drugs when they are combined with echinocandins using the checkerboard method for testing yeasts. By contrast, in the case of moulds, the minimum effective concentration (MEC) correlates better with the in vivo activity than the MIC when echinocandins are tested, and when these drugs are combined with other antifungals, MIC-2 (50% of growth inhibition) seems the most appropriate endpoint. Criteria based on drug pharmacodynamics is the most useful to define the echinocandin endpoints that best correlate with their in vivo efficacy.     

Sample size determination in clinical trials with multiple co‐primary endpoints including mixed continuous and binary variables

In the field of pharmaceutical drug development, there have been extensive discussions on the establishment of statistically significant results that demonstrate the efficacy of a new treatment with multiple co‐primary endpoints. When designing a clinical trial with such multiple co‐primary endpoints, it is critical to determine the appropriate sample size for indicating the statistical significance of all the co‐primary endpoints with preserving the desired overall power because the type II error rate increases with the number of co‐primary endpoints. We consider overall power functions and sample size determinations with multiple co‐primary endpoints that consist of mixed continuous and binary variables, and provide numerical examples to illustrate the behavior of the overall power functions and sample sizes. In formulating the problem, we assume that response variables follow a multivariate normal distribution, where binary variables are observed in a dichotomized normal distribution with a certain point of dichotomy. Numerical examples show that the sample size decreases as the correlation increases when the individual powers of each endpoint are approximately and mutually equal.     

A simple and flexible Holm gatekeeping procedure

Major objectives of a clinical trial are commonly stated in a hierarchical order as primary and secondary. The parallel gatekeeping testing strategy provides an opportunity to assess secondary objectives when all or partial primary objectives are achieved. The current available gatekeeping procedures have different pros and cons so users either need to justify the assumption associated with some procedures or tolerate suboptimal power performance of other procedures. By applying the Holm test with a flexible alpha splitting technique, we propose a procedure which (1) is powerful for assessing the primary objectives, (2) can be used when no assumption can be made on the dependency structure of test statistics, and (3) has the full flexibility to allocate user-preferred alpha to assess the secondary objectives based on the number of primary objectives achieved. A real clinical trial example is used for illustration of the proposed procedure.     

Which and How Many Patients Should Be Included in Randomised Controlled Trials to Demonstrate the Efficacy of Biologics in Primary Sj?gren’s Syndrome?

Objective

The goal of this study was to determine how the choice of the primary endpoint influenced sample size estimates in randomised controlled trials (RCTs) of treatments for primary Sjögren’s syndrome (pSS).

Methods

We reviewed all studies evaluating biotechnological therapies in pSS to identify their inclusion criteria and primary endpoints. Then, in a large cohort (ASSESS), we determined the proportion of patients who would be included in RCTs using various inclusion criteria sets. Finally, we used the population of a large randomised therapeutic trial in pSS (TEARS) to assess the impact of various primary objectives and endpoints on estimated sample sizes. These analyses were performed only for the endpoints indicating greater efficacy of rituximab compared to the placebo.

Results

We identified 18 studies. The most common inclusion criteria were short disease duration; systemic involvement; high mean visual analogue scale (VAS) scores for dryness, pain, and fatigue; and biological evidence of activity. In the ASSESS cohort, 35 percent of patients had recent-onset disease (lower than 4 years), 68 percent systemic manifestations, 68 percent high scores on two of three VASs, and 52 percent biological evidence of activity. The primary endpoints associated with the smallest sample sizes (nlower than 200) were a VAS dryness score improvement higher to 20 mm by week 24 or variable improvements (10, 20, or 30 mm) in fatigue VAS by week 6 or 16. For patients with systemic manifestations, the ESSDAI change may be the most logical endpoint, as it reflects all domains of disease activity. However, the ESSDAI did not improve significantly with rituximab therapy in the TEARS study. Ultrasound score improvement produced the smallest sample size estimate in the TEARS study.

Conclusion

This study provides valuable information for designing future RCTs on the basis of previously published studies. Previous RCTs used inclusion criteria that selected a small part of the entire pSS population. The endpoint was usually based on VASs assessing patient complaints. In contrast to VAS dryness cut-offs, VAS fatigue cut-offs did not affect estimated sample sizes. SGUS improvement produced the smallest estimated sample size. Further studies are required to validate standardised SGUS modalities and assessment criteria. Thus, researchers should strive to develop a composite primary endpoint and to determine its best cut-off and assessment time point.     

The analysis of multiple endpoints in clinical trials

Treatment comparisons in randomized clinical trials usually involve several endpoints such that conventional significance testing can seriously inflate the overall Type I error rate. One option is to select a single primary endpoint for formal statistical inference, but this is not always feasible. Another approach is to apply Bonferroni correction (i.e., multiply each P-value by the total number of endpoints). Its conservatism for correlated endpoints is examined for multivariate normal data. A third approach is to derive an appropriate global test statistic and this paper explores one such test applicable to any set of asymptotically normal test statistics. Quantitative, binary, and survival endpoints are all considered within this general framework. Two examples are presented and the relative merits of the proposed strategies are discussed.     

Joint regression analysis for survival data in the presence of two sets of semi‐competing risks

In many clinical trials, multiple time‐to‐event endpoints including the primary endpoint (e.g., time to death) and secondary endpoints (e.g., progression‐related endpoints) are commonly used to determine treatment efficacy. These endpoints are often biologically related. This work is motivated by a study of bone marrow transplant (BMT) for leukemia patients, who may experience the acute graft‐versus‐host disease (GVHD), relapse of leukemia, and death after an allogeneic BMT. The acute GVHD is associated with the relapse free survival, and both the acute GVHD and relapse of leukemia are intermediate nonterminal events subject to dependent censoring by the informative terminal event death, but not vice versa, giving rise to survival data that are subject to two sets of semi‐competing risks. It is important to assess the impacts of prognostic factors on these three time‐to‐event endpoints. We propose a novel statistical approach that jointly models such data via a pair of copulas to account for multiple dependence structures, while the marginal distribution of each endpoint is formulated by a Cox proportional hazards model. We develop an estimation procedure based on pseudo‐likelihood and carry out simulation studies to examine the performance of the proposed method in finite samples. The practical utility of the proposed method is further illustrated with data from the motivating example.     

Properties of a nonparametric test for early comparison of treatments in clinical trials in the presence of surrogate endpoints

A nonparametric test is derived for comparing treatments with respect to the final endpoint in clinical trials in which the final endpoint has been observed for a random subset of patients, but results are available for a surrogate endpoint for a larger sample of patients. The test is an adaptation of the Wilcoxon-Mann-Whitney two-sample test, with an adjustment that involves a comparison of the ranks of the surrogate endpoints between patients with and without final endpoints. The validity of the test depends on the assumption that the patients with final endpoints represent a random sample of the patients registered in the study. This assumption is viable in trials in which the final endpoint is evaluated at a "landmark" timepoint in the patients' natural history. A small sample simulation study demonstrates that the test has a size that is close to the nominal value for all configurations evaluated. When compared with the conventional test based only on the final endpoints, the new test delivers substantial increases in power only when the surrogate endpoint is highly correlated with the true endpoint. Our research indicates that, in the absence of modeling assumptions, auxiliary information derived from surrogate endpoints can provide significant additional information only under special circumstances.     

Stepwise gatekeeping procedures in clinical trial applications

This paper discusses multiple testing problems in which families of null hypotheses are tested in a sequential manner and each family serves as a gatekeeper for the subsequent families. Gatekeeping testing strategies of this type arise frequently in clinical trials with multiple objectives, e.g., multiple endpoints and/or multiple dose-control comparisons. It is demonstrated in this paper that the parallel gatekeeping procedure of Dmitrienko, Offen and Westfall (2003) admits a simple stepwise representation (n null hypotheses can be tested in n steps rather than 2n steps required in the closed procedure). The stepwise representation considerably simplifies the implementation of gatekeeping procedures in practice and provides an important insight into the nature of gatekeeping inferences. The derived stepwise gatekeeping procedure is illustrated using clinical trial examples.     

Testing for Treatment Effects on Subsets of Endpoints

Multiple endpoints are tested to assess an overall treatment effect and also to identify which endpoints or subsets of endpoints contributed to treatment differences. The conventional p‐value adjustment methods, such as single‐step, step‐up, or step‐down procedures, sequentially identify each significant individual endpoint. Closed test procedures can also detect individual endpoints that have effects via a step‐by‐step closed strategy. This paper proposes a global‐based statistic for testing an a priori number, say, r of the k endpoints, as opposed to the conventional approach of testing one (r = 1) endpoint. The proposed test statistic is an extension of the single‐step p‐value‐based statistic based on the distribution of the smallest p‐value. The test maintains strong control of the FamilyWise Error (FWE) rate under the null hypothesis of no difference in any (sub)set of r endpoints among all possible combinations of the k endpoints. After rejecting the null hypothesis, the individual endpoints in the sets that are rejected can be tested further, using a univariate test statistic in a second step, if desired. However, the second step test only weakly controls the FWE. The proposed method is illustrated by application to a psychosis data set.     

Sample Size Reassessment and Hypothesis Testing in Adaptive Survival Trials

Mid-study design modifications are becoming increasingly accepted in confirmatory clinical trials, so long as appropriate methods are applied such that error rates are controlled. It is therefore unfortunate that the important case of time-to-event endpoints is not easily handled by the standard theory. We analyze current methods that allow design modifications to be based on the full interim data, i.e., not only the observed event times but also secondary endpoint and safety data from patients who are yet to have an event. We show that the final test statistic may ignore a substantial subset of the observed event times. An alternative test incorporating all event times is found, where a conservative assumption must be made in order to guarantee type I error control. We examine the power of this approach using the example of a clinical trial comparing two cancer therapies.     

The validation of surrogate endpoints in meta-analyses of randomized experiments

The validation of surrogate endpoints has been studied by Prentice (1989). He presented a definition as well as a set of criteria, which are equivalent only if the surrogate and true endpoints are binary. Freedman et al. (1992) supplemented these criteria with the so-called 'proportion explained'. Buyse and Molenberghs (1998) proposed replacing the proportion explained by two quantities: (1) the relative effect linking the effect of treatment on both endpoints and (2) an individual-level measure of agreement between both endpoints. The latter quantity carries over when data are available on several randomized trials, while the former can be extended to be a trial-level measure of agreement between the effects of treatment of both endpoints. This approach suggests a new method for the validation of surrogate endpoints, and naturally leads to the prediction of the effect of treatment upon the true endpoint, given its observed effect upon the surrogate endpoint. These ideas are illustrated using data from two sets of multicenter trials: one comparing chemotherapy regimens for patients with advanced ovarian cancer, the other comparing interferon-alpha with placebo for patients with age-related macular degeneration.     

A randomized controlled phase IIb trial of antigen-antibody immunogenic complex therapeutic vaccine in chronic hepatitis B patients

Background

The safety of the immune complexes composed of yeast-derived hepatitis B surface antigen (HBsAg) and antibodies (abbreviated as YIC) among healthy adults and chronic hepatitis B patients has been proved in phase I and phase IIa trial. A larger number of patients for study of dosage and efficacy are therefore needed.

Methods and Principal Findings

Two hundred forty two HBeAg-positive chronic hepatitis B patients were immunized with six injections of either 30 µg YIC, 60 µg of YIC or alum adjuvant as placebo at four-week intervals under code. HBV markers and HBV DNA were monitored during immunization and 24 weeks after the completion of immunization. The primary endpoint was defined as loss of HBeAg, or presence of anti-HBe antibody or suppression of HBV DNA, while the secondary endpoint was both HBeAg seroconversion and suppression of HBV DNA. Statistical significance was not reached in primary endpoints four weeks after the end of treatment among three groups, however, at the end of follow-up, HBeAg sero-conversion rate was 21.8%(17/78) and 9% (7/78) in the 60 µg YIC and placebo groups respectively (p = 0.03), with 95% confidence intervals at 1.5% to 24.1%. Using generalized estimating equations (GEEs) model, a significant difference of group effects was found between 60 µg YIC and the placebo groups in terms of the primary endpoint. Eleven serious adverse events occurred, which were 5.1%, 3.6%, and 5.0% in the placebo, 30 µg YIC and 60 µg YIC groups respectively (p>0.05).

Conclusions

Though statistical differences in the preset primary and secondary endpoints among the three groups were not reached, a late and promising HBeAg seroconversion effect was shown in the 60 µg YIC immunized regimen. By increasing the number of patients and injections, the therapeutic efficacy of YIC in chronic hepatitis B patients will be further evaluated.

Trial Registration

ChiCTR.org ChiCTR-TRC-00000022     

Magnetic field therapy in patients with cytostatics‐induced polyneuropathy: A prospective randomized placebo‐controlled phase‐III study

No causal treatment for chemotherapy‐induced peripheral neuropathy (CIPN) is known. Therefore, there is an urgent need to develop a therapy for CIPN. Only scarce clinical data are available concerning magnetic field therapy (MFT) in this context. We conducted a unicentric, randomized, double‐blind, placebo‐controlled phase‐III trial of an MFT device versus placebo. In this study, we randomized 44 patients with CIPN to two treatment groups, where 21 patients were treated with MFT (Group 1) and 23 patients received placebo (Group 2). We evaluated the efficacy of MFT at baseline (T₁), after 3 weeks of study treatment (T₂), and after 3 months of study treatment (T₃). The primary endpoint was nerve conduction velocity (NCV), while secondary endpoints were the Common Toxicity Criteria (CTCAE) score and the Pain Detect End Score at T₃. Seventeen of the patients in Group 1 and 14 patients in Group 2 completed the respective study treatment. The primary endpoint, significant improvement of NCV at T₃, was achieved by MFT (P = 0.015), particularly for sensory neurotoxicity of the peroneal nerve. Also, in respect to the secondary endpoints, significant improvement (P = 0.04) was achieved in terms of the patients’ subjectively perceived neurotoxicity (CTCAE score), but not of neuropathic pain (P = 0.11). From data in the randomized study presented here, a positive effect on the reduction of neurotoxicity can be assumed for the MFT device. Patients with sensory neurotoxicity in the lower limbs, especially, should therefore be offered this therapy. Bioelectromagnetics. 38:85–94, 2017. © 2016 The Authors. Bioelectromagnetics published by Wiley Periodicals, Inc.     

A simple meta-analytic approach for using a binary surrogate endpoint to predict the effect of intervention on true endpoint

A surrogate endpoint is an endpoint that is obtained sooner, at lower cost, or less invasively than the true endpoint for a health outcome and is used to make conclusions about the effect of intervention on the true endpoint. In this approach, each previous trial with surrogate and true endpoints contributes an estimated predicted effect of intervention on true endpoint in the trial of interest based on the surrogate endpoint in the trial of interest. These predicted quantities are combined in a simple random-effects meta-analysis to estimate the predicted effect of intervention on true endpoint in the trial of interest. Validation involves comparing the average prediction error of the aforementioned approach with (i) the average prediction error of a standard meta-analysis using only true endpoints in the other trials and (ii) the average clinically meaningful difference in true endpoints implicit in the trials. Validation is illustrated using data from multiple randomized trials of patients with advanced colorectal cancer in which the surrogate endpoint was tumor response and the true endpoint was median survival time.