The consumer's risk in clinical trials

In any formal statistical test of the null hypothesis (the statement that a population parameter is equal to a specific value), there are two possible types of error. Type 1 or alpha error has occurred if the investigator rejects the null hypothesis when it is true. For example, an experimental treatment is declared an advance over standard treatment when it is not. Type 2 or beta error has occurred if the null hypothesis is not rejected when it is false. In this case, the investigator concludes that the experimental treatment is no different than the standard when it actually is. The two types of error can be conceptualized, respectively, as the consumer's risk and the producer's risk. In many reports of clinical trial methodology, it is the producer's risk that is emphasized. It is understandable why producer's risk would be of concern to authors of clinical studies. There are, however, numerous potential sources of consumer's risk. It is the latter type of risk that is the primary subject of this report.     

Performance of the relative-rate test under nonstationary models of nucleotide substitution.

Relative-rate tests have previously been developed to compare the substitution rates of two sequences or two groups of sequences. These tests usually assume that the process of nucleotide substitution is stationary and the same for all lineages, i.e., uniform. In this study, we conducted simulations to assess the performance of the relative-rate tests when the molecular-clock (MC) hypothesis is true (i.e., there is no rate difference between lineages), but the stationarity and uniformity assumptions are violated. Kimura's and bias-corrected LogDet distances were used. We found that the computation of the variances and covariances of LogDet distances had to be modified, because the constraint that the sum of the frequencies of the 16 nucleotide pair types is equal to 1 must be imposed. Comparison of the rates of two single sequences (Wu and Li's test) or two groups of sequences (Li and Bousquet's test) gave similar results. When the sequences are long (> or = 500 nt), the test based on LogDet distances and their appropriate variances and covariances is appropriate even when the substitution process is not stationary and/or not uniform. That is, at the 5% significance level, the test rejects the MC hypothesis in about 5% of the simulation replicates. In contrast, if the sequences are short (< or = 200 bases) and highly divergent, the LogDet test is very conservative due to overestimation of the variances of the distances. When the uniformity assumption is violated, the relative-rate test based on Kimura's distances can be severely misleading because of differences in base composition between sequences. However, if the uniformity assumption held and so the base frequencies remained similar among sequences, the rate of rejection turned out to be close to 5%, especially with short sequences. Under such conditions, the test using Kimura's distances performs better than the LogDet test. The reason seems to be that these distances are less affected by a reduction in the number of sites than the LogDet distances because they depend on only two parameters.     

Trimmed Weighted Simes' Test for Two One‐Sided Hypotheses With Arbitrarily Correlated Test Statistics

The two‐sided Simes test is known to control the type I error rate with bivariate normal test statistics. For one‐sided hypotheses, control of the type I error rate requires that the correlation between the bivariate normal test statistics is non‐negative. In this article, we introduce a trimmed version of the one‐sided weighted Simes test for two hypotheses which rejects if (i) the one‐sided weighted Simes test rejects and (ii) both p‐values are below one minus the respective weighted Bonferroni adjusted level. We show that the trimmed version controls the type I error rate at nominal significance level α if (i) the common distribution of test statistics is point symmetric and (ii) the two‐sided weighted Simes test at level 2α controls the level. These assumptions apply, for instance, to bivariate normal test statistics with arbitrary correlation. In a simulation study, we compare the power of the trimmed weighted Simes test with the power of the weighted Bonferroni test and the untrimmed weighted Simes test. An additional result of this article ensures type I error rate control of the usual weighted Simes test under a weak version of the positive regression dependence condition for the case of two hypotheses. This condition is shown to apply to the two‐sided p‐values of one‐ or two‐sample t‐tests for bivariate normal endpoints with arbitrary correlation and to the corresponding one‐sided p‐values if the correlation is non‐negative. The Simes test for such types of bivariate t‐tests has not been considered before. According to our main result, the trimmed version of the weighted Simes test then also applies to the one‐sided bivariate t‐test with arbitrary correlation.     

Ecologists should not use statistical significance tests to interpret simulation model results

Simulation models are widely used to represent the dynamics of ecological systems. A common question with such models is how changes to a parameter value or functional form in the model alter the results. Some authors have chosen to answer that question using frequentist statistical hypothesis tests (e.g. ANOVA). This is inappropriate for two reasons. First, p‐values are determined by statistical power (i.e. replication), which can be arbitrarily high in a simulation context, producing minuscule p‐values regardless of the effect size. Second, the null hypothesis of no difference between treatments (e.g. parameter values) is known a priori to be false, invalidating the premise of the test. Use of p‐values is troublesome (rather than simply irrelevant) because small p‐values lend a false sense of importance to observed differences. We argue that modelers should abandon this practice and focus on evaluating the magnitude of differences between simulations. Synthesis Researchers analyzing field or lab data often test ecological hypotheses using frequentist statistics (t‐tests, ANOVA, etc.) that focus on p‐values. Field and lab data usually have limited sample sizes, and p‐values are valuable for quantifying the probability of making incorrect inferences in that situation. However, modern ecologists increasingly rely on simulation models to address complex questions, and those who were trained in frequentist statistics often apply the hypothesis‐testing approach inappropriately to their simulation results. Our paper explains why p‐values are not informative for interpreting simulation models, and suggests better ways to evaluate the ecological significance of model results.     

Group sequential methods for an ordinal logistic random-effects model under misspecification

Interim analyses in clinical trials are planned for ethical as well as economic reasons. General results have been published in the literature that allow the use of standard group sequential methodology if one uses an efficient test statistic, e.g., when Wald-type statistics are used in random-effects models for ordinal longitudinal data. These models often assume that the random effects are normally distributed. However, this is not always the case. We will show that, when the random-effects distribution is misspecified in ordinal regression models, the joint distribution of the test statistics over the different interim analyses is still a multivariate normal distribution, but a sandwich-type correction to the covariance matrix is needed in order to obtain the correct covariance matrix. The independent increment structure is also investigated. A bias in estimation will occur due to the misspecification. However, we will also show that the treatment effect estimate will be unbiased under the null hypothesis, thus maintaining the type I error. Extensive simulations based on a toenail dermatophyte onychomycosis trial are used to illustrate our results.     

Family-based tests of association in the presence of linkage

Linkage analysis may not provide the necessary resolution for identification of the genes underlying phenotypic variation. This is especially true for gene-mapping studies that focus on complex diseases that do not exhibit Mendelian inheritance patterns. One positional genomic strategy involves application of association methodology to areas of identified linkage. Detection of association in the presence of linkage localizes the gene(s) of interest to more-refined regions in the genome than is possible through linkage analysis alone. This strategy introduces a statistical complexity when family-based association tests are used: the marker genotypes among siblings are correlated in linked regions. Ignoring this correlation will compromise the size of the statistical hypothesis test, thus clouding the interpretation of test results. We present a method for computing the expectation of a wide range of association test statistics under the null hypothesis that there is linkage but no association. To standardize the test statistic, an empirical variance-covariance estimator that is robust to the sibling marker-genotype correlation is used. This method is widely applicable: any type of phenotypic measure or family configuration can be used. For example, we analyze a deletion in the A2M gene at the 5' splice site of "exon II" of the bait region in Alzheimer disease (AD) discordant sibships. Since the A2M gene lies in a chromosomal region (chromosome 12p) that consistently has been linked to AD, association tests should be conducted under the null hypothesis that there is linkage but no association.     

Bayesian Data Analysis: A Fresh Approach to Power Issues and Null Hypothesis Interpretation

One of the first things one learns in a basic psychology or statistics course is that you cannot prove the null hypothesis that there is no difference between two conditions such as a patient group and a normal control group. This remains true. However now, thanks to ongoing progress by a special group of devoted methodologists, even when the result of an inferential test is p?>?.05, it is now possible to rigorously and quantitatively conclude that (a) the null hypothesis is actually unlikely, and (b) that the alternative hypothesis of an actual difference between treatment and control is more probable than the null. Alternatively, it is also possible to conclude quantitatively that the null hypothesis is much more likely than the alternative. Without Bayesian statistics, we couldn’t say anything if a simple inferential analysis like a t-test yielded p?>?.05. The present, mostly non-quantitative article describes free resources and illustrative procedures for doing Bayesian analysis, with t-test and ANOVA examples.

     

The "late" Ca channel in squid axons

Squid giant axons were injected with aequorin and then treated with seawater containing 50 mM Ca and 100-465 mM K+. Measurements of light production suggested a phasic entry of Ca as well as an enhanced steady-state aequorin glow. After a test K+ depolarization, the aequorin-injected axon was stimulated for 30 min in Li seawater that was Ca-free, a procedure known to reduce [Na]i to about one-half the normal concentration. Reapplication of the elevated K+ test solution now showed that the Ca entry was virtually abolished by this stimulation in Li. A subsequent stimulation of the axon in Na seawater for 30 min resulted in recovery of the response to depolarization by high K+ noted in a normal fresh axon. In axons first tested for a high K+ response and then stimulated in Na seawater for 30 min (where [Na]i increases approximately 30%), there was approximately eight fold enhancement in this response to a test polarization. Axons depolarized with 465 mM K seawater in the absence of external Ca for several minutes were still capable of producing a large phasic entry of Ca when [Ca]0 was made 50 mM, which suggests that it is Ca entry itself rather than membrane depolarization that produced inactivation. Responses to stimulation at 60 pulses/s in Na seawater containing 50 mM Ca are at best only 5% of those measured with high K solutions. The response to repetitive stimulation is not measurable if [Ca]o is made 1 mM, whereas the response to steady depolarization is scarcely affected.     

Test Procedures in Configural Frequency Analysis (CFA) Controlling the Local and Multiple Level

The test statistics used until now in the CFA have been developed under the assumption of the overall hypothesis of total independence. Therefore, the multiple test procedures based on these statistics are really only different tests of the overall hypothesis. If one likes to test a special cell hypothesis, one should only assume that this hypothesis is true and not the whole overall hypothesis. Such cell tests can then be used as elements of a multiple test procedure. In this paper it is shown that the usual test procedures can be very anticonservative (except of the two-dimensional, and, for some procedures, the three-dimensional case), and corrected test procedures are developed. Furthermore, for the construction of multiple tests controlling the multiple level, modifications of Holm's (1979) procedure are proposed which lead to sharper results than his general procedure and can also be performed very easily.     

On the power of some binomial modifications of the Bonferroni multiple test

Widely used in testing statistical hypotheses, the Bonferroni multiple test has a rather low power that entails a high risk to accept falsely the overall null hypothesis and therefore to not detect really existing effects. We suggest that when the partial test statistics are statistically independent, it is possible to reduce this risk by using binomial modifications of the Bonferroni test. Instead of rejecting the null hypothesis when at least one of n partial null hypotheses is rejected at a very high level of significance (say, 0.005 in the case of n = 10), as it is prescribed by the Bonferroni test, the binomial tests recommend to reject the null hypothesis when at least k partial null hypotheses (say, k = [n/2]) are rejected at much lower level (up to 30-50%). We show that the power of such binomial tests is essentially higher as compared with the power of the original Bonferroni and some modified Bonferroni tests. In addition, such an approach allows us to combine tests for which the results are known only for a fixed significance level. The paper contains tables and a computer program which allow to determine (retrieve from a table or to compute) the necessary binomial test parameters, i.e. either the partial significance level (when k is fixed) or the value of k (when the partial significance level is fixed).     

How well is the probability of tumor cure after fractionated irradiation described by Poisson statistics?

The probability of tumor cure in a homogeneous population of tumors exposed to fractionated radiotherapy was modeled using numerical simulations and compared with the predictions of Poisson statistics, assuming exact knowledge of the relevant tumor parameters (clonogen number, radiosensitivity, and growth kinetics). The results show that although Poisson statistics (based on exact knowledge of all parameters) accurately describes the probability of tumor cure when no proliferation occurs during treatment, it underestimates the cure rate when proliferation does occur. In practice, however, the inaccuracy is not likely to be more than about 10%. When the tumor parameters are unknown and are estimated by fitting an empirical Poisson model to tumor-cure data from a homogeneous population of proliferative tumors, the resulting estimates of tumor growth rate and radiosensitivity accurately reflect the true values, but the estimate of initial clonogen number is biased downward. A new formula that is more accurate than Poisson statistics in predicting the probability of tumor cure when proliferation occurs during treatment is discussed.     

Nonparametric comparison of two survival-time distributions in the presence of dependent censoring

When testing the null hypothesis that treatment arm-specific survival-time distributions are equal, the log-rank test is asymptotically valid when the distribution of time to censoring is conditionally independent of randomized treatment group given survival time. We introduce a test of the null hypothesis for use when the distribution of time to censoring depends on treatment group and survival time. This test does not make any assumptions regarding independence of censoring time and survival time. Asymptotic validity of this test only requires a consistent estimate of the conditional probability that the survival event is observed given both treatment group and that the survival event occurred before the time of analysis. However, by not making unverifiable assumptions about the data-generating mechanism, there exists a set of possible values of corresponding sample-mean estimates of these probabilities that are consistent with the observed data. Over this subset of the unit square, the proposed test can be calculated and a rejection region identified. A decision on the null that considers uncertainty because of censoring that may depend on treatment group and survival time can then be directly made. We also present a generalized log-rank test that enables us to provide conditions under which the ordinary log-rank test is asymptotically valid. This generalized test can also be used for testing the null hypothesis when the distribution of censoring depends on treatment group and survival time. However, use of this test requires semiparametric modeling assumptions. A simulation study and an example using a recent AIDS clinical trial are provided.     

Freeze-drying of red blood cells: how useful are freeze/thaw experiments for optimization of the cooling rate?

A red blood cell suspension, prepared according to a high-yield HES cryopreservation protocol, was frozen at selected cooling rates of 50, 220, 1250, 4200, and 13,500 K/min. After either thawing or vacuum-drying, the cell recovery was determined using a modified saline stability test. As expected, the recovery of thawed samples followed the theory of Mazur's two-factor hypothesis. The best result was found at a cooling rate of 220 K/min. In contrast, the recovery of freeze-dried and rehydrated samples was very poor at that rate, but maximal at 4200 K/min where thawing caused almost complete hemolysis. This discrepancy is attributed to different damaging mechanisms involved with the respective sample processing subsequent to freezing. While thawing leads to increased devitrification and recrystallization at supraoptimal cooling rates for cryopreservation, the resultant almost vitreous sample structure seems to be advantageous for vacuum-drying. It can be concluded that freeze/thaw experiments are not sufficient for optimization of the cooling rate for freeze-drying.     

Theoretical and empirical power of regression and maximum-likelihood methods to map quantitative trait loci in general pedigrees

Both theoretical calculations and simulation studies have been used to compare and contrast the statistical power of methods for mapping quantitative trait loci (QTLs) in simple and complex pedigrees. A widely used approach in such studies is to derive or simulate the expected mean test statistic under the alternative hypothesis of a segregating QTL and to equate a larger mean test statistic with larger power. In the present study, we show that, even when the test statistic under the null hypothesis of no linkage follows a known asymptotic distribution (the standard being chi(2)), it cannot be assumed that the distribution under the alternative hypothesis is noncentral chi(2). Hence, mean test statistics cannot be used to indicate power differences, and a comparison between methods that are based on simulated average test statistics may lead to the wrong conclusion. We illustrate this important finding, through simulations and analytical derivations, for a recently proposed new regression method for the analysis of general pedigrees to map quantitative trait loci. We show that this regression method is not necessarily more powerful nor computationally more efficient than a maximum-likelihood variance-component approach. We advocate the use of empirical power to compare trait-mapping methods.     

Applications of the Wei-Lachin Multivariate One-Sided Test for Multiple Outcomes on Possibly Different Scales

Many studies aim to assess whether a therapy has a beneficial effect on multiple outcomes simultaneously relative to a control. Often the joint null hypothesis of no difference for the set of outcomes is tested using separate tests with a correction for multiple tests, or using a multivariate T ²-like MANOVA or global test. However, a more powerful test in this case is a multivariate one-sided or one-directional test directed at detecting a simultaneous beneficial treatment effect on each outcome, though not necessarily of the same magnitude. The Wei-Lachin test is a simple 1 df test obtained from a simple sum of the component statistics that was originally described in the context of a multivariate rank analysis. Under mild conditions this test provides a maximin efficient test of the null hypothesis of no difference between treatment groups for all outcomes versus the alternative hypothesis that the experimental treatment is better than control for some or all of the component outcomes, and not worse for any. Herein applications are described to a simultaneous test for multiple differences in means, proportions or life-times, and combinations thereof, all on potentially different scales. The evaluation of sample size and power for such analyses is also described. For a test of means of two outcomes with a common unit variance and correlation 0.5, the sample size needed to provide 90% power for two separate one-sided tests at the 0.025 level is 64% greater than that needed for the single Wei-Lachin multivariate one-directional test at the 0.05 level. Thus, a Wei-Lachin test with these operating characteristics is 39% more efficient than two separate tests. Likewise, compared to a T ²-like omnibus test on 2 df, the Wei-Lachin test is 32% more efficient. An example is provided in which the Wei-Lachin test of multiple components has superior power to a test of a composite outcome.     

An Exact Control‐Versus‐Treatment Comparison Test Based on Ranked Set Samples

Summary A control‐versus‐treatment multiple comparison test procedure is developed based on ranked set samples. The test, constructed based on K‐independent exact median confidence intervals corresponding to the K populations, rejects the null hypothesis of equal medians if the median confidence intervals of the control group and any one of the other K ? 1 treatment groups are disjoint. The use of the proposed test is illustrated with data from an agricultural experiment.     

Score Tests for Exploring Complex Models: Application to HIV Dynamics Models

In biostatistics, more and more complex models are being developed. This is particularly the case in system biology. Fitting complex models can be very time‐consuming, since many models often have to be explored. Among the possibilities are the introduction of explanatory variables and the determination of random effects. The particularity of this use of the score test is that the null hypothesis is not itself very simple; typically, some random effects may be present under the null hypothesis. Moreover, the information matrix cannot be computed, but only an approximation based on the score. This article examines this situation with the specific example of HIV dynamics models. We examine the score test statistics for testing the effect of explanatory variables and the variance of random effect in this complex situation. We study type I errors and the statistical powers of this score test statistics and we apply the score test approach to a real data set of HIV‐infected patients.     

K+ transport properties of K+ channels in the plasma membrane of Vicia faba guard cells

Electrical properties of the plasma membrane of guard cell protoplasts isolated from stomates of Vicia faba leaves were studied by application of the whole-cell configuration of the patch-clamp technique. The two types of K+ currents that have recently been identified in guard cells may allow efflux of K+ during stomatal closing, and uptake of K+ during stomatal opening (Schroeder et al., 1987). A detailed characterization of ion transport properties of the inward-rectifying (IK+,in) and the outward-rectifying (IK+,out) K+ conductance is presented here. The permeability ratios of IK+,in and IK+,out currents for K+ over monovalent alkali metal ions were determined. The resulting permeability sequences (PK+ greater than PRb+ greater than PNa+ greater than PLi+ much greater than PCs+) corresponded closely to the ion specificity of guard cell movements in V. faba. Neither K+ currents exhibited significant inactivation when K+ channels were activated for prolonged periods (greater than 10 min). The absence of inactivation may permit long durations of K+ fluxes, which occur during guard cell movements. Activation potentials of inward K+ currents were not shifted when external K+ concentrations were changed. This differs strongly from the behavior of inward-rectifying K+ channels in animal tissue. Blue light and fusicoccin induce hyperpolarization by stimulation of an electrogenic pump. From slow-whole-cell recordings it was concluded that electrogenic pumps require cytoplasmic substrates for full activation and that the magnitude of the pump current is sufficient to drive K+ uptake through IK+,in channels. First, direct evidence was gained for the hypothesis that IK+,in channels are a molecular pathway for K+ accumulation by the finding that IK+,in was blocked by Al3+ ions, which are known to inhibit stomatal opening but not closing. The results presented in this study strongly support a prominent role for IK+,in and IK+,out channels in K+ transport across the plasma membrane of guard cells.     

Microsatellite allele sizes: a simple test to assess their significance on genetic differentiation

The mutation process at microsatellite loci typically occurs at high rates and with stepwise changes in allele sizes, features that may introduce bias when using classical measures of population differentiation based on allele identity (e.g., F(ST), Nei's Ds genetic distance). Allele size-based measures of differentiation, assuming a stepwise mutation process [e.g., Slatkin's R(ST), Goldstein et al.'s (deltamu)(2)], may better reflect differentiation at microsatellite loci, but they suffer high sampling variance. The relative efficiency of allele size- vs. allele identity-based statistics depends on the relative contributions of mutations vs. drift to population differentiation. We present a simple test based on a randomization procedure of allele sizes to determine whether stepwise-like mutations contributed to genetic differentiation. This test can be applied to any microsatellite data set designed to assess population differentiation and can be interpreted as testing whether F(ST) = R(ST). Computer simulations show that the test efficiently identifies which of F(ST) or R(ST) estimates has the lowest mean square error. A significant test, implying that R(ST) performs better than F(ST), is obtained when the mutation rate, mu, for a stepwise mutation process is (a) >/= m in an island model (m being the migration rate among populations) or (b) >/= 1/t in the case of isolated populations (t being the number of generations since population divergence). The test also informs on the efficiency of other statistics used in phylogenetical reconstruction [e.g., Ds and (deltamu)(2)], a nonsignificant test meaning that allele identity-based statistics perform better than allele size-based ones. This test can also provide insights into the evolutionary history of populations, revealing, for example, phylogeographic patterns, as illustrated by applying it on three published data sets.     

Effect of different cycling frequencies during incremental exercise on the venous plasma potassium concentration in humans

The effect of different muscle shortening velocity was studied during cycling at a pedalling rate of 60 and 120 rev.min(-1) on the [K+]v in humans. Twenty-one healthy young men aged 22.5+/-2.2 years, body mass 72.7+/-6.4 kg, VO2 max 3.720+/-0.426 l. min(-1), performed an incremental exercise test until exhaustion. The power output increased by 30 W every 3 min, using an electrically controlled ergometer Ergoline 800 S (see Zoladz et al. J. Physiol. 488: 211-217, 1995). The test was performed twice: once at a cycling frequency of 60 rev.min(-1) (test A) and a few days later at a frequency of 120 rev. min(-1) (test B). At rest and at the end of each step (i.e. the last 15 s) antecubital venous blood samples for [K+]p were taken. Gas exchange variables were measured continuously (breath-by-breath) using Oxycon Champion Jaeger. The pre-exercise [K+]v in both tests was not significantly different amounting to 4.24+/-0.36 mmol.l(-1) in test A, and 4.37+/-0.45 mmol.l(-1) in test B. However, the [K+]p during cycling at 120 rev. min(-1) was significantly higher (p<0.001, ANOVA for repeated measurements) at each power output when compared to cycling at 60 rev.min(-1). The maximal power output reached 293+/-31 W in test A which was significantly higher (p<0.001) than in test B, which amounted to 223+/-40 W. The VO2max values in both tests reached 3.720+/-0.426 l. min(-1) vs 3.777+/-0.514 l. min(-1). These values were not significantly different. When the [K+]v was measured during incremental cycling exercise, a linear increase in [K+]v was observed in both tests. However, a significant (p<0.05) upward shift in the [K+]v and a % VO2max relationship was detected during cycling at 120 rev.min(-1). The [K+]v measured at the VO2max level in tests A and B amounted to 6.00+/-0.47 mmol.l-1 vs 6.04+/-0.41 mmol.l-1, respectively. This difference was not significant. It may thus be concluded that: a) generation of the same external mechanical power output during cycling at a pedalling rate of 120 rev.min(-1) causes significantly higher [K+]v changes than when cycling at 60 rev.min(-1), b) the increase of venous plasma potassium concentration during dynamic incremental exercise is linearly related to the metabolic cost of work expressed by the percentage of VO2max (increase as reported previously by Vollestad et al. J. Physiol. 475: 359-368, 1994), c) there is a tendency towards upward up shift in the [K+]v and % VO2max relation during cycling at 120 rev.min(-1) when compared to cycling at 60 rev.min(-1).