Pairwise Comparisons Using Trimmed Means or M-Estimators when Working with Dependent Groups

For J dependent groups, let θ_j, j = 1, …, J, be some measure of location associated with the jth group. A common goal is computing confidence intervals for the pairwise differences, θ_j — θ_k, j < k, such that the simultaneous probability coverage is 1 — α. If means are used, it is well known that slight departures from normality (as measured by the Kolmogorov distance) toward a heavy-tailed distribution can substantially inflate the standard error of the sample mean, which in turn can result in relatively low power. Also, when distributions differ in shape, or when sampling from skewed distributions with relatively light tails, practical problems arise when the goal is to obtain confidence intervals with simultaneous probability coverage reasonably close to the nominal level. Extant theoretical and simulation results suggest replacing means with trimmed means. The Tukey-McLaughlin method is easily adapted to the problem at hand via the Bonferroni inequality, but this paper illustrates that practical concerns remain. Here, the main result is that the percentile t bootstrap method, used in conjunction with trimmed means, gives improved probability coverage and substantially better power. A method based on a one-step M-estimator is also considered but found to be less satisfactory.     

Confidence intervals for the mean of diagnostic test charge data containing zeros

In this paper, we consider the problem of interval estimation for the mean of diagnostic test charges. Diagnostic test charge data may contain zero values, and the nonzero values can often be modeled by a log-normal distribution. Under such a model, we propose three different interval estimation procedures: a percentile-t bootstrap interval based on sufficient statistics and two likelihood-based confidence intervals. For theoretical properties, we show that the two likelihood-based one-sided confidence intervals are only first-order accurate and that the bootstrap-based one-sided confidence interval is second-order accurate. For two-sided confidence intervals, all three proposed methods are second-order accurate. A simulation study in finite-sample sizes suggests all three proposed intervals outperform a widely used minimum variance unbiased estimator (MVUE)-based interval except for the case of one-sided lower end-point intervals when the skewness is very small. Among the proposed one-sided intervals, the bootstrap interval has the best coverage accuracy. For the two-sided intervals, when the sample size is small, the bootstrap method still yields the best coverage accuracy unless the skewness is very small, in which case the bias-corrected ML method has the best accuracy. When the sample size is large, all three proposed intervals have similar coverage accuracy. Finally, we analyze with the proposed methods one real example assessing diagnostic test charges among older adults with depression.     

A Comparison of Robust Estimators in the Presence of Arbitrarily Right Censored Data

Robust estimation of a location parameter is considered when the data from an unknown symmetric population are subject to arbitrary right-censorship. Comparisons are made between various M-estimators, several L-estimators (trimmed means), and the Kaplan-Meier median. Ten sampling distributions, two uniform censoring distributions, and three sample sizes are examined. A Cauchy censoring distribution is also considered when the sample size is equal to twenty for each of the ten sampling distributions. Performance is based on the estimated mean square error.     

Multiple Comparisons Among Dependent Groups Based on a Modified One‐Step M‐Estimator

Currently, among multiple comparison procedures for dependent groups, a bootstrap‐t with a 20% trimmed mean performs relatively well in terms of both Type I error probabilities and power. However, trimmed means suffer from two general concerns described in the paper. Robust M‐estimators address these concerns, but now no method has been found that gives good control over the probability of a Type I error when sample sizes are small. The paper suggests using instead a modified one‐step M‐estimator that retains the advantages of both trimmed means and robust M‐estimators. Yet another concern is that the more successful methods for trimmed means can be too conservative in terms of Type I errors. Two methods for performing all pairwise multiple comparisons are considered. In simulations, both methods avoid a familywise error (FWE) rate larger than the nominal level. The method based on comparing measures of location associated with the marginal distributions can have an actual FWE that is well below the nominal level when variables are highly correlated. However, the method based on difference scores performs reasonably well with very small sample sizes, and it generally performs better than any of the methods studied in Wilcox (1997b).     

Trimmed logit method for estimating the ED50 in quantal bioassay

Trimmed nonparametric procedures such as the trimmed Spearman-Karber method have been proposed in the literature for overcoming the deficiencies of the probit and logit models in the analysis of quantal bioassay data. However, there are situations where the median effective dose (ED50) is not calculable with the trimmed Spearman-Karber method, but is estimable with a parametric model. Also, it is helpful to have a parametric model for estimating percentiles of the dose-response curve such as the ED10 and ED25. A trimmed logit method that combines the advantages of a parametric model with that of trimming in dealing with heavy-tailed distributions is presented here. These advantages are substantiated with examples of actual bioassay data. Simulation results are presented to support the validity of the trimmed logit method, which has been found to work well in our experience with over 200 data sets. A computer program for computing the ED50 and associated 95% asymptotic confidence interval, based on the trimmed logit method, can be obtained from the authors.     

Contrasting Diversity Values: Statistical Inferences Based on Overlapping Confidence Intervals

Ecologists often contrast diversity (species richness and abundances) using tests for comparing means or indices. However, many popular software applications do not support performing standard inferential statistics for estimates of species richness and/or density. In this study we simulated the behavior of asymmetric log-normal confidence intervals and determined an interval level that mimics statistical tests with P(α) = 0.05 when confidence intervals from two distributions do not overlap. Our results show that 84% confidence intervals robustly mimic 0.05 statistical tests for asymmetric confidence intervals, as has been demonstrated for symmetric ones in the past. Finally, we provide detailed user-guides for calculating 84% confidence intervals in two of the most robust and highly-used freeware related to diversity measurements for wildlife (i.e., EstimateS, Distance).     

Confidence Intervals for the Relative Risk Ratio Parameter from Survival Data Under a Random Censorship Model in Biomedical and Epidemiologic Studies (By Simulation)

The present paper reports the results of a Monte Carlo simulation study to examine the performance of several approximate confidence intervals for the Relative Risk Ratio (RRR) parameter in an epidemiologic study, involving two groups of individuals. The first group consists of n₁ individuals, called the experimental group, who are exposed to some carcinogen, say radiation, whose effect on the incidence of some form of cancer, say skin cancer, is being investigated. The second group consists of n₂ individuals (called the control group) who are not exposed to the carcinogen. Two cases are considered in which the life times (or time to cancer) in the two groups follow (i) the exponential and (ii) the Weibull distributions. The case when the life times follow a Rayleigh distribution follows as a particular case. A general random censorship model is considered in which the life times of the individuals are censored on the right by random censoring times following (i) the exponential and (ii) the Weibull distributions. The Relative Risk Ratio parameter in the study is defined as the ratio of the hazard rates in the two distributions of the times to cancer. Approximate confidence intervals are constructed for the RRR parameter using its maximum likelihood estimator (m.l.e) and several other methods, including a method due to FIELLER. SPROTT'S (1973) and Cox's (1953) suggestions, as well as the Box-Cox (1964) transformation, are also utilized to construct approximate confidence intervals. The performance of these confidence intervals in small samples is investigated by means of some Monte Carlo simulations based on 500 random samples. Our simulation study indicates that many of these confidence intervals perform quite well in samples of size 10 and 15, in terms of the coverage probability and expected length of the interval.     

Some Adaptive Robust Estimators which Work with Real Data

An adaptive R-estimator θ_A and an adaptive trimmed mean MAT are proposed. The performance of these and a number of other robust estimators are studied on real data sets, drawn from the astronomical, behavioural, biomedical, chemical, engineering and physical sciences. In the case of sets that can be assumed to have come from symmetric distributions, the best performer is θ_A. The next best performers are the Hodges-Lehmann estimator, Bisquare (7.5) and Huber (1.5), in that order. MAT works well with all kinds of sets–symmetric or skewed. Extensions of these results to ANOVA and regression models are mentioned.     

Sample size determination for confidence intervals on the population mean and on the difference between two population means

Sample size determination is usually based on the premise that a hypothesis test is to be used. A confidence interval can sometimes serve better than a hypothesis test. In this paper a method is presented for sample size determination based on the premise that a confidence interval for a simple mean, or for the difference between two means, with normally distributed data is to be used. For this purpose, a concept of power relevant to confidence intervals is given. Some useful tables giving required sample size using this method are also presented.     

A Note on Testing Hypotheses about Trimmed Means

A well known result is that skewness can cause problems when testing hypotheses about measures of location, particulary when a one-sided test is of interest. Wilcox (1994) reports both theoretical and simulation results showing that when testing hypotheses about trimmed means, control over Type I error probabilities can be substantially better than methods for means. However, at least in some situations, control over the probability of a Type I error might still be judged to be inadequate. One way of adressing this concern is to combine trimmed means with the bootsrap method advocated by Westfall and Yuong (1993). This note reports simulation results indicating that there are situations where substantial improvements over Type I error probabilities are indeed obtained.     

Calculating confidence intervals for prediction error in microarray classification using resampling

Cross-validation based point estimates of prediction accuracy are frequently reported in microarray class prediction problems. However these point estimates can be highly variable, particularly for small sample numbers, and it would be useful to provide confidence intervals of prediction accuracy. We performed an extensive study of existing confidence interval methods and compared their performance in terms of empirical coverage and width. We developed a bootstrap case cross-validation (BCCV) resampling scheme and defined several confidence interval methods using BCCV with and without bias-correction. The widely used approach of basing confidence intervals on an independent binomial assumption of the leave-one-out cross-validation errors results in serious under-coverage of the true prediction error. Two split-sample based methods previously proposed in the literature tend to give overly conservative confidence intervals. Using BCCV resampling, the percentile confidence interval method was also found to be overly conservative without bias-correction, while the bias corrected accelerated (BCa) interval method of Efron returns substantially anti-conservative confidence intervals. We propose a simple bias reduction on the BCCV percentile interval. The method provides mildly conservative inference under all circumstances studied and outperforms the other methods in microarray applications with small to moderate sample sizes.     

Symmetric Intervals and Confidence Intervals

We show that the symmetric “confidence intervals” of WESTLAKE (1972, 1976), widely used and referred to in bioequivalence studies, are not confidence intervals in any accepted sense. Nevertheless, meaningful symmetric intervals can be constructed in the context of Bayesian or fiducial inference.     

On Parametric Confidence Intervals for the Cost‐Effectiveness Ratio

When comparing two competing interventions, confidence intervals for cost‐effectiveness ratios (CERs) provide information on the uncertainty in their point estimates. Techniques for constructing these confidence intervals are much debated. We provide a formal comparison of the Fieller, symmetric and Bonferroni methods for constructing confidence intervals for the CER using only the joint asymptotic distribution of the incremental cost and incremental effectiveness of the two interventions being compared. We prove the existence of a finite interval under the Fieller method when the incremental effectiveness is statistically significant. When this difference is not significant the Fieller method yields an unbounded confidence interval. The Fieller interval is always wider than the symmetric interval, but the latter is an approximation to the Fieller interval when the incremental effectiveness is highly significant. The Bonferroni method is shown to produce the widest interval. Because it accounts for the likely correlation between cost and effectiveness measures, and the intuitively appealing relationship between the existence of a bounded interval and the significance of the incremental effectiveness, the Fieller interval is to be preferred in reporting a confidence interval for the CER.     

A study on confidence intervals for incremental cost-effectiveness ratios

In health policy and economics studies, the incremental cost-effectiveness ratio (ICER) has long been used to compare the economic consequences relative to the health benefits of therapies. Due to the skewed distributions of the costs and ICERs, much research has been done on how to obtain confidence intervals of ICERs, using either parametric or nonparametric methods, with or without the presence of censoring. In this paper, we will examine and compare the finite sample performance of many approaches via simulation studies. For the special situation when the health effect of the treatment is not statistically significant, we will propose a new bootstrapping approach to improve upon the bootstrap percentile method that is currently available. The most efficient way of constructing confidence intervals will be identified and extended to the censored data case. Finally, a data example from a cardiovascular clinical trial is used to demonstrate the application of these methods.     

Confidence Intervals for Population Allele Frequencies: The General Case of Sampling from a Finite Diploid Population of Any Size

The estimation of population allele frequencies using sample data forms a central component of studies in population genetics. These estimates can be used to test hypotheses on the evolutionary processes governing changes in genetic variation among populations. However, existing studies frequently do not account for sampling uncertainty in these estimates, thus compromising their utility. Incorporation of this uncertainty has been hindered by the lack of a method for constructing confidence intervals containing the population allele frequencies, for the general case of sampling from a finite diploid population of any size. In this study, we address this important knowledge gap by presenting a rigorous mathematical method to construct such confidence intervals. For a range of scenarios, the method is used to demonstrate that for a particular allele, in order to obtain accurate estimates within 0.05 of the population allele frequency with high probability (%), a sample size of is often required. This analysis is augmented by an application of the method to empirical sample allele frequency data for two populations of the checkerspot butterfly (Melitaea cinxia L.), occupying meadows in Finland. For each population, the method is used to derive % confidence intervals for the population frequencies of three alleles. These intervals are then used to construct two joint % confidence regions, one for the set of three frequencies for each population. These regions are then used to derive a % confidence interval for Jost''s D, a measure of genetic differentiation between the two populations. Overall, the results demonstrate the practical utility of the method with respect to informing sampling design and accounting for sampling uncertainty in studies of population genetics, important for scientific hypothesis-testing and also for risk-based natural resource management.     

Estimation of past demographic parameters from the distribution of pairwise differences when the mutation rates vary among sites: application to human mitochondrial DNA.

Distributions of pairwise differences often called "mismatch distributions" have been extensively used to estimate the demographic parameters of past population expansions. However, these estimations relied on the assumption that all mutations occurring in the ancestry of a pair of genes lead to observable differences (the infinite-sites model). This mutation model may not be very realistic, especially in the case of the control region of mitochondrial DNA, where this methodology has been mostly applied. In this article, we show how to infer past demographic parameters by explicitly taking into account a finite-sites model with heterogeneity of mutation rates. We also propose an alternative way to derive confidence intervals around the estimated parameters, based on a bootstrap approach. By checking the validity of these confidence intervals by simulations, we find that only those associated with the timing of the expansion are approximately correctly estimated, while those around the population sizes are overly large. We also propose a test of the validity of the estimated demographic expansion scenario, whose proper behavior is verified by simulation. We illustrate our method with human mitochondrial DNA, where estimates of expansion times are found to be 10-20% larger when taking into account heterogeneity of mutation rates than under the infinite-sites model.     

Confidence intervals for the substitution number in the nucleotide substitution models

In the nucleotide substitution model for molecular evolution, a major task in the exploration of an evolutionary process is to estimate the substitution number per site of a protein or DNA sequence. The usual estimators are based on the observation of the difference proportion of the two nucleotide sequences. However, a more objective approach is to report a confidence interval with precision rather than only providing point estimators. The conventional confidence intervals used in the literature for the substitution number are constructed by the normal approximation. The performance and construction of confidence intervals for evolutionary models have not been much investigated in the literature. In this article, the performance of these conventional confidence intervals for one-parameter and two-parameter models are explored. Results show that the coverage probabilities of these intervals are unsatisfactory when the true substitution number is small. Since the substitution number may be small in many situations for an evolutionary process, the conventional confidence interval cannot provide accurate information for these cases. Improved confidence intervals for the one-parameter model with desirable coverage probability are proposed in this article. A numerical calculation shows the substantial improvement of the new confidence intervals over the conventional confidence intervals.     

Likelihood based observability analysis and confidence intervals for predictions of dynamic models

ABSTRACT: BACKGROUND: Predicting a system's behavior based on a mathematical model is a primary task in Systems Biology. If the model parameters are estimated from experimental data, the parameter uncertainty has to be translated into confidence intervals for model predictions. For dynamic models of biochemical networks, the nonlinearity in combination with the large number of parameters hampers the calculation of prediction confidence intervals and renders classical approaches as hardly feasible. RESULTS: In this article reliable confidence intervals are calculated based on the prediction profile likelihood. Such prediction confidence intervals of the dynamic states can be utilized for a data-based observability analysis. The method is also applicable if there are non-identifiable parameters yielding to some insufficiently specified modelpredictions that can be interpreted as non-observability. Moreover, a validation profile likelihood is introduced that should be applied when noisy validation experiments are to be interpreted. CONCLUSIONS: The presented methodology allows the propagation of uncertainty from experimental to model pre-dictions. Although presented in the context of ordinary differential equations, the concept is general and also applicable to other types of models. Matlab code which can be used as a template to implement the method is provided at http://www.fdmold.uni-freiburg.de/~ckreutz/PPL .     

Improved error bounds for genetic distances from DNA sequences

The genetic distance between two DNA sequences may be measured by the average number of nucleotide substitutions per position that has occurred since the two sequences diverged from a common ancestor. Estimates of this quantity can be derived from Markov models for the substitution process, while the variances are estimated using the delta method and confidence intervals calculated assuming normality. However, when the sampling distribution of the estimator deviates from normality, such intervals will not be accurate. For simple one-parameter models of nucleotide substitution, we propose a transformation of normal confidence intervals, which yields an almost exact approximation to the true confidence intervals of the distance estimators. To calculate confidence intervals for more complicated models, we propose the saddlepoint approximation. A simulation study shows that the saddlepoint-derived confidence intervals are a real improvement over existing methods.