A statistical approach to quasi-extinction forecasting

Forecasting population decline to a certain critical threshold (the quasi-extinction risk) is one of the central objectives of population viability analysis (PVA), and such predictions figure prominently in the decisions of major conservation organizations. In this paper, we argue that accurate forecasting of a population's quasi-extinction risk does not necessarily require knowledge of the underlying biological mechanisms. Because of the stochastic and multiplicative nature of population growth, the ensemble behaviour of population trajectories converges to common statistical forms across a wide variety of stochastic population processes. This paper provides a theoretical basis for this argument. We show that the quasi-extinction surfaces of a variety of complex stochastic population processes (including age-structured, density-dependent and spatially structured populations) can be modelled by a simple stochastic approximation: the stochastic exponential growth process overlaid with Gaussian errors. Using simulated and real data, we show that this model can be estimated with 20-30 years of data and can provide relatively unbiased quasi-extinction risk with confidence intervals considerably smaller than (0,1). This was found to be true even for simulated data derived from some of the noisiest population processes (density-dependent feedback, species interactions and strong age-structure cycling). A key advantage of statistical models is that their parameters and the uncertainty of those parameters can be estimated from time series data using standard statistical methods. In contrast for most species of conservation concern, biologically realistic models must often be specified rather than estimated because of the limited data available for all the various parameters. Biologically realistic models will always have a prominent place in PVA for evaluating specific management options which affect a single segment of a population, a single demographic rate, or different geographic areas. However, for forecasting quasi-extinction risk, statistical models that are based on the convergent statistical properties of population processes offer many advantages over biologically realistic models.     

Calculating confidence intervals for prediction error in microarray classification using resampling

Cross-validation based point estimates of prediction accuracy are frequently reported in microarray class prediction problems. However these point estimates can be highly variable, particularly for small sample numbers, and it would be useful to provide confidence intervals of prediction accuracy. We performed an extensive study of existing confidence interval methods and compared their performance in terms of empirical coverage and width. We developed a bootstrap case cross-validation (BCCV) resampling scheme and defined several confidence interval methods using BCCV with and without bias-correction. The widely used approach of basing confidence intervals on an independent binomial assumption of the leave-one-out cross-validation errors results in serious under-coverage of the true prediction error. Two split-sample based methods previously proposed in the literature tend to give overly conservative confidence intervals. Using BCCV resampling, the percentile confidence interval method was also found to be overly conservative without bias-correction, while the bias corrected accelerated (BCa) interval method of Efron returns substantially anti-conservative confidence intervals. We propose a simple bias reduction on the BCCV percentile interval. The method provides mildly conservative inference under all circumstances studied and outperforms the other methods in microarray applications with small to moderate sample sizes.     

Predicting the time to quasi-extinction for populations far below their carrying capacity

Populations threatened by extinction are often far below their carrying capacity. A population collapse or quasi-extinction is defined to occur when the population size reaches some given lower density. If this density is chosen to be large enough for the demographic stochasticity to be ignored compared to environmental stochasticity, then the logarithm of the population size may be modelled by a Brownian motion until quasi-extinction occurs. The normal-gamma mixture of inverse Gaussian distributions can then be applied to define prediction intervals for the time to quasi-extinction in such processes. A similar mixture is used to predict the population size at a finite time for the same process provided that quasi-extinction has not occurred before that time. Stochastic simulations indicate that the coverage of the prediction interval is very close to the probability calculated theoretically. As an illustration, the method is applied to predict the time to extinction of a declining population of white stork in southwestern Germany.     

Comparison of confidence intervals for adjusted attributable risk estimates under multinomial sampling

The epidemiologic concept of the adjusted attributable risk is a useful approach to quantitatively describe the importance of risk factors on the population level. It measures the proportional reduction in disease probability when a risk factor is eliminated from the population, accounting for effects of confounding and effect-modification by nuisance variables. The computation of asymptotic variance estimates for estimates of the adjusted attributable risk is often done by applying the delta method. Investigations on the delta method have shown, however, that the delta method generally tends to underestimate the standard error, leading to biased confidence intervals. We compare confidence intervals for the adjusted attributable risk derived by applying computer intensive methods like the bootstrap or jackknife to confidence intervals based on asymptotic variance estimates using an extensive Monte Carlo simulation and within a real data example from a cohort study in cardiovascular disease epidemiology. Our results show that confidence intervals based on bootstrap and jackknife methods outperform intervals based on asymptotic theory. Best variants of computer intensive confidence intervals are indicated for different situations.     

Empirical Bayes interval estimates that are conditionally equal to unadjusted confidence intervals or to default prior credibility intervals

Problems involving thousands of null hypotheses have been addressed by estimating the local false discovery rate (LFDR). A previous LFDR approach to reporting point and interval estimates of an effect-size parameter uses an estimate of the prior distribution of the parameter conditional on the alternative hypothesis. That estimated prior is often unreliable, and yet strongly influences the posterior intervals and point estimates, causing the posterior intervals to differ from fixed-parameter confidence intervals, even for arbitrarily small estimates of the LFDR. That influence of the estimated prior manifests the failure of the conditional posterior intervals, given the truth of the alternative hypothesis, to match the confidence intervals. Those problems are overcome by changing the posterior distribution conditional on the alternative hypothesis from a Bayesian posterior to a confidence posterior. Unlike the Bayesian posterior, the confidence posterior equates the posterior probability that the parameter lies in a fixed interval with the coverage rate of the coinciding confidence interval. The resulting confidence-Bayes hybrid posterior supplies interval and point estimates that shrink toward the null hypothesis value. The confidence intervals tend to be much shorter than their fixed-parameter counterparts, as illustrated with gene expression data. Simulations nonetheless confirm that the shrunken confidence intervals cover the parameter more frequently than stated. Generally applicable sufficient conditions for correct coverage are given. In addition to having those frequentist properties, the hybrid posterior can also be motivated from an objective Bayesian perspective by requiring coherence with some default prior conditional on the alternative hypothesis. That requirement generates a new class of approximate posteriors that supplement Bayes factors modified for improper priors and that dampen the influence of proper priors on the credibility intervals. While that class of posteriors intersects the class of confidence-Bayes posteriors, neither class is a subset of the other. In short, two first principles generate both classes of posteriors: a coherence principle and a relevance principle. The coherence principle requires that all effect size estimates comply with the same probability distribution. The relevance principle means effect size estimates given the truth of an alternative hypothesis cannot depend on whether that truth was known prior to observing the data or whether it was learned from the data.     

Genetic counseling in rare syndromes: a resampling method for determining an approximate confidence interval for gene location with linkage data from a single pedigree.

Multipoint linkage analysis is a powerful method for mapping a rare disease gene on the human gene map despite limited genotype and pedigree data. However, there is no standard procedure for determining a confidence interval for gene location by using multipoint linkage analysis. A genetic counselor needs to know the confidence interval for gene location in order to determine the uncertainty of risk estimates provided to a consultant on the basis of DNA studies. We describe a resampling, or "bootstrap," method for deriving an approximate confidence interval for gene location on the basis of data from a single pedigree. This method was used to define an approximate confidence interval for the location of a gene causing nonsyndromal X-linked mental retardation in a single pedigree. The approach seemed robust in that similar confidence intervals were derived by using different resampling protocols. Quantitative bounds for the confidence interval were dependent on the genetic map chosen. Once an approximate confidence interval for gene location was determined for this pedigree, it was possible to use multipoint risk analysis to estimate risk intervals for women of unknown carrier status. Despite the limited genotype data, the combination of the resampling method and multipoint risk analysis had a dramatic impact on the genetic advice available to consultants.     

Natural history of cervical intraepithelial neoplasia: a meta-analysis

OBJECTIVE: To determine the probabilities of transition of stages in the cervical cancer by conducting a meta-studies on the topic. STUDY DESIGN: We identified health states of interest in the natural history of cervical precancer, identified all possible papers that could meet selection criteria, developed relevance and acceptability criteria for inclusion, then thoroughly reviewed the selected studies. To determine the transition probability data we used a random effects model. We determined probabilities for 4 health state transitions. The 6-month mean predictive transition probability (95% confidence intervals with "prediction interval" in parentheses) for high grade squamous intraepithelial lesions (HSIL) to cancer was 0.0037 (0.00004, 0.03386), for low grade squamous intraepithelial lesions (LSIL) to HSIL was 0.0362 (0.00055, 0.23220), for HSIL to LSIL was 0.0282 (0.00027, 0.35782), and for LSIL to normal was 0.0740 (0.00119, 0.42672). CONCLUSION: The transition probabilities between cervical cancer health states for 6-month intervals are small; however, the cumulative risk of cervical cancer is significant. Markers to identify the cervical precursors that will lead to the transition to cervical cancer are needed.     

Estimation of a parameter and its exact confidence interval following sequential sample size reestimation trials

For confirmatory trials of regulatory decision making, it is important that adaptive designs under consideration provide inference with the correct nominal level, as well as unbiased estimates, and confidence intervals for the treatment comparisons in the actual trials. However, naive point estimate and its confidence interval are often biased in adaptive sequential designs. We develop a new procedure for estimation following a test from a sample size reestimation design. The method for obtaining an exact confidence interval and point estimate is based on a general distribution property of a pivot function of the Self-designing group sequential clinical trial by Shen and Fisher (1999, Biometrics55, 190-197). A modified estimate is proposed to explicitly account for futility stopping boundary with reduced bias when block sizes are small. The proposed estimates are shown to be consistent. The computation of the estimates is straightforward. We also provide a modified weight function to improve the power of the test. Extensive simulation studies show that the exact confidence intervals have accurate nominal probability of coverage, and the proposed point estimates are nearly unbiased with practical sample sizes.     

Online Prediction of the Running Time of Tasks

We describe and evaluate the Running Time Advisor (RTA), a system that can predict the running time of a compute-bound task on a typical shared, unreserved commodity host. The prediction is computed from linear time series predictions of host load and takes the form of a confidence interval that neatly expresses the error associated with the measurement and prediction processes – error that must be captured to make statistically valid decisions based on the predictions. Adaptive applications make such decisions in pursuit of consistent high performance, choosing, for example, the host where a task is most likely to meet its deadline. We begin by describing the system and summarizing the results of our previously published work on host load prediction. We then describe our algorithm for computing predictions of running time from host load predictions. We next evaluate the system using over 100,000 randomized testcases run on 39 different hosts, finding that is indeed capable of computing correct and useful confidence intervals. Finally, we report on our experience with using the RTA in application-oriented real-time scheduling in distributed systems.     

A simulation study on the accuracy of position and effect estimates of linked QTL and their asymptotic standard deviations using multiple interval mapping in an F2 scheme

Approaches like multiple interval mapping using a multiple-QTL model for simultaneously mapping QTL can aid the identification of multiple QTL, improve the precision of estimating QTL positions and effects, and are able to identify patterns and individual elements of QTL epistasis. Because of the statistical problems in analytically deriving the standard errors and the distributional form of the estimates and because the use of resampling techniques is not feasible for several linked QTL, there is the need to perform large-scale simulation studies in order to evaluate the accuracy of multiple interval mapping for linked QTL and to assess confidence intervals based on the standard statistical theory. From our simulation study it can be concluded that in comparison with a monogenetic background a reliable and accurate estimation of QTL positions and QTL effects of multiple QTL in a linkage group requires much more information from the data. The reduction of the marker interval size from 10 cM to 5 cM led to a higher power in QTL detection and to a remarkable improvement of the QTL position as well as the QTL effect estimates. This is different from the findings for (single) interval mapping. The empirical standard deviations of the genetic effect estimates were generally large and they were the largest for the epistatic effects. These of the dominance effects were larger than those of the additive effects. The asymptotic standard deviation of the position estimates was not a good criterion for the accuracy of the position estimates and confidence intervals based on the standard statistical theory had a clearly smaller empirical coverage probability as compared to the nominal probability. Furthermore the asymptotic standard deviation of the additive, dominance and epistatic effects did not reflect the empirical standard deviations of the estimates very well, when the relative QTL variance was smaller/equal to 0.5. The implications of the above findings are discussed.     

PROJECTING THE TREND OF STELLER SEA LION POPULATIONS IN WESTERN ALASKA

This paper attempts to project the trends of Steller sea lion ( Eumetopias jubatus ) populations in six subdivisions of the western Alaska population. The overall Western Alaska population has declined dramatically since the 1970s. Trends in half of the areas appear to have leveled-off and possibly to be on the increase. Bootstrapping has been used to provide confidence intervals on predictions for the 2004 counts. For the three areas in which we expect increases, the 95% confidence intervals on predictions were: Eastern Gulf (2,430–3,740), Central Gulf (3,260–3,660) and Central Aleutians (5,160–6,580). The Western Gulf counts have been somewhat erratic, with a gradual rate of decrease (about 2% per year) and wide confidence limits on a linear prediction (logarithmic scale) of 2,690–3,240. Trends in the Eastern Aleutians have been even more erratic, so that about all that can be inferred is that the population may be roughly stabilized. Only the Western Aleutians appear to be rapidly declining at about 10% per year, with a 95% confidence interval on a linear trend of 610–1,100. The predictions were made before the 2004 counts and are in reasonable accord with the 2004 counts. Age structure changes do not appeat to provide a viable explanation for the changing trends.     

On Parametric Confidence Intervals for the Cost‐Effectiveness Ratio

When comparing two competing interventions, confidence intervals for cost‐effectiveness ratios (CERs) provide information on the uncertainty in their point estimates. Techniques for constructing these confidence intervals are much debated. We provide a formal comparison of the Fieller, symmetric and Bonferroni methods for constructing confidence intervals for the CER using only the joint asymptotic distribution of the incremental cost and incremental effectiveness of the two interventions being compared. We prove the existence of a finite interval under the Fieller method when the incremental effectiveness is statistically significant. When this difference is not significant the Fieller method yields an unbounded confidence interval. The Fieller interval is always wider than the symmetric interval, but the latter is an approximation to the Fieller interval when the incremental effectiveness is highly significant. The Bonferroni method is shown to produce the widest interval. Because it accounts for the likely correlation between cost and effectiveness measures, and the intuitively appealing relationship between the existence of a bounded interval and the significance of the incremental effectiveness, the Fieller interval is to be preferred in reporting a confidence interval for the CER.     

Two-sample nonparametric estimation and confidence intervals under truncation.

We consider point estimates and confidence intervals for the difference in location or scale between two populations when the observations are subject to truncation. We suggest procedures analogous to those for the complete-sample case. A rigorous justification is presented to support the proposed confidence interval procedure. Finally, some simulations verify the properties of the estimators and confidence intervals. We illustrate the procedure using data on tumor size.     

A comparison of reflected versus test-based confidence intervals for the median survival time, based on censored data

The small-sample performance of some recently proposed nonparametric methods of constructing confidence intervals for the median survival time, based on randomly right-censored data, is compared with that of two new methods. Most of these methods are equivalent for large samples. All proposed intervals are either 'test-based' or 'reflected' intervals, in the sense defined in the paper. Coverage probabilities for the interval estimates were obtained by exact calculation for uncensored data, and by stimulation for three life distributions and four censoring patterns. In the range of situations studied, 'test-based' methods often have less than nominal coverage, while the coverage of the new 'reflected' confidence intervals is closer to nominal (although somewhat conservative), and these intervals are easy to compute.     

On Confidence Interval Construction for Establishing Equivalence of Two Binary-Outcome Treatments in Matched-Pair Studies in the Presence of Incomplete Data

Matched-pair design is often adopted in equivalence or non-inferiority trials to increase the efficiency of binary-outcome treatment comparison. Briefly, subjects are required to participate in two binary-outcome treatments (e.g., old and new treatments via crossover design) under study. To establish the equivalence between the two treatments at the α significance level, a (1−α)100% confidence interval for the correlated proportion difference is constructed and determined if it is entirely lying in the interval (−δ ₀,δ ₀) for some clinically acceptable threshold δ ₀ (e.g., 0.05). Nonetheless, some subjects may not be able to go through both treatments in practice and incomplete data thus arise. In this article, a hybrid method for confidence interval construction for correlated rate difference is proposed to establish equivalence between two treatments in matched-pair studies in the presence of incomplete data. The basic idea is to recover variance estimates from readily available confidence limits for single parameters. We compare the hybrid Agresti–Coull, Wilson score and Jeffreys confidence intervals with the asymptotic Wald and score confidence intervals with respect to their empirical coverage probabilities, expected confidence widths, ratios of left non-coverage probability, and total non-coverage probability. Our simulation studies suggest that the Agresti–Coull hybrid confidence intervals is better than the score-test-based and likelihood-ratio-based confidence interval in small to moderate sample sizes in the sense that the hybrid confidence interval controls its true coverage probabilities around the pre-assigned coverage level well and yield shorter expected confidence widths. A real medical equivalence trial with incomplete data is used to illustrate the proposed methodologies.     

No Longer Confidential: Estimating the Confidence of Individual Regression Predictions

Quantitative predictions in computational life sciences are often based on regression models. The advent of machine learning has led to highly accurate regression models that have gained widespread acceptance. While there are statistical methods available to estimate the global performance of regression models on a test or training dataset, it is often not clear how well this performance transfers to other datasets or how reliable an individual prediction is–a fact that often reduces a user’s trust into a computational method. In analogy to the concept of an experimental error, we sketch how estimators for individual prediction errors can be used to provide confidence intervals for individual predictions. Two novel statistical methods, named CONFINE and CONFIVE, can estimate the reliability of an individual prediction based on the local properties of nearby training data. The methods can be applied equally to linear and non-linear regression methods with very little computational overhead. We compare our confidence estimators with other existing confidence and applicability domain estimators on two biologically relevant problems (MHC–peptide binding prediction and quantitative structure-activity relationship (QSAR)). Our results suggest that the proposed confidence estimators perform comparable to or better than previously proposed estimation methods. Given a sufficient amount of training data, the estimators exhibit error estimates of high quality. In addition, we observed that the quality of estimated confidence intervals is predictable. We discuss how confidence estimation is influenced by noise, the number of features, and the dataset size. Estimating the confidence in individual prediction in terms of error intervals represents an important step from plain, non-informative predictions towards transparent and interpretable predictions that will help to improve the acceptance of computational methods in the biological community.     

Variance calculations and confidence intervals for estimates of the attributable risk based on logistic models

The attributable risk (AR), defined as AR = [Pr(disease) - Pr(disease/no exposure)]/Pr(disease), measures the proportion of disease risk that is attributable to an exposure. Recently Bruzzi et al. (1985, American Journal of Epidemiology 122, 904-914) presented point estimates of AR based on logistic models for case-control data to allow for confounding factors and secondary exposures. To produce confidence intervals, we derived variance estimates for AR under the logistic model and for various designs for sampling controls. Calculations for discrete exposure and confounding factors require covariances between estimates of the risk parameters of the logistic model and the proportions of cases with given levels of exposure and confounding factors. These covariances are estimated from Taylor series expansions applied to implicit functions. Similar calculations for continuous exposures are derived using influence functions. Simulations indicate that those asymptotic procedures yield reliable variance estimates and confidence intervals with near nominal coverage. An example illustrates the usefulness of variance calculations in selecting a logistic model that is neither so simplified as to exhibit systematic lack of fit nor so complicated as to inflate the variance of the estimate of AR.     

Estimating population trends using population viability analyses for the conservation ofCapra pyrenaica

Large herbivore populations can suffer important oscillations with considerable effects on ecosystem functions and services, yet our capacity to predict population fate is limited and conditional upon the availability of data. This study investigated the interannual variation in the growth rate of populations ofCapra pyrenaica Schinz, 1838, and its extinction risk by comparing the dynamics of populations that were stable for more than two decades (Gredos and Tortosa-Beceite), populations that had increased recently (Tejeda-Almijara), and populations that were in decline (Cazorla-Segura) or extinct (the Pyrenees population; hereafter, bucardo). To estimate quasi-extinction threshold assessments (50% of population extinct in this study), which have implications for the conservation of the species, we used empirical data and the predictions derived from several theoretical models. The results indicate that when variance of log population growth rate reaches a specific threshold, the probability of quasi-extinction increased drastically. ForC. pyrenaica, we recommend keeping population variance < 0.05, which will reduce the likelihood that the irruptive oscillations caused by environmental and demographic stochasticity will put the population at risk. Models to predict the dynamics ofC. pyrenaica populations should incorporate temporal stochasticity because, in this study, it strongly increased the likelihood that a population declined.     

Threshold and other departures from linear-quadratic curvature in the non-cancer mortality dose-response curve in the Japanese atomic bomb survivors

Recently released data on non-cancer mortality in Japanese atomic bomb survivors are analysed using a variety of generalised relative risk models that take account of errors in estimates of dose to assess the dose-response at low doses. If linear-threshold, quadratic-threshold or linear-quadratic-threshold relative risk models (the dose-response is assumed to be linear, quadratic or linear-quadratic above the threshold, respectively) are fitted to the non-cancer data there are no statistically significant (p>0.10) indications of threshold departures from linearity, quadratic curvature or linear-quadratic curvature. These findings are true irrespective of the assumed magnitude of dosimetric error, between 25%–45% geometric standard deviations. In general, increasing the assumed magnitude of dosimetric error had little effect on the central estimates of the threshold, but somewhat widened the associated confidence intervals. If a power of dose model is fitted, there is little evidence (p>0.10) that the power of dose in the dose-response is statistically significantly different from 1, again irrespective of the assumed magnitude of dosimetric errors in the range 25%–45%. Again, increasing the size of the errors resulted in wider confidence intervals on the power of dose, without marked effect on the central estimates. In general these findings remain true for various non-cancer disease subtypes.