Uncertainty analysis of forest carbon sink forecast with varying measurement errors: a data assimilation approach

Aims Accurate forecast of ecosystem states is critical for improving natural resource management and climate change mitigation. Assimilating observed data into models is an effective way to reduce uncertainties in ecological forecasting. However, influences of measurement errors on parameter estimation and forecasted state changes have not been carefully examined. This study analyzed the parameter identifiability of a process-based ecosystem carbon cycle model, the sensitivity of parameter estimates and model forecasts to the magnitudes of measurement errors and the information contributions of the assimilated data to model forecasts with a data assimilation approach.Methods We applied a Markov Chain Monte Carlo method to assimilate eight biometric data sets into the Terrestrial ECOsystem model. The data were the observations of foliage biomass, wood biomass, fine root biomass, microbial biomass, litter fall, litter, soil carbon and soil respiration, collected at the Duke Forest free-air CO₂ enrichment facilities from 1996 to 2005. Three levels of measurement errors were assigned to these data sets by halving and doubling their original standard deviations.Important findings Results showed that only less than half of the 30 parameters could be constrained, though the observations were extensive and the model was relatively simple. Higher measurement errors led to higher uncertainties in parameters estimates and forecasted carbon (C) pool sizes. The long-term predictions of the slow turnover pools were affected less by the measurement errors than those of fast turnover pools. Assimilated data contributed less information for the pools with long residence times in long-term forecasts. These results indicate the residence times of C pools played a key role in regulating propagation of errors from measurements to model forecasts in a data assimilation system. Improving the estimation of parameters of slow turnover C pools is the key to better forecast long-term ecosystem C dynamics.     

Predicting and correcting bias caused by measurement error in line transect sampling using multiplicative error models

Line transect sampling is one of the most widely used methods for animal abundance assessment. Standard estimation methods assume certain detection on the transect, no animal movement, and no measurement errors. Failure of the assumptions can cause substantial bias. In this work, the effect of error measurement on line transect estimators is investigated. Based on considerations of the process generating the errors, a multiplicative error model is presented and a simple way of correcting estimates based on knowledge of the error distribution is proposed. Using beta models for the error distribution, the effect of errors and of the proposed correction is assessed by simulation. Adequate confidence intervals for the corrected estimates are obtained using a bootstrap variance estimate for the correction and the delta method. As noted by Chen (1998, Biometrics 54, 899-908), even unbiased estimators of the distances might lead to biased density estimators, depending on the actual error distribution. In contrast with the findings of Chen, who used an additive model, unbiased estimation of distances, given a multiplicative model, lead to overestimation of density. Some error distributions result in observed distance distributions that make efficient estimation impossible, by removing the shoulder present in the original detection function. This indicates the need to improve field methods to reduce measurement error. An application of the new methods to a real data set is presented.     

Estimation in capture-recapture models when covariates are subject to measurement errors

We consider estimation problems in capture-recapture models when the covariates or the auxiliary variables are measured with errors. The naive approach, which ignores measurement errors, is found to be unacceptable in the estimation of both regression parameters and population size: it yields estimators with biases increasing with the magnitude of errors, and flawed confidence intervals. To account for measurement errors, we derive a regression parameter estimator using a regression calibration method. We develop modified estimators of the population size accordingly. A simulation study shows that the resulting estimators are more satisfactory than those from either the naive approach or the simulation extrapolation (SIMEX) method. Data from a bird species Prinia flaviventris in Hong Kong are analyzed with and without the assumption of measurement errors, to demonstrate the effects of errors on estimations.     

The use of Markovian metapopulation models: a comparison of three methods reducing the dimensionality of transition matrices

The use of Markovian models is an established way for deriving the complete distribution of the size of a population and the probability of extinction. However, computationally impractical transition matrices frequently result if this mathematical approach is applied to natural populations. Binning, or aggregating population sizes, has been used to permit a reduction in the dimensionality of matrices. Here, we present three deterministic binning methods and study the errors due to binning for a metapopulation model. Our results indicate that estimation errors of the investigated methods are not consistent and one cannot make generalizations about the quality of a method. For some compared output variables of populations studied, binning methods that caused a strong reduction in dimensionality of matrices resulted in better estimations than methods that produced a weaker reduction. The main problem with deterministic binning methods is that they do not properly take into account the stochastic population process itself. Straightforward usage of binning methods may lead to substantial errors in population-dynamical predictions.     

Dispersal kernel of green rice leafhopper estimated from truncated data

Characterizing dispersal kernels from truncated data is important for managing and predicting population dynamics. We used mark-recapture data from 10 previously published replicated experiments at three host plant development stages (seedling, tillering, and heading) to estimate parameters of the normal and exponential dispersal kernels for green rice leafhopper, Nephotettix cincticeps (Uhler). We compared classic statistical methods for estimating untruncated distribution parameters from truncated data with maximum likelihood (MLE) and the method of statistical moments for simulated and empirical data. Simulations showed that both methods provided accurate parameter estimates with similar precision. The method of moments is algebraically complex, but simple to calculate, while the MLE methods require numerical solutions of nonlinear equations. Simulations also showed that accurate, precise estimates of the parameters of the untruncated distributions could be attained even under severe truncation with sufficient numbers of recaptures. Both diffusivity and the exponential mean were higher with later plant growth stage, showing that insects moved farther and faster at the heading stage. Precision of the estimates was not strongly related to percent capture, size of the experimental field, or the number of leafhoppers captured. The leptokurtic exponential kernel fit the data better than the normal kernel for all the experiments. These results support an alternative explanation for the strong density-dependent population regulation of this species at the heading stage. Instead of leafhopper density per se, the increase in movement at this stage could integrate the populations in the separate fields, leveling densities throughout the landscape.     

NEE观测误差分布类型对陆地生态系统机理模型参数估计的影响——以长白山温带阔叶红松林为例

基于观测数据的陆地生态系统模型参数估计有助于提高模型的模拟和预测能力,降低模拟不确定性.在已有参数估计研究中,涡度相关技术测定的净生态系统碳交换量(NEE)数据的随机误差通常被假设为服从零均值的正态分布.然而近年来已有研究表明NEE数据的随机误差更服从双指数分布.为探讨NEE观测误差分布类型的不同选择对陆地生态系统机理模型参数估计以及碳通量模拟结果造成的差异,以长白山温带阔叶红松林为研究区域,采用马尔可夫链-蒙特卡罗方法,利用2003～2005年测定的NEE数据对陆地生态系统机理模型CEVSA2的敏感参数进行估计,对比分析了两种误差分布类型(正态分布和双指数分布)的参数估计结果以及碳通量模拟的差异.结果表明,基于正态观测误差模拟的总初级生产力和生态系统呼吸的年总量分别比基于双指数观测误差的模拟结果高61～86 g C m-2 a-1和107～116 g C m-2 a-1,导致前者模拟的NEE年总量较后者低29～47 g C m-2 a-1,特别在生长旺季期间有明显低估.在参数估计研究中,不能忽略观测误差的分布类型以及相应的目标函数的选择,它们的不合理设置可能对参数估计以及模拟结果产生较大影响.     

Characterization and Simulation of Uncertain Frequency Distributions: Effects of Distribution Choice,Variability, Uncertainty,and Parameter Dependence

We use bootstrap simulation to characterize uncertainty in parametric distributions, including Normal, Lognormal, Gamma, Weibull, and Beta, commonly used to represent variability in probabilistic assessments. Bootstrap simulation enables one to estimate sampling distributions for sample statistics, such as distribution parameters, even when analytical solutions are not available. Using a two-dimensional framework for both uncertainty and variability, uncertainties in cumulative distribution functions were simulated. The mathematical properties of uncertain frequency distributions were evaluated in a series of case studies during which the parameters of each type of distribution were varied for sample sizes of 5, 10, and 20. For positively skewed distributions such as Lognormal, Weibull, and Gamma, the range of uncertainty is widest at the upper tail of the distribution. For symmetric unbounded distributions, such as Normal, the uncertainties are widest at both tails of the distribution. For bounded distributions, such as Beta, the uncertainties are typically widest in the central portions of the distribution. Bootstrap simulation enables complex dependencies between sampling distributions to be captured. The effects of uncertainty, variability, and parameter dependencies were studied for several generic functional forms of models, including models in which two-dimensional random variables are added, multiplied, and divided, to show the sensitivity of model results to different assumptions regarding model input distributions, ranges of variability, and ranges of uncertainty and to show the types of errors that may be obtained from mis-specification of parameter dependence. A total of 1,098 case studies were simulated. In some cases, counter-intuitive results were obtained. For example, the point value of the 95th percentile of uncertainty for the 95th percentile of variability of the product of four Gamma or Weibull distributions decreases as the coefficient of variation of each model input increases and, therefore, may not provide a conservative estimate. Failure to properly characterize parameter uncertainties and their dependencies can lead to orders-of-magnitude mis-estimates of both variability and uncertainty. In many cases, the numerical stability of two-dimensional simulation results was found to decrease as the coefficient of variation of the inputs increases. We discuss the strengths and limitations of bootstrap simulation as a method for quantifying uncertainty due to random sampling error.     

Influences of error distributions of net ecosystem exchange on parameter estimation of a process-based terrestrial model: A case of broad-leaved Korean pine mixed forest in Changbaishan, China

Model predictions can be improved by parameter estimation from measurements. It was assumed that measurement errors of net ecosystem exchange (NEE) of CO₂ follow a normal distribution. However, recent studies have shown that errors in eddy covariance measurements closely follow a double exponential distribution. In this paper, we compared effects of different distributions of measurement errors of NEE data on parameter estimation. NEE measurements in the Changbaishan forest were assimilated into a process-based terrestrial ecosystem model. We used the Markov chain Monte Carlo method to derive probability density functions of estimated parameters. Our results showed that modeled annual total gross primary production (GPP) and ecosystem respiration (Re) using the normal error distribution were higher than those using the double exponential distribution by 61–86 gC m^?2 a^?1 and 107–116 gC m^?2 a^?1, respectively. As a result, modeled annual sum of NEE using the normal error distribution was lower by 29–47 gC m^?2 a^?1 than that using the double exponential error distribution. Especially, modeled daily NEE based on the normal distribution underestimated the strong carbon sink in the Changbaishan forest in the growing season. We concluded that types of measurement error distributions and corresponding cost functions can substantially influence the estimation of parameters and carbon fluxes.     

Quantification of Errors in Ordinal Outcome Scales Using Shannon Entropy: Effect on Sample Size Calculations

Objective

Clinical trial outcomes often involve an ordinal scale of subjective functional assessments but the optimal way to quantify results is not clear. In stroke, the most commonly used scale, the modified Rankin Score (mRS), a range of scores (“Shift”) is proposed as superior to dichotomization because of greater information transfer. The influence of known uncertainties in mRS assessment has not been quantified. We hypothesized that errors caused by uncertainties could be quantified by applying information theory. Using Shannon’s model, we quantified errors of the “Shift” compared to dichotomized outcomes using published distributions of mRS uncertainties and applied this model to clinical trials.

Methods

We identified 35 randomized stroke trials that met inclusion criteria. Each trial’s mRS distribution was multiplied with the noise distribution from published mRS inter-rater variability to generate an error percentage for “shift” and dichotomized cut-points. For the SAINT I neuroprotectant trial, considered positive by “shift” mRS while the larger follow-up SAINT II trial was negative, we recalculated sample size required if classification uncertainty was taken into account.

Results

Considering the full mRS range, error rate was 26.1%±5.31 (Mean±SD). Error rates were lower for all dichotomizations tested using cut-points (e.g. mRS 1; 6.8%±2.89; overall p<0.001). Taking errors into account, SAINT I would have required 24% more subjects than were randomized.

Conclusion

We show when uncertainty in assessments is considered, the lowest error rates are with dichotomization. While using the full range of mRS is conceptually appealing, a gain of information is counter-balanced by a decrease in reliability. The resultant errors need to be considered since sample size may otherwise be underestimated. In principle, we have outlined an approach to error estimation for any condition in which there are uncertainties in outcome assessment. We provide the user with programs to calculate and incorporate errors into sample size estimation.     

Bayesian Updating of Model-Based Risk Estimates Using Imperfect Public Health Surveillance Data

Model-based estimation of the human health risks resulting from exposure to environmental contaminants can be an important tool for structuring public health policy. Due to uncertainties in the modeling process, the outcomes of these assessments are usually probabilistic representations of a range of possible risks. In some cases, health surveillance data are available for the assessment population over all or a subset of the risk projection period and this additional information can be used to augment the model-based estimates. We use a Bayesian approach to update model-based estimates of health risks based on available health outcome data. Updated uncertainty distributions for risk estimates are derived using Monte Carlo sampling, which allows flexibility to model realistic situations including measurement error in the observable outcomes. We illustrate the approach by using imperfect public health surveillance data on lung cancer deaths to update model-based lung cancer mortality risk estimates in a population exposed to ionizing radiation from a uranium processing facility.     

Characterising Uncertainty in Expert Assessments: Encoding Heavily Skewed Judgements

When limited or no observed data are available, it is often useful to obtain expert knowledge about parameters of interest, including point estimates and the uncertainty around these values. However, it is vital to elicit this information appropriately in order to obtain valid estimates. This is particularly important when the experts’ uncertainty about these estimates is strongly skewed, for instance when their best estimate is the same as the lowest value they consider possible. Also this is important when interest is in the aggregation of elicited values. In this paper, we compare alternative distributions for describing such estimates. The distributions considered include the lognormal, mirror lognormal, Normal and scaled Beta. The case study presented here involves estimation of the number of species in coral reefs, which requires eliciting counts within broader taxonomic groups, with highly skewed uncertainty estimates. This paper shows substantial gain in using the scaled Beta distribution, compared with Normal or lognormal distributions. We demonstrate that, for this case study on counting species, applying the novel encoding methodology developed in this paper can facilitate the acquisition of more rigorous estimates of (hierarchical) count data and credible bounds. The approach can also be applied to the more general case of enumerating a sampling frame via elicitation.     

Bayesian analysis for nonlinear regression model under skewed errors,with application in growth curves

We have considered a Bayesian approach for the nonlinear regression model by replacing the normal distribution on the error term by some skewed distributions, which account for both skewness and heavy tails or skewness alone. The type of data considered in this paper concerns repeated measurements taken in time on a set of individuals. Such multiple observations on the same individual generally produce serially correlated outcomes. Thus, additionally, our model does allow for a correlation between observations made from the same individual. We have illustrated the procedure using a data set to study the growth curves of a clinic measurement of a group of pregnant women from an obstetrics clinic in Santiago, Chile. Parameter estimation and prediction were carried out using appropriate posterior simulation schemes based in Markov Chain Monte Carlo methods. Besides the deviance information criterion (DIC) and the conditional predictive ordinate (CPO), we suggest the use of proper scoring rules based on the posterior predictive distribution for comparing models. For our data set, all these criteria chose the skew‐t model as the best model for the errors. These DIC and CPO criteria are also validated, for the model proposed here, through a simulation study. As a conclusion of this study, the DIC criterion is not trustful for this kind of complex model.     

On Method of Moments Estimation in Linear Mixed Effects Models with Measurement Error on Covariates and Response with Application to a Longitudinal Study of Gene-Environment Interaction

We study a linear mixed effects model for longitudinal data, where the response variable and covariates with fixed effects are subject to measurement error. We propose a method of moment estimation that does not require any assumption on the functional forms of the distributions of random effects and other random errors in the model. For a classical measurement error model we apply the instrumental variable approach to ensure identifiability of the parameters. Our methodology, without instrumental variables, can be applied to Berkson measurement errors. Using simulation studies, we investigate the finite sample performances of the estimators and show the impact of measurement error on the covariates and the response on the estimation procedure. The results show that our method performs quite satisfactory, especially for the fixed effects with measurement error (even under misspecification of measurement error model). This method is applied to a real data example of a large birth and child cohort study.     

Process and measurement errors of population size: their mutual effects on precision and bias of estimates for demographic parameters

Knowing the parameters of population growth and regulation is fundamental for answering many ecological questions and the successful implementation of conservation strategies. Moreover, detecting a population trend is often a legal obligation. Yet, inherent process and measurement errors aggravate the ability to estimate these parameters from population time-series. We use numerical simulations to explore how the lengths of the time-series, process and measurement error influence estimates of demographic parameters. We first generate time-series of population sizes with given demographic parameters for density-dependent stochastic population growth, but assume that these population sizes are estimated with measurement errors. We then fit parameters for population growth, habitat capacity, total error and long-term trends to the ‘measured’ time-series data using non-linear regression. The length of the time-series and measurement error introduce a substantial bias in the estimates for population growth rate and to a lesser degree on estimates for habitat capacity, while process error has little effect on parameter bias. The total error term of the statistical model is dominated by process error as long as the latter is larger than the measurement error. A decline in population size is difficult to document as soon as either error becomes moderate, trends are not very pronounced, and time-series are short (<10–15 seasons). Detecting an annual decline of 1% within 6-year reporting periods, as required for the European Union for the species of Community Interest, appears unachievable.     

A comparison of methods to estimate cross-environment genetic correlations

Advanced techniques for quantitative genetic parameter estimation may not always be necessary to answer broad genetic questions. However, simpler methods are often biased, and the extent of this determines their usefulness. In this study we compare family mean correlations to least squares and restricted error maximum likelihood (REML) variance component approaches to estimating cross-environment genetic correlations. We analysed empirical data from studies where both types of estimates were made, and from studies in our own laboratories. We found that the agreement between estimates was better when full-sib rather than half-sib estimates of cross-environment genetic correlations were used and when mean family size increased. We also note biases in REML estimation that may be especially important when testing to see if correlations differ from 0 or 1. We conclude that correlations calculated from family means can be used to test for the presence of genetic correlations across environments, which is sufficient for some research questions. Variance component approaches should be used when parameter estimation is the objective, or if the goal is anything other than determining broad patterns.     

Thyroid cancer following scalp irradiation: a reanalysis accounting for uncertainty in dosimetry

In the 1940s and 1950s, over 20,000 children in Israel were treated for tinea capitis (scalp ringworm) by irradiation to induce epilation. Follow-up studies showed that the radiation exposure was associated with the development of malignant thyroid neoplasms. Despite this clear evidence of an effect, the magnitude of the dose-response relationship is much less clear because of probable errors in individual estimates of dose to the thyroid gland. Such errors have the potential to bias dose-response estimation, a potential that was not widely appreciated at the time of the original analyses. We revisit this issue, describing in detail how errors in dosimetry might occur, and we develop a new dose-response model that takes the uncertainties of the dosimetry into account. Our model for the uncertainty in dosimetry is a complex and new variant of the classical multiplicative Berkson error model, having components of classical multiplicative measurement error as well as missing data. Analysis of the tinea capitis data suggests that measurement error in the dosimetry has only a negligible effect on dose-response estimation and inference as well as on the modifying effect of age at exposure.     

On observation distributions for state space models of population survey data

1.?State space models are starting to replace more simple time series models in analyses of temporal dynamics of populations that are not perfectly censused. By simultaneously modelling both the dynamics and the observations, consistent estimates of population dynamical parameters may be obtained. For many data sets, the distribution of observation errors is unknown and error models typically chosen in an ad-hoc manner. 2.?To investigate the influence of the choice of observation error on inferences, we analyse the dynamics of a replicated time series of red kangaroo surveys using a state space model with linear state dynamics. Surveys were performed through aerial counts and Poisson, overdispersed Poisson, normal and log-normal distributions may all be adequate for modelling observation errors for the data. We fit each of these to the data and compare them using AIC. 3.?The state space models were fitted with maximum likelihood methods using a recent importance sampling technique that relies on the Kalman filter. The method relaxes the assumption of Gaussian observation errors required by the basic Kalman filter. Matlab code for fitting linear state space models with Poisson observations is provided. 4.?The ability of AIC to identify the correct observation model was investigated in a small simulation study. For the parameter values used in the study, without replicated observations, the correct observation distribution could sometimes be identified but model selection was prone to misclassification. On the other hand, when observations were replicated, the correct distribution could typically be identified. 5.?Our results illustrate that inferences may differ markedly depending on the observation distributions used, suggesting that choosing an adequate observation model can be critical. Model selection and simulations show that for the models and parameter values in this study, a suitable observation model can typically be identified if observations are replicated. Model selection and replication of observations, therefore, provide a potential solution when the observation distribution is unknown.     

Allowing for random errors in radiation dose estimates for the atomic bomb survivor data

The presence of random errors in the individual radiation dose estimates for the A-bomb survivors causes underestimation of radiation effects in dose-response analyses, and also distorts the shape of dose-response curves. Statistical methods are presented which will adjust for these biases, provided that a valid statistical model for the dose estimation errors is used. Emphasis is on clarifying some rather subtle statistical issues. For most of this development the distinction between radiation dose and exposure is not critical. The proposed methods involve downward adjustment of dose estimates, but this does not imply that the dosimetry system is faulty. Rather, this is a part of the dose-response analysis required to remove biases in the risk estimates. The primary focus of this report is on linear dose-response models, but methods for linear-quadratic models are also considered briefly. Some plausible models for the dose estimation errors are considered, which have typical errors in a range of 30-40% of the true values, and sensitivity analysis of the resulting bias corrections is provided. It is found that for these error models the resulting estimates of excess cancer risk based on linear models are about 6-17% greater than estimates that make no allowance for dose estimation errors. This increase in risk estimates is reduced to about 4-11% if, as has often been done recently, survivors with dose estimates above 4 Gy are eliminated from the analysis.     

A resolution of the ascertainment sampling problem. II. Generalizations and numerical results

The ascertainment problem arises when families are sampled by a nonrandom process and some assumption about this sampling process must be made in order to estimate genetic parameters. Under classical ascertainment assumptions, estimation of genetic parameters cannot be separated from estimation of the parameters of the ascertainment process, so that any misspecification of the ascertainment process causes biases in estimation of the genetic parameters. Ewens and Shute proposed a resolution to this problem, involving conditioning the likelihood of the sample on the part of the data which is "relevant to ascertainment." The usefulness of this approach can only be assessed by examining the properties (in particular, bias and standard error) of the estimates which arise by using it for a wide range of parameter values and family size distributions and then comparing these biases and standard errors with those arising under classical ascertainment procedures. These comparisons are carried out in the present paper, and we also compare the proposed method with procedures which condition on, or ignore, parts of the data.     

Shared uncertainty in measurement error problems, with application to Nevada Test Site fallout data

In radiation epidemiology, it is often necessary to use mathematical models in the absence of direct measurements of individual doses. When complex models are used as surrogates for direct measurements to estimate individual doses that occurred almost 50 years ago, dose estimates will be associated with considerable error, this error being a mixture of (a) classical measurement error due to individual data such as diet histories and (b) Berkson measurement error associated with various aspects of the dosimetry system. In the Nevada Test Site(NTS) Thyroid Disease Study, the Berkson measurement errors are correlated within strata. This article concerns the development of statistical methods for inference about risk of radiation dose on thyroid disease, methods that account for the complex error structure inherence in the problem. Bayesian methods using Markov chain Monte Carlo and Monte-Carlo expectation-maximization methods are described, with both sharing a key Metropolis-Hastings step. Regression calibration is also considered, but we show that regression calibration does not use the correlation structure of the Berkson errors. Our methods are applied to the NTS Study, where we find a strong dose-response relationship between dose and thyroiditis. We conclude that full consideration of mixtures of Berkson and classical uncertainties in reconstructed individual doses are important for quantifying the dose response and its credibility/confidence interval. Using regression calibration and expectation values for individual doses can lead to a substantial underestimation of the excess relative risk per gray and its 95% confidence intervals.