Probabilistic methods for addressing uncertainty and variability in biological models: application to a toxicokinetic model

Population variability and uncertainty are important features of biological systems that must be considered when developing mathematical models for these systems. In this paper we present probability-based parameter estimation methods that account for such variability and uncertainty. Theoretical results that establish well-posedness and stability for these methods are discussed. A probabilistic parameter estimation technique is then applied to a toxicokinetic model for trichloroethylene using several types of simulated data. Comparison with results obtained using a standard, deterministic parameter estimation method suggests that the probabilistic methods are better able to capture population variability and uncertainty in model parameters.     

Importance of dose‐response model form in probabilistic risk assessment: A case study of health effects from methylmercury in fish

We compared the effect of uncertainty in dose‐response model form on health risk estimates to the effect of uncertainty and variability in exposure. We used three different dose‐response models to characterize neurological effects in children exposed in utero to methylmercury, and applied these models to calculate risks to a native population exposed to potentially contaminated fish from a reservoir in British Columbia. Uncertainty in model form was explicitly incorporated into the risk estimates. The selection of dose‐response model strongly influenced both mean risk estimates and distributions of risk, and had a much greater impact than altering exposure distributions. We conclude that incorporating uncertainty in dose‐response model form is at least as important as accounting for variability and uncertainty in exposure parameters in probabilistic risk assessment.     

Characterization and Simulation of Uncertain Frequency Distributions: Effects of Distribution Choice,Variability, Uncertainty,and Parameter Dependence

We use bootstrap simulation to characterize uncertainty in parametric distributions, including Normal, Lognormal, Gamma, Weibull, and Beta, commonly used to represent variability in probabilistic assessments. Bootstrap simulation enables one to estimate sampling distributions for sample statistics, such as distribution parameters, even when analytical solutions are not available. Using a two-dimensional framework for both uncertainty and variability, uncertainties in cumulative distribution functions were simulated. The mathematical properties of uncertain frequency distributions were evaluated in a series of case studies during which the parameters of each type of distribution were varied for sample sizes of 5, 10, and 20. For positively skewed distributions such as Lognormal, Weibull, and Gamma, the range of uncertainty is widest at the upper tail of the distribution. For symmetric unbounded distributions, such as Normal, the uncertainties are widest at both tails of the distribution. For bounded distributions, such as Beta, the uncertainties are typically widest in the central portions of the distribution. Bootstrap simulation enables complex dependencies between sampling distributions to be captured. The effects of uncertainty, variability, and parameter dependencies were studied for several generic functional forms of models, including models in which two-dimensional random variables are added, multiplied, and divided, to show the sensitivity of model results to different assumptions regarding model input distributions, ranges of variability, and ranges of uncertainty and to show the types of errors that may be obtained from mis-specification of parameter dependence. A total of 1,098 case studies were simulated. In some cases, counter-intuitive results were obtained. For example, the point value of the 95th percentile of uncertainty for the 95th percentile of variability of the product of four Gamma or Weibull distributions decreases as the coefficient of variation of each model input increases and, therefore, may not provide a conservative estimate. Failure to properly characterize parameter uncertainties and their dependencies can lead to orders-of-magnitude mis-estimates of both variability and uncertainty. In many cases, the numerical stability of two-dimensional simulation results was found to decrease as the coefficient of variation of the inputs increases. We discuss the strengths and limitations of bootstrap simulation as a method for quantifying uncertainty due to random sampling error.     

Extrapolation in Risk Assessment: Improving the Quantification of Uncertainty,and Improving Information to Reduce the Uncertainty

Risk assessments inevitably extrapolate from the known to the unknown. The resulting calculation of risk involves two fundamental kinds of uncertainty: uncertainty owing to intrinsically unpredictable (random) components of the future events, and uncertainty owing to imperfect prediction formulas (parameter uncertainty and error in model structure) that are used to predict the component that we think is predictable. Both types of uncertainty weigh heavily both in health and ecological risk assessments. Our first responsibility in conducting risk assessments is to ensure that the reported risks correctly reflect our actual level of uncertainty (of both types). The statistical methods that lend themselves to correct quantification of the uncertainty are also effective for combining different sources of information. One way to reduce uncertainty is to use all the available data. To further sharpen future risk assessments, it is useful to partition the uncertainty between the random component and the component due to parameter uncertainty, so that we can quantify the expected reduction in uncertainty that can be achieved by investing in a given amount of future data. An example is developed to illustrate the potential for use of comparative data, from toxicity testing on other species or other chemicals, to improve the estimates of low-effect concentration in a particular case with sparse case-specific data.     

Application of uncertainty and variability in LCA

As yet, the application of an uncertainty and variability analysis is not common practice in LCAs. A proper analysis will be facilitated when it is clear which types of uncertainties and variabilities exist in LCAs and which tools are available to deal with them. Therefore, a framework is developed to classify types of uncertainty and variability in LCAs. Uncertainty is divided in (1) parameter uncertainty, (2) model uncertainty, and (3) uncertainty due to choices, while variability covers (4) spatial variability, (5) temporal variability, and (6) variability between objects and sources. A tool to deal with parameter uncertainty and variability between objects and sources in both the inventory and the impact assessment is probabilistic simulation. Uncertainty due to choices can be dealt with in a scenario analysis or reduced by standardisation and peer review. The feasibility of dealing with temporal and spatial variability is limited, implying model uncertainty in LCAs. Other model uncertainties can be reduced partly by more sophisticated modelling, such as the use of non-linear inventory models in the inventory and multi media models in the characterisation phase.     

Role of Ecological Modeling in Risk Assessment

Ecological models are useful tools for evaluating the ecological significance of observed or predicted effects of toxic chemicals on individual organisms. Current risk estimation approaches using hazard quotients for individual-level endpoints have limited utility for assessing risks at the population, ecosystem, and landscape levels, which are the most relevant indicators for environmental management. In this paper, we define different types of ecological models, summarize their input and output variables, and present examples of the role of some recommended models in chemical risk assessments. A variety of population and ecosystem models have been applied successfully to evaluate ecological risks, including population viability of endangered species, habitat fragmentation, and toxic chemical issues. In particular, population models are widely available, and their value in predicting dynamics of natural populations has been demonstrated. Although data are often limited on vital rates and doseresponse functions needed for ecological modeling, accurate prediction of ecological effects may not be needed for all assessments. Often, a comparative assessment of risk (e.g., relative to baseline or reference) is of primary interest. Ecological modeling is currently a valuable approach for addressing many chemical risk assessment issues, including screening-level evaluations.     

The 3MRA Risk Assessment Framework—A Flexible Approach for Performing Multimedia,Multipathway, and Multireceptor Risk Assessments Under Uncertainty

A flexible framework for conducting nationwide multimedia, multipathway and multireceptor risk assessments (3MRA) under uncertainty was developed to estimate protective chemical concentration limits in a source area. The framework consists of two components: risk assessment and uncertainty analysis. The risk component utilizes linked source, fate/transport, exposure and risk assessment models to estimate the risk exposures for the receptors of concern. Both human and ecological receptors are included in the risk assessment framework. The flexibility of the framework is based on its ability to address problems varying in spatial scales from site-specific to regional and even national levels; and its ability to accommodate varying types of source, fate/transport, exposure and risk assessment models. The uncertainty component of the 3MRA framework is based on a two-stage Monte Carlo methodology. It allows the calculation of uncertainty in risk estimates, and the incorporation of the effects of uncertainty on the determination of regulatory concentration limits as a function of variability and uncertainty in input data, as well as potential errors in fate and transport and risk and exposure models. The framework can be adapted to handle a wide range of multimedia risk assessment problems. Two examples are presented to illustrate its use, and to demonstrate how regulatory decisions can be structured to incorporate the uncertainty in risk estimates.     

The Intersection of Independent Lies: Increasing Realism in Ecological Risk Assessment

In 1966, Levins presented a philosophical discussion on making inference about populations using clusters of models. In this article we provide an overview of model inference in ecological risk assessment, discuss the benefits and trade-offs of increasing model realism, show the similarities and differences between Levins' model clusters and those used in ecological risk assessment, and present how risk assessment models can incorporate Levins' ideas of truth through independent lies. Two aspects of Levins' philosophy are directly relevant to risk assessment. First, confidence in our interpretation of risk is increased when multiple risk assessments yield similar qualitative results. Second, model clusters should be evaluated to determine if they maximize precision, generality, or realism or a mix of the three. In the later case, the evaluation of each model will differ depending on whether it is more general, precise, or realistic relative to the other models used. We conclude that risk assessments can be strengthened using Levins' idea, but that Levins' caution that model outcome should not be mistaken for truth is still applicable.     

Ecology,Environmental Impact Statements,and Ecological Risk Assessment: A Brief Historical Perspective

Ecological risk assessment will continue to increase in importance as a conceptual and methodological basis for evaluating environmental impacts as required by the National Environmental Policy Act. Understanding the historical strengths and limitations of more traditional environmental assessments performed in support of the NEPA can facilitate the effective incorporation of ecological risk assessment into the NEPA process. Such integration will also benefit from a knowledge of the historical and continuing development of the ecological risk assessment process, as well as from a recognition of the contri butions from modern quantitative ecology and ecosystem science. Adopting a risk-based approach can improve the NEPA process by providing a framework for consistent and comprehensive ecological assessment and by providing a conceptual and methodological basis for addressing the varied uncertainties attendant to environmental assessments. The primary concern in integrating ecological risk assessment into the NEPA process is that ecological risk assessment not merely become a new name for traditional environmental impact assessments. While the integration of ecological risk assessment into the NEPA process occurs, it is important to begin to outline the next transition in environmental assessment capabilities. Operationally linking ecological risk assessment methods with formal decision models appears as a worthwhile objective in beginning this transition.     

Scientific data and theories for salmonellosis dose–response assessment

An “expansive” risk assessment approach is illustrated, characterizing dose–response relationships for salmonellosis in light of the full body of evidence for human and murine superorganisms. Risk assessments often require analysis of costs and benefits for supporting public health decisions. Decision-makers and the public need to understand uncertainty in such analyses for two reasons. Uncertainty analyses provide a range of possibilities within a framework of present scientific knowledge, thus helping to avoid undesirable consequences associated with the selected policies. And, it encourages the risk assessors to scrutinize all available data and models, thus helping avoid subjective or systematic errors. Without the full analysis of uncertainty, decisions could be biased by judgments based solely on default assumptions, beliefs, and statistical analyses of selected correlative data. Alternative data and theories that incorporate variability and heterogeneity for the human and murine superorganisms, particularly colonization resistance, are emerging as major influences for microbial risk assessment. Salmonellosis risk assessments are often based on conservative default models derived from selected sets of outbreak data that overestimate illness. Consequently, the full extent of uncertainty of estimates of annual number of illnesses is not incorporated in risk assessments and the presently used models may be incorrect.     

Contribution of crop model structure,parameters and climate projections to uncertainty in climate change impact assessments

Climate change impact assessments are plagued with uncertainties from many sources, such as climate projections or the inadequacies in structure and parameters of the impact model. Previous studies tried to account for the uncertainty from one or two of these. Here, we developed a triple‐ensemble probabilistic assessment using seven crop models, multiple sets of model parameters and eight contrasting climate projections together to comprehensively account for uncertainties from these three important sources. We demonstrated the approach in assessing climate change impact on barley growth and yield at Jokioinen, Finland in the Boreal climatic zone and Lleida, Spain in the Mediterranean climatic zone, for the 2050s. We further quantified and compared the contribution of crop model structure, crop model parameters and climate projections to the total variance of ensemble output using Analysis of Variance (ANOVA). Based on the triple‐ensemble probabilistic assessment, the median of simulated yield change was ?4% and +16%, and the probability of decreasing yield was 63% and 31% in the 2050s, at Jokioinen and Lleida, respectively, relative to 1981–2010. The contribution of crop model structure to the total variance of ensemble output was larger than that from downscaled climate projections and model parameters. The relative contribution of crop model parameters and downscaled climate projections to the total variance of ensemble output varied greatly among the seven crop models and between the two sites. The contribution of downscaled climate projections was on average larger than that of crop model parameters. This information on the uncertainty from different sources can be quite useful for model users to decide where to put the most effort when preparing or choosing models or parameters for impact analyses. We concluded that the triple‐ensemble probabilistic approach that accounts for the uncertainties from multiple important sources provide more comprehensive information for quantifying uncertainties in climate change impact assessments as compared to the conventional approaches that are deterministic or only account for the uncertainties from one or two of the uncertainty sources.     

The Use of Probabilistic Risk Assessment in Establishing Drinking Water Quality Objectives

There has been a trend in recent years toward the use of probabilistic methods for the analysis of uncertainty and variability in risk assessment. By developing a plausible distribution of risk, it is possible to obtain a more complete characterization of risk than is provided by either best estimates or upper limits. We describe in this paper a general framework for evaluating uncertainty and variability in risk estimation and outline how this framework can be used in the establishment of drinking water quality objectives. In addition to characterizing uncertainty and variability in risk, this framework also facilitates the identification of specific factors that contribute most to uncertainty and variability. The application of these probabilistic risk assessment methods is illustrated using tetrachloroethylene and trihalomethanes as examples.     

Uncertainty in assessing the impacts of global change with coupled dynamic species distribution and population models

Concern over rapid global changes and the potential for interactions among multiple threats are prompting scientists to combine multiple modelling approaches to understand impacts on biodiversity. A relatively recent development is the combination of species distribution models, land‐use change predictions, and dynamic population models to predict the relative and combined impacts of climate change, land‐use change, and altered disturbance regimes on species' extinction risk. Each modelling component introduces its own source of uncertainty through different parameters and assumptions, which, when combined, can result in compounded uncertainty that can have major implications for management. Although some uncertainty analyses have been conducted separately on various model components – such as climate predictions, species distribution models, land‐use change predictions, and population models – a unified sensitivity analysis comparing various sources of uncertainty in combined modelling approaches is needed to identify the most influential and problematic assumptions. We estimated the sensitivities of long‐run population predictions to different ecological assumptions and parameter settings for a rare and endangered annual plant species (Acanthomintha ilicifolia, or San Diego thornmint). Uncertainty about habitat suitability predictions, due to the choice of species distribution model, contributed most to variation in predictions about long‐run populations.     

Review of Information Resources to Support Human Exposure Assessment Models

Efforts to model human exposures to chemicals are growing more sophisticated and encompass increasingly complex exposure scenarios. The scope of such analyses has increased, growing from assessments of single exposure pathways to complex evaluations of aggregate or cumulative chemical exposures occurring within a variety of settings and scenarios. In addition, quantitative modeling techniques have evolved from simple deterministic analyses using single point estimates for each necessary input parameter to more detailed probabilistic analyses that can accommodate distributions of input parameters and assessment results. As part of an overall effort to guide development of a comprehensive framework for modeling human exposures to chemicals, available information resources needed to derive input parameters for human exposure assessment models were compiled and critically reviewed. Ongoing research in the area of exposure assessment parameters was also identified. The results of these efforts are summarized and other relevant information that will be needed to apply the available data in a comprehensive exposure model is discussed. Critical data gaps in the available information are also identified. Exposure assessment modeling and associated research would benefit from the collection of additional data as well as by enhancing the accessibility of existing and evolving information resources.     

A fish-based index of estuarine ecological quality incorporating information from both scientific fish survey and experts knowledge

In the Water Framework Directive (European Union) context, a multimetric fish based index is required to assess the ecological status of French estuarine water bodies. A first indicator called ELFI was developed, however similarly to most indicators, the method to combine the core metrics was rather subjective and this indicator does not provide uncertainty assessment. Recently, a Bayesian method to build indicators was developed and appeared relevant to select metrics sensitive to global anthropogenic pressure, to combine them objectively in an index and to provide a measure of uncertainty around the diagnostic. Moreover, the Bayesian framework is especially well adapted to integrate knowledge and information not included in surveys data. In this context, the present study used this Bayesian method to build a multimetric fish based index of ecological quality accounting for experts knowledge. The first step consisted in elaborating a questionnaire to collect assessments from different experts then in building relevant priors to summarize those assessments for each water body. Then, these priors were combined with surveys data in the index to complement the diagnosis of quality. Finally, a comparison between diagnoses using only fish data and using both information sources underlined experts knowledge contribution. Regarding the results, 68% of the diagnosis matched demonstrating that including experts knowledge thanks to the Bayesian framework confirmed or slightly modified the diagnosis provided by survey data but influenced uncertainty around the diagnostic and appeared especially relevant in terms of risk management.     

Bayesian Updating of Model-Based Risk Estimates Using Imperfect Public Health Surveillance Data

Model-based estimation of the human health risks resulting from exposure to environmental contaminants can be an important tool for structuring public health policy. Due to uncertainties in the modeling process, the outcomes of these assessments are usually probabilistic representations of a range of possible risks. In some cases, health surveillance data are available for the assessment population over all or a subset of the risk projection period and this additional information can be used to augment the model-based estimates. We use a Bayesian approach to update model-based estimates of health risks based on available health outcome data. Updated uncertainty distributions for risk estimates are derived using Monte Carlo sampling, which allows flexibility to model realistic situations including measurement error in the observable outcomes. We illustrate the approach by using imperfect public health surveillance data on lung cancer deaths to update model-based lung cancer mortality risk estimates in a population exposed to ionizing radiation from a uranium processing facility.     

Probabilistic Sensitivity Analysis of Public Health Risk Assessment from Contaminated Soil

Recently, there has been a growing trend toward using stochastic (probabilistic) methods in ecological and public health risk assessment. These methods are favored because they overcome the problem of compounded conservatism and allow the systematic consideration of uncertainty and variability typically encountered in risk assessment. This article demonstrates a new methodology for the analysis of uncertainty in risk assessment using the first-order reliability method (FORM). The reliability method is formulated such that the probability that incremental lifetime cancer risk exceeds a predefined threshold level is calculated. Furthermore, the stochastic sensitivity of this probability with respect to the random variables is provided. The emphasis is on exploring the different types of probabilistic sensitivity obtained through the reliability analysis. The method is applied to a case study given by Thompson et al. (1992) on cancer risk resulting from dermal contact with benzo(a)pyrene (BaP)-contaminated soils. The reliability results matched those of the Monte Carlo simulation method. On average, the Monte Carlo simulation method required about 35 times as many function evaluations as that of FORM to calculate the probability of exceeding the target risk level. The analysis emphasizes the significant impact that the uncertainty in cancer potency factor has on the probabilistic modeling results compared with other parameters.