The 3MRA Risk Assessment Framework—A Flexible Approach for Performing Multimedia,Multipathway, and Multireceptor Risk Assessments Under Uncertainty

A flexible framework for conducting nationwide multimedia, multipathway and multireceptor risk assessments (3MRA) under uncertainty was developed to estimate protective chemical concentration limits in a source area. The framework consists of two components: risk assessment and uncertainty analysis. The risk component utilizes linked source, fate/transport, exposure and risk assessment models to estimate the risk exposures for the receptors of concern. Both human and ecological receptors are included in the risk assessment framework. The flexibility of the framework is based on its ability to address problems varying in spatial scales from site-specific to regional and even national levels; and its ability to accommodate varying types of source, fate/transport, exposure and risk assessment models. The uncertainty component of the 3MRA framework is based on a two-stage Monte Carlo methodology. It allows the calculation of uncertainty in risk estimates, and the incorporation of the effects of uncertainty on the determination of regulatory concentration limits as a function of variability and uncertainty in input data, as well as potential errors in fate and transport and risk and exposure models. The framework can be adapted to handle a wide range of multimedia risk assessment problems. Two examples are presented to illustrate its use, and to demonstrate how regulatory decisions can be structured to incorporate the uncertainty in risk estimates.     

Regional Ecological Risk Assessment of a Near Shore Marine Environment: Cherry Point,WA

We conducted a regional ecological risk assessment for a near shore marine environment in northwestern Washington State using the Relative Risk Model. The objectives of this study were threefold: (1) to analyze cumulative impacts from multiple sources of chemical and non-chemical stressors in the near shore region and upland watersheds of Cherry Point (2) to determine the utility of Monte Carlo type uncertainty analysis in a rank-based regional risk assessment and (3) to investigate the effects of model habitat characterization on risk estimates. We used geographic information systems to compile and compare spatial data to determine ranks for sub-regions within the study area. By quantitatively combining ranks with exposure and effects filters, we estimated total relative risk between sub-regions and relative contributions of stressors. Finally, we used Monte Carlo analysis and an alternative ranking scheme to evaluate the effects of model and parameter uncertainty on risk predictions. The regional risk assessment results suggest the major contributors of risk are vessel traffic, upland urban and agricultural land use and shoreline recreational activities. This assessment demonstrated the applicability of regional risk assessment to marine near shore regions and the benefit of Monte Carlo analysis in describing uncertainty in a Relative Risk Model regional risk assessment.     

Analytical uncertainty propagation in life cycle inventory and impact assessment: application to an automobile front panel

Background, aim, and scope

Uncertainty information is essential for the proper use of life cycle assessment (LCA) and environmental assessments in decision making. So far, parameter uncertainty propagation has mainly been studied using Monte Carlo techniques that are relatively computationally heavy to conduct, especially for the comparison of multiple scenarios, often limiting its use to research or to inventory only. Furthermore, Monte Carlo simulations do not automatically assess the sensitivity and contribution to overall uncertainty of individual parameters. The present paper aims to develop and apply to both inventory and impact assessment an explicit and transparent analytical approach to uncertainty. This approach applies Taylor series expansions to the uncertainty propagation of lognormally distributed parameters.

Materials and methods

We first apply the Taylor series expansion method to analyze the uncertainty propagation of a single scenario, in which case the squared geometric standard deviation of the final output is determined as a function of the model sensitivity to each input parameter and the squared geometric standard deviation of each parameter. We then extend this approach to the comparison of two or more LCA scenarios. Since in LCA it is crucial to account for both common inventory processes and common impact assessment characterization factors among the different scenarios, we further develop the approach to address this dependency. We provide a method to easily determine a range and a best estimate of (a) the squared geometric standard deviation on the ratio of the two scenario scores, “A/B”, and (b) the degree of confidence in the prediction that the impact of scenario A is lower than B (i.e., the probability that A/B<1). The approach is tested on an automobile case study and resulting probability distributions of climate change impacts are compared to classical Monte Carlo distributions.

Results

The probability distributions obtained with the Taylor series expansion lead to results similar to the classical Monte Carlo distributions, while being substantially simpler; the Taylor series method tends to underestimate the 2.5% confidence limit by 1-11% and the 97.5% limit by less than 5%. The analytical Taylor series expansion easily provides the explicit contributions of each parameter to the overall uncertainty. For the steel front end panel, the factor contributing most to the climate change score uncertainty is the gasoline consumption (>75%). For the aluminum panel, the electricity and aluminum primary production, as well as the light oil consumption, are the dominant contributors to the uncertainty. The developed approach for scenario comparisons, differentiating between common and independent parameters, leads to results similar to those of a Monte Carlo analysis; for all tested cases, we obtained a good concordance between the Monte Carlo and the Taylor series expansion methods regarding the probability that one scenario is better than the other.

Discussion

The Taylor series expansion method addresses the crucial need of accounting for dependencies in LCA, both for common LCI processes and common LCIA characterization factors. The developed approach in Eq. 8, which differentiates between common and independent parameters, estimates the degree of confidence in the prediction that scenario A is better than B, yielding results similar to those found with Monte Carlo simulations.

Conclusions

The probability distributions obtained with the Taylor series expansion are virtually equivalent to those from a classical Monte Carlo simulation, while being significantly easier to obtain. An automobile case study on an aluminum front end panel demonstrated the feasibility of this method and illustrated its simultaneous and consistent application to both inventory and impact assessment. The explicit and innovative analytical approach, based on Taylor series expansions of lognormal distributions, provides the contribution to the uncertainty from each parameter and strongly reduces calculation time.     

Soil Ingestion Distributions for Monte Carlo Risk Assessment in Children

Monte Carlo environmental risk assessment requires estimates of the exposure distributions. An exposure of principal concern is often soil ingestion among children. We estimate the long-term (annual) average soil ingestion exposure distribution using daily soil ingestion estimates from children who participated in a mass-balance study at Anaconda, MT. The estimated distribution is accompanied by uncertainty estimates. The estimates take advantage of developing knowledge about bias in soil ingestion estimates and are robust. The estimates account for small particle size soil, use the median trace element estimate for subject days, account for the small sample variance of the median estimates, and use best linear unbiased predictors to estimate the cumulative long term soil ingestion distribution. Bootstrapping is used to estimate the uncertainty of the distribution estimates. The median soil ingestion is estimated as 24?mg/d (sd = 4?mg/d), with the 95 percentile soil ingestion estimated as 91?mg/d (sd = 16.6?mg/d). Strategies are discussed for use of these estimates in Monte Carlo risk assessment.     

Probabilistic Sensitivity Analysis of Public Health Risk Assessment from Contaminated Soil

Recently, there has been a growing trend toward using stochastic (probabilistic) methods in ecological and public health risk assessment. These methods are favored because they overcome the problem of compounded conservatism and allow the systematic consideration of uncertainty and variability typically encountered in risk assessment. This article demonstrates a new methodology for the analysis of uncertainty in risk assessment using the first-order reliability method (FORM). The reliability method is formulated such that the probability that incremental lifetime cancer risk exceeds a predefined threshold level is calculated. Furthermore, the stochastic sensitivity of this probability with respect to the random variables is provided. The emphasis is on exploring the different types of probabilistic sensitivity obtained through the reliability analysis. The method is applied to a case study given by Thompson et al. (1992) on cancer risk resulting from dermal contact with benzo(a)pyrene (BaP)-contaminated soils. The reliability results matched those of the Monte Carlo simulation method. On average, the Monte Carlo simulation method required about 35 times as many function evaluations as that of FORM to calculate the probability of exceeding the target risk level. The analysis emphasizes the significant impact that the uncertainty in cancer potency factor has on the probabilistic modeling results compared with other parameters.     

Estimating global arthropod species richness: refining probabilistic models using probability bounds analysis

A key challenge in the estimation of tropical arthropod species richness is the appropriate management of the large uncertainties associated with any model. Such uncertainties had largely been ignored until recently, when we attempted to account for uncertainty associated with model variables, using Monte Carlo analysis. This model is restricted by various assumptions. Here, we use a technique known as probability bounds analysis to assess the influence of assumptions about (1) distributional form and (2) dependencies between variables, and to construct probability bounds around the original model prediction distribution. The original Monte Carlo model yielded a median estimate of 6.1 million species, with a 90 % confidence interval of [3.6, 11.4]. Here we found that the probability bounds (p-bounds) surrounding this cumulative distribution were very broad, owing to uncertainties in distributional form and dependencies between variables. Replacing the implicit assumption of pure statistical independence between variables in the model with no dependency assumptions resulted in lower and upper p-bounds at 0.5 cumulative probability (i.e., at the median estimate) of 2.9–12.7 million. From here, replacing probability distributions with probability boxes, which represent classes of distributions, led to even wider bounds (2.4–20.0 million at 0.5 cumulative probability). Even the 100th percentile of the uppermost bound produced (i.e., the absolutely most conservative scenario) did not encompass the well-known hyper-estimate of 30 million species of tropical arthropods. This supports the lower estimates made by several authors over the last two decades.     

Two-Dimensional Monte Carlo Simulation and Beyond: A Comparison of Several Probabilistic Risk Assessment Methods Applied to a Superfund Site

Four different probabilistic risk assessment methods were compared using the data from the Sangamo Weston/Lake Hartwell Superfund site. These were one-dimensional Monte Carlo, two-dimensional Monte Carlo considering uncertainty in the concentration term, two-dimensional Monte Carlo considering uncertainty in ingestion rate, and microexposure event analysis. Estimated high-end risks ranged from 2.0×10^?4 to 3.3×10^?3. Microexposure event analysis produced a lower risk estimate than any of the other methods due to incorporation of time-dependent changes in the concentration term.     

Ten Years of the Relative Risk Model and Regional Scale Ecological Risk Assessment

It has been 10 years since the publication of the relative risk model (RRM) for regional scale ecological risk assessment. The approach has since been used successfully for a variety of freshwater, marine, and terrestrial environments in North America, South America, and Australia. During this period the types of stressors have been expanded to include more than contaminants. Invasive species, habitat loss, stream alteration and blockage, temperature, change in land use, and climate have been incorporated into the assessments. Major developments in the RRM have included the extensive use of geographical information systems, uncertainty analysis using Monte Carlo techniques, and its application to retrospective assessments to determine causation. The future uses of the RRM include assessments for forestry and conservation management, an increasing use in invasive species evaluation, and in sustainability. Developments in risk communication, the use of Bayesian approaches, and in uncertainty analyses are on the horizon.     

Treatments of Uncertainty and Variability in Ecological Risk Assessment of Single-Species Populations

The selection of the most appropriate model for an ecological risk assessment depends on the application, the data and resources available, the knowledge base of the assessor, the relevant endpoints, and the extent to which the model deals with uncertainty. Since ecological systems are highly variable and our knowledge of model input parameters is uncertain, it is important that models include treatments of uncertainty and variability, and that results are reported in this light. In this paper we discuss treatments of variation and uncertainty in a variety of population models. In ecological risk assessments, the risk relates to the probability of an adverse event in the context of environmental variation. Uncertainty relates to ignorance about parameter values, e.g., measurement error and systematic error. An assessment of the full distribution of risks, under variability and parameter uncertainty, will give the most comprehensive and flexible endpoint. In this paper we present the rationale behind probabilistic risk assessment, identify the sources of uncertainty relevant for risk assessment and provide an overview of a range of population models. While all of the models reviewed have some utility in ecology, some have more comprehensive treatments of uncertainty than others. We identify the models that allow probabilistic assessments and sensitivity analyses, and we offer recommendations for further developments that aim towards more comprehensive and reliable ecological risk assessments for populations.     

Calculating and Describing Uncertainty in Risk Assessment: The Bayesian Approach

Quantification of uncertainty associated with risk estimates is an important part of risk assessment. In recent years, use of second-order distributions, and two-dimensional simulations have been suggested for quantifying both variability and uncertainty. These approaches are better interpreted within the Bayesian framework. To help practitioners better use such methods and interpret the results, in this article, we describe propagation and interpretation of uncertainty in the Bayesian paradigm. We consider both the estimation problem where some summary measures of the risk distribution (e.g., mean, variance, or selected percentiles) are to be estimated, and the prediction problem, where the risk values for some specific individuals are to be predicted. We discuss some connections and differences between uncertainties in estimation and prediction problems, and present an interpretation of a decomposition of total variability/uncertainty into variability and uncertainty in terms of expected squared error of prediction and its reduction from perfect information. We also discuss the role of Monte Carlo methods in characterizing uncertainty. We explain the basic ideas using a simple example, and demonstrate Monte Carlo calculations using another example from the literature.     

Analytical Propagation of Uncertainty in Life Cycle Assessment Using Matrix Formulation

Inventory data and characterization factors in life cycle assessment (LCA) contain considerable uncertainty. The most common method of parameter uncertainty propagation to the impact scores is Monte Carlo simulation, which remains a resource‐intensive option—probably one of the reasons why uncertainty assessment is not a regular step in LCA. An analytical approach based on Taylor series expansion constitutes an effective means to overcome the drawbacks of the Monte Carlo method. This project aimed to test the approach on a real case study, and the resulting analytical uncertainty was compared with Monte Carlo results. The sensitivity and contribution of input parameters to output uncertainty were also analytically calculated. This article outlines an uncertainty analysis of the comparison between two case study scenarios. We conclude that the analytical method provides a good approximation of the output uncertainty. Moreover, the sensitivity analysis reveals that the uncertainty of the most sensitive input parameters was not initially considered in the case study. The uncertainty analysis of the comparison of two scenarios is a useful means of highlighting the effects of correlation on uncertainty calculation. This article shows the importance of the analytical method in uncertainty calculation, which could lead to a more complete uncertainty analysis in LCA practice.     

Quality-by-Design II: Application of Quantitative Risk Analysis to the Formulation of Ciprofloxacin Tablets

Qualitative risk assessment methods are often used as the first step to determining design space boundaries; however, quantitative assessments of risk with respect to the design space, i.e., calculating the probability of failure for a given severity, are needed to fully characterize design space boundaries. Quantitative risk assessment methods in design and operational spaces are a significant aid to evaluating proposed design space boundaries. The goal of this paper is to demonstrate a relatively simple strategy for design space definition using a simplified Bayesian Monte Carlo simulation. This paper builds on a previous paper that used failure mode and effects analysis (FMEA) qualitative risk assessment and Plackett-Burman design of experiments to identity the critical quality attributes. The results show that the sequential use of qualitative and quantitative risk assessments can focus the design of experiments on a reduced set of critical material and process parameters that determine a robust design space under conditions of limited laboratory experimentation. This approach provides a strategy by which the degree of risk associated with each known parameter can be calculated and allocates resources in a manner that manages risk to an acceptable level.     

Properties of cell death models calibrated and compared using Bayesian approaches

Using models to simulate and analyze biological networks requires principled approaches to parameter estimation and model discrimination. We use Bayesian and Monte Carlo methods to recover the full probability distributions of free parameters (initial protein concentrations and rate constants) for mass‐action models of receptor‐mediated cell death. The width of the individual parameter distributions is largely determined by non‐identifiability but covariation among parameters, even those that are poorly determined, encodes essential information. Knowledge of joint parameter distributions makes it possible to compute the uncertainty of model‐based predictions whereas ignoring it (e.g., by treating parameters as a simple list of values and variances) yields nonsensical predictions. Computing the Bayes factor from joint distributions yields the odds ratio (～20‐fold) for competing ‘direct’ and ‘indirect’ apoptosis models having different numbers of parameters. Our results illustrate how Bayesian approaches to model calibration and discrimination combined with single‐cell data represent a generally useful and rigorous approach to discriminate between competing hypotheses in the face of parametric and topological uncertainty.     

Scientific data and theories for salmonellosis dose–response assessment

An “expansive” risk assessment approach is illustrated, characterizing dose–response relationships for salmonellosis in light of the full body of evidence for human and murine superorganisms. Risk assessments often require analysis of costs and benefits for supporting public health decisions. Decision-makers and the public need to understand uncertainty in such analyses for two reasons. Uncertainty analyses provide a range of possibilities within a framework of present scientific knowledge, thus helping to avoid undesirable consequences associated with the selected policies. And, it encourages the risk assessors to scrutinize all available data and models, thus helping avoid subjective or systematic errors. Without the full analysis of uncertainty, decisions could be biased by judgments based solely on default assumptions, beliefs, and statistical analyses of selected correlative data. Alternative data and theories that incorporate variability and heterogeneity for the human and murine superorganisms, particularly colonization resistance, are emerging as major influences for microbial risk assessment. Salmonellosis risk assessments are often based on conservative default models derived from selected sets of outbreak data that overestimate illness. Consequently, the full extent of uncertainty of estimates of annual number of illnesses is not incorporated in risk assessments and the presently used models may be incorrect.     

Bayesian Updating of Model-Based Risk Estimates Using Imperfect Public Health Surveillance Data

Model-based estimation of the human health risks resulting from exposure to environmental contaminants can be an important tool for structuring public health policy. Due to uncertainties in the modeling process, the outcomes of these assessments are usually probabilistic representations of a range of possible risks. In some cases, health surveillance data are available for the assessment population over all or a subset of the risk projection period and this additional information can be used to augment the model-based estimates. We use a Bayesian approach to update model-based estimates of health risks based on available health outcome data. Updated uncertainty distributions for risk estimates are derived using Monte Carlo sampling, which allows flexibility to model realistic situations including measurement error in the observable outcomes. We illustrate the approach by using imperfect public health surveillance data on lung cancer deaths to update model-based lung cancer mortality risk estimates in a population exposed to ionizing radiation from a uranium processing facility.     

Species Sensitivity Distributions Revisited: A Critical Appraisal

We revisit the assumptions associated with the derivation and application of species sensitivity distributions (SSDs). Our questions are (1) Do SSDs clarify or obscure the setting of ecological effects thresholds for risk assessment? and (2) Do SSDs reduce or introduce uncertainty into risk assessment? Our conclusions are that if we could determine a community sensitivity distribution, this would provide a better estimate of an ecologically relevant effects threshold and therefore be an improvement for risk assessment. However, the distributions generated are typically based on haphazard collections of species and endpoints and by adjusting these to reflect more realistic trophic structures we show that effects thresholds can be shifted but in a direction and to an extent that is not predictable. Despite claims that the SSD approach uses all available data to assess effects, we demonstrate that in certain frequently used applications only a small fraction of the species going into the SSD determine the effects threshold. If the SSD approach is to lead to better risk assessments, improvements are needed in how the theory is put into practice. This requires careful definition of the risk assessment targets and of the species and endpoints selected for use in generating SSDs.     

Using Manufacturing Design Space Concepts for Stability Risk Assessment—Gabapentin NIPTE/FDA Case Study

A quantitative, model-based risk assessment process was evaluated using Bayesian parameter estimation to determine the posterior distribution of the probability of a model tablet formulation’s (gabapentin) ability to meet end-of-expiry stability criteria-based manufacturing controls. Experimental data was obtained from an FDA-supported, multi-year project that involved researchers at nine universities working collaboratively with industrial and governmental scientists under the leadership of the National Institute for Pharmaceutical Technology and Education (NITPE). The risk assessment process involved the development of a design space manufacturing model and shelf life stability model that shared stability-related critical quality attributes (CQAs). Monte Carlo simulations of the design space and shelf life models that uses model parameter uncertainty to estimate the probability of shelf life failure as a function of manufacturing control. The resultant linked design space and shelf life stability models were tested by comparing model predicted and observed long-term stability data generated under a variety of pilot scale production conditions.     

Uncertainty in LCA case study due to allocation approaches and life cycle impact assessment methods

Purpose

Uncertainty is present in many forms in life cycle assessment (LCA). However, little attention has been paid to analyze the variability that methodological choices have on LCA outcomes. To address this variability, common practice is to conduct a sensitivity analysis, which is sometimes treated only at a qualitative level. Hence, the purpose of this paper was to evaluate the uncertainty and the sensitivity in the LCA of swine production due to two methodological choices: the allocation approach and the life cycle impact assessment (LCIA) method.

Methods

We used a comparative case study of swine production to address uncertainty due to methodological choices. First, scenario variation through a sensitivity analysis of the approaches used to address the multi-functionality problem was conducted for the main processes of the system product, followed by an impact assessment using five LCIA methods at the midpoint level. The results from the sensitivity analysis were used to generate 10,000 independent simulations using the Monte Carlo method and then compared using comparison indicators in histogram graphics.

Results and discussion

Regardless of the differences between the absolute values of the LCA obtained due to the allocation approach and LCIA methods used, the overall ranking of scenarios did not change. The use of the substitution method to address the multi-functional processes in swine production showed the highest values for almost all of the impact categories, except for freshwater ecotoxicity; therefore, this method introduced the greater variations into our analysis. Regarding the variation of the LCIA method, for acidification, eutrophication, and freshwater ecotoxicity, the results were very sensitive. The uncertainty analysis with the Monte Carlo simulations showed a wide range of results and an almost equal probability of all the scenarios be the preferable option to decrease the impacts on acidification, eutrophication, and freshwater ecotoxicity. Considering the aggregate result variation across allocation approaches and LCIA methods, the uncertainty is too high to identify a statistically significant alternative.

Conclusions

The uncertainty analysis showed that performing only a sensitivity analysis could mislead the decision-maker with respect to LCA results; our analysis with the Monte Carlo simulation indicates no significant difference between the alternatives compared. Although the uncertainty in the LCA outcomes could not be decreased due to the wide range of possible results, to some extent, the uncertainty analysis can lead to a less uncertain decision-making by demonstrating the uncertainties between the compared alternatives.