Optimizing experimental design for comparing models of brain function

This article presents the first attempt to formalize the optimization of experimental design with the aim of comparing models of brain function based on neuroimaging data. We demonstrate our approach in the context of Dynamic Causal Modelling (DCM), which relates experimental manipulations to observed network dynamics (via hidden neuronal states) and provides an inference framework for selecting among candidate models. Here, we show how to optimize the sensitivity of model selection by choosing among experimental designs according to their respective model selection accuracy. Using Bayesian decision theory, we (i) derive the Laplace-Chernoff risk for model selection, (ii) disclose its relationship with classical design optimality criteria and (iii) assess its sensitivity to basic modelling assumptions. We then evaluate the approach when identifying brain networks using DCM. Monte-Carlo simulations and empirical analyses of fMRI data from a simple bimanual motor task in humans serve to demonstrate the relationship between network identification and the optimal experimental design. For example, we show that deciding whether there is a feedback connection requires shorter epoch durations, relative to asking whether there is experimentally induced change in a connection that is known to be present. Finally, we discuss limitations and potential extensions of this work.     

生态效益评价内容和评价指标筛选

生态效益评价的重要性已经广为人知,但评价内容和指标不统一,大大影响了评价结果的可信度。基本上是各说各有理,很难相互比较。在分析国内外相关生态系统评价方法基础上,提出生态效益评价框架,强调生态效益与经济效益和社会效益共同构成了人类社会的价值判断标准和决策依据,是经济效益和社会效益的基础,应该包括生态系统整体贡献,考虑与生态系统的动态变化的关系;并探讨了生态效益评价指标筛选的原则(关联性、灵敏性、层级性、决策导向性、代表性、可行性、独立性及经济适用性和社会可接受性);提出了生态效益指标筛选的多准则综合法和生态效益评价指标检验的3个标准(可定量化、专一化和震撼性)。本研究将为生态效益指标体系的构建提供重要参考。     

An Approach for Incorporating Information on Chemical Availability in Soils into Risk Assessment and Risk-Based Decision Making,Prepared by: The New England Environmentally Acceptable Endpoints Workgroup

A regional workgroup comprised of individuals from regulatory agencies, uni versities, and consulting companies was formed to develop an approach for incor porating information on chemical availability in soils into risk assessment and risk based decision making. The approach consists of the following decision framework for including information on chemical availability: (1) Determine the usefulness of incorporating information on bioavailability; (2) Identify information needs from a conceptual model of exposure for the site and from exposure pathways judged critical to the assessment; (3) Identify soil factors that affect bioavailability; (4) Determine the type or form of information (measures and/or models) that can be used within the risk assessment and risk management process; (5) Select methods (measures and/or models) based on the “weight of evidence” or strength of the bioavailability information they will provide and how that information will be used for risk assessment and risk based decision making; (6) Incorporate information into the risk assessment and risk based decision making. These fac tors can be integrated into existing risk based approaches for site management such as Superfund, state approaches, and the ASTM Risk Based Corrective Action Process (RBCA). Consistent with risk assessment guidance, an assessment of chemical availability in soils must consider current as well as reasonably foresee able conditions. The approach recognizes that information on chemical availabil ity is contextual and depends on the receptor and pathway. Further, the value of information depends on how well it is accepted and/or validated for use in regulatory decision making. The workgroup identified four principles for select ing methods (measures and/or models) for obtaining information on chemical availability and for evaluating information on chemical availability for use in risk assessments: (1) soil chemical relevance, (2) pathway relevance, (3) receptor relevance, and (4) acceptance of the method.     

Dynamic modelling and analysis of biochemical networks: mechanism-based models and model-based experiments

Systems biology applies quantitative, mechanistic modelling to study genetic networks, signal transduction pathways and metabolic networks. Mathematical models of biochemical networks can look very different. An important reason is that the purpose and application of a model are essential for the selection of the best mathematical framework. Fundamental aspects of selecting an appropriate modelling framework and a strategy for model building are discussed. Concepts and methods from system and control theory provide a sound basis for the further development of improved and dedicated computational tools for systems biology. Identification of the network components and rate constants that are most critical to the output behaviour of the system is one of the major problems raised in systems biology. Current approaches and methods of parameter sensitivity analysis and parameter estimation are reviewed. It is shown how these methods can be applied in the design of model-based experiments which iteratively yield models that are decreasingly wrong and increasingly gain predictive power.     

Bayesian natural selection and the evolution of perceptual systems

In recent years, there has been much interest in characterizing statistical properties of natural stimuli in order to better understand the design of perceptual systems. A fruitful approach has been to compare the processing of natural stimuli in real perceptual systems with that of ideal observers derived within the framework of Bayesian statistical decision theory. While this form of optimization theory has provided a deeper understanding of the information contained in natural stimuli as well as of the computational principles employed in perceptual systems, it does not directly consider the process of natural selection, which is ultimately responsible for design. Here we propose a formal framework for analysing how the statistics of natural stimuli and the process of natural selection interact to determine the design of perceptual systems. The framework consists of two complementary components. The first is a maximum fitness ideal observer, a standard Bayesian ideal observer with a utility function appropriate for natural selection. The second component is a formal version of natural selection based upon Bayesian statistical decision theory. Maximum fitness ideal observers and Bayesian natural selection are demonstrated in several examples. We suggest that the Bayesian approach is appropriate not only for the study of perceptual systems but also for the study of many other systems in biology.     

Guidelines for Guidelines: Are They Up to the Task? A Comparative Assessment of Clinical Practice Guideline Development Handbooks

Objectives

We conducted a comparative review of clinical practice guideline development handbooks. We aimed to identify the main guideline development tasks, assign weights to the importance of each task using expert opinions and identify the handbooks that provided a comprehensive coverage of the tasks.

Methods

We systematically searched and included handbooks published (in English language) by national, international or professional bodies responsible for evidenced-based guideline development. We reviewed the handbooks to identify the main guideline development tasks and scored each handbook for each task from 0 (the handbook did not mention the task) to 2 (the task suitably addressed and explained), and calculated a weighted score for each handbook. The tasks included in over 75% of the handbooks were considered as ‘necessary’ tasks.

Result

Nineteen guideline development handbooks and twenty seven main tasks were identified. The guideline handbooks’ weighted scores ranged from 100 to 220. Four handbooks scored over 80% of the maximum possible score, developed by the National Institute for Health and Clinical Excellence, Swiss Centre for International Health, Scottish Intercollegiate Guidelines Network and World Health Organization. Necessary tasks were: selecting the guideline topic, determining the guideline scope, identifying relevant existing guidelines, involving the consumers, forming guideline development group,, developing clinical questions, systematic search for evidence, selecting relevant evidence, appraising identifies research evidence, making group decision, grading available evidence, creating recommendations, final stakeholder consultation, guideline implementation strategies, updating recommendations and correcting potential errors.

Discussion

Adequate details for evidence based development of guidelines were still lacking from many handbooks. The tasks relevant to ethical issues and piloting were missing in most handbooks. The findings help decision makers in identifying the necessary tasks for guideline development, provide an updated comparative list of guideline development handbooks, and provide a checklist to assess the comprehensiveness of guideline development processes.     

Evaluation of Sampling-Based Methods for Sensitivity Analysis: Case Study for the E. coli Food Safety Process Risk Model

This article evaluates selected sensitivity analysis methods applicable to risk assessment models with two-dimensional probabilistic frameworks, using a microbial food safety process risk model as a test-bed. Six sampling-based sensitivity analysis methods were evaluated including Pearson and Spearman correlation, sample and rank linear regression, and sample and rank stepwise regression. In a two-dimensional risk model, the identification of key controllable inputs that can be priorities for risk management can be confounded by uncertainty. However, despite uncertainty, results show that key inputs can be distinguished from those that are unimportant, and inputs can be grouped into categories of similar levels of importance. All selected methods are capable of identifying unimportant inputs, which is helpful in that efforts to collect data to improve the assessment or to focus risk management strategies can be prioritized elsewhere. Rank-based methods provided more robust insights with respect to the key sources of variability in that they produced narrower ranges of uncertainty for sensitivity results and more clear distinctions when comparing the importance of inputs or groups of inputs. Regression-based methods have advantages over correlation approaches because they can be configured to provide insight regarding interactions and nonlinearities in the model.     

Methods for selecting fixed-effect models for heterogeneous codon evolution,with comments on their application to gene and genome data

Background

Models of codon evolution have proven useful for investigating the strength and direction of natural selection. In some cases, a priori biological knowledge has been used successfully to model heterogeneous evolutionary dynamics among codon sites. These are called fixed-effect models, and they require that all codon sites are assigned to one of several partitions which are permitted to have independent parameters for selection pressure, evolutionary rate, transition to transversion ratio or codon frequencies. For single gene analysis, partitions might be defined according to protein tertiary structure, and for multiple gene analysis partitions might be defined according to a gene's functional category. Given a set of related fixed-effect models, the task of selecting the model that best fits the data is not trivial.

Results

In this study, we implement a set of fixed-effect codon models which allow for different levels of heterogeneity among partitions in the substitution process. We describe strategies for selecting among these models by a backward elimination procedure, Akaike information criterion (AIC) or a corrected Akaike information criterion (AICc). We evaluate the performance of these model selection methods via a simulation study, and make several recommendations for real data analysis. Our simulation study indicates that the backward elimination procedure can provide a reliable method for model selection in this setting. We also demonstrate the utility of these models by application to a single-gene dataset partitioned according to tertiary structure (abalone sperm lysin), and a multi-gene dataset partitioned according to the functional category of the gene (flagellar-related proteins of Listeria).

Conclusion

Fixed-effect models have advantages and disadvantages. Fixed-effect models are desirable when data partitions are known to exhibit significant heterogeneity or when a statistical test of such heterogeneity is desired. They have the disadvantage of requiring a priori knowledge for partitioning sites. We recommend: (i) selection of models by using backward elimination rather than AIC or AICc, (ii) use a stringent cut-off, e.g., p = 0.0001, and (iii) conduct sensitivity analysis of results. With thoughtful application, fixed-effect codon models should provide a useful tool for large scale multi-gene analyses.

     

基于GIS的渭河下游河流沿线区域生态风险评价

以渭河下游河流沿线区域为研究区,基于风险度量的基本原理和区域生态风险评价的流程框图,构建了综合生态风险值的计算模型,即综合生态风险值是综合风险概率、综合生态损失度和综合社会经济易损度的叠加.在研究区历史资料考证和2002年ETM+遥感影像景观分类的基础上,通过选取干旱、洪水、污染和水土流失,利用地理信息系统和现代地理学中的大量数学方法,按照评价步骤和计算模型,将研究区4种生态风险源综合评价结果划分为低风险区,较低风险区,中等风险区,较高风险区和高风险区5个级别.研究结果为环境管理和生态风险决策提供数量化的理论依据和数据支持.     

城市生态系统灵敏度模型评述

系统复杂性是困扰城市生态系统研究和管理的因素之一,成熟的系统分析模型可以帮助研究者及管理者应对这种挑战。德国系统思想大师Vester教授开发的灵敏度模型就是致力于解决城市规划管理中的复杂性问题,并在一系列的实践应用中取得了显著成效。网状思维(Networked Thinking)与生物控制论观点是Vester教授的核心思想,也是灵敏度模型的思想基础。模型主要分为三大层次:最底层是以数据的收集与筛选过程为代表的信息组织层次;随后是系统解释层次,主要是对系统关系网进行控制论解释;最后是系统综合评价层次,采用生物控制论观点对系统结构、行为等进行检验与评价。为方便用户使用,模型被分解为9个标准步骤,每一部分都包含数个实用的数学分析工具。模型还明确提出了4个等级的系统控制论指标(或特征)体系,以帮助使用者更好地进行系统思考。灵敏度模型本身具有众多的原创性贡献,但同时也有一定的局限性。这些经验与教训为未来的城市生态系统建模工作提供了宝贵启示。     

Environmental Risk Management Decision-Making in a Societal Context

Recent studies point to the need for improved understanding of environmental management frameworks designed to combine qualitative public and quantitative technical inputs in decision-making processes. Flux in public perception and concern about risks imply frameworks must be iterative in nature and incorporate a variety of assessment triggers in the form of decision points. A conceptual model is proposed here to explain the de facto operation of standard risk analytic frameworks within the broader sociopolitical milieu of public policy. The model is presented as a decision flow diagram that emphasizes setting environmental management goals based on societal input and the formulation of decision criteria for selecting management actions to achieve those goals. Prospective and retrospective decision control points operate to select management options that, respectively, avoid or reduce actual or predicted effects. Feedback loops that modify risk management outcomes are identified. Technical and scientific inputs (i.e., risk analysis) are assigned an essential information role within the framework and are responsible for informing the management process with the results of appropriately conducted and reviewed investigations. The proposed model is intended primarily to indicate how environmental risk management decision-making and associated technical assessments may be influenced by social pressures. It is hoped this understanding will lead to analytical transparency and better public communication of the environmental implications of policy options.     

A simple framework for selection of water quality models

Water quality modeling is no longer just the preserve of specialists who seek to describe water quality processes but also for use by non specialists in everyday water quality management issues. With so many models already developed, it becomes prudent to adapt them to a situation than to develop a completely new model that would probably do the same simulations. The question is: which is the most appropriate model to apply to a situation? The specialist can always draw on past experiences to make a decision. However, this is not the case for the non specialist. A lot of different criteria can be used to decide which model to use for a particular situation based on some important factors. The objectives of modeling exercises differ and each water body is unique so there cannot be hard and fast rules on which is the best criteria for selecting the appropriate model. Furthermore, there is usually hardly any time on the project work plan allocated for model selection. Therefore there is need for a simple procedure to select the appropriate model. The objective of this paper was to develop a simple framework for selecting water quality models to aid the non specialist. The framework was then applied to a case study in order to evaluate its usefulness. The results from the case study show that after a thorough literature review, models can be evaluated against chosen criteria and the most appropriate model singled out. It was concluded that the framework is only effective if the research objective is adequately defined and the models are reviewed thoroughly, but it saves time for the actual modeling exercise.     

森林-农业景观格局变化的社会-生态系统模拟模型方法进展

社会-生态系统（SES）模拟模型是景观格局分析和决策的有效工具,能表征景观格局变化的社会-生态效应及景观决策的复杂反馈机制。文献综述了森林-农业景观格局的SES模型方法进展发现：（1）多数模型对景观过程与社会经济决策的反馈关系分析不足;（2）应集成多种情景模拟和景观效应分析方法,完善现有SES模型的理论方法基础;（3）通过集成格局优化模型和自主体模型会有效改进SES模型功能,具体途径包括：集成情景-生态效应的景观格局模拟方法、完善景观决策的理论基础、加强集成模型的不确定性分析、降低模型复杂性和综合定性-定量数据等。研究结果有助于理解多尺度森林-农业景观格局在社会-生态系统中的重要作用,能更好地支持跨学科集成模型开发与应用。     

Prioritizing Biosecurity Risks Using a Participatory Decision-Making Tool

A decision framework called Deliberative Multi-Criteria Evaluation (DMCE) was developed and deployed to prioritize biosecurity risks using a variety of subjective and objective information. To aid stakeholders in the prioritization of Emergency Plant Pest (EPP) species risk, we presented them with outputs from a Stella-based bio-economic pest risk model, and probable ecological and socioeconomic impacts. The stakeholder participants weighted the consequence criteria they deemed to have the highest expected impact. The methodology featured an uncertain set of parameters, multiple iterations of criteria weighting along with real-time sensitivity analysis. Of the five criteria, economic cost was weighted the highest at 26% while landscape amenity was weighted the lowest at the 10–12% range. The increased understanding and support gained by stakeholders through the DMCE process provided a greater likelihood of the sanctioning of the policies concerned and progress toward desired outcomes.     

A Risk Elimination Coordination Method for Large Group Decision-Making in Natural Disaster Emergencies

ABSTRACT

In view of the risk characteristics of large group decision-making in emergencies and the difficultly in forming the decision scheme with low risk, the causes of generating risk in urgent group decisions are analyzed. The coordination framework for risk elimination in emergency decisions is constructed. The methods of risk measure and elimination for emergency decisions are proposed to construct the risk elimination coordination mechanism for emergency decisions so that the risk of emergency group decisions is gradually constringed to obtain the emergency decision scheme with risk degree low enough. Finally, an example case is used to validate the realization process and the validity of risk elimination coordination method and mechanism.     

Applications of artificial neural networks predicting macroinvertebrates in freshwaters

To facilitate decision support in freshwater ecosystem protection and restoration management, habitat suitability models can be very valuable. Data driven methods such as artificial neural networks (ANNs) are particularly useful in this context, seen their time-efficient development and relatively high reliability. However, specialized and technical literature on neural network modelling offers a variety of model development criteria to select model architecture, training procedure, etc. This may lead to confusion among ecosystem modellers and managers regarding the optimal training and validation methodology. This paper focuses on the analysis of ANN development and application for predicting macroinvertebrate communities, a species group commonly used in freshwater assessment worldwide. This review reflects on the different aspects regarding model development and application based on a selection of 26 papers reporting the use of ANN models for the prediction of macroinvertebrates. This analysis revealed that the applied model training and validation methodologies can often be improved and moreover crucial steps in the modelling process are often poorly documented. Therefore, suggestions to improve model development, assessment and application in ecological river management are presented. In particular, data pre-processing determines to a high extent the reliability of the induced models and their predictive relevance. This also counts for the validation criteria, that need to be better tuned to the practical simulation requirements. Moreover, the use of sensitivity methods can help to extract knowledge on the habitat preference of species and allow peer-review by ecological experts. The selection of relevant input variables remains a critical challenge as well. Model coupling is a missing crucial step to link human activities, hydrology, physical habitat conditions, water quality and ecosystem status. This last aspect is probably the most valuable aspect to enable decision support in water management based on ANN models.     

Ranking of Business Process Simulation Software Tools with DEX/QQ Hierarchical Decision Model

The omnipresent need for optimisation requires constant improvements of companies’ business processes (BPs). Minimising the risk of inappropriate BP being implemented is usually performed by simulating the newly developed BP under various initial conditions and “what-if” scenarios. An effectual business process simulations software (BPSS) is a prerequisite for accurate analysis of an BP. Characterisation of an BPSS tool is a challenging task due to the complex selection criteria that includes quality of visual aspects, simulation capabilities, statistical facilities, quality reporting etc. Under such circumstances, making an optimal decision is challenging. Therefore, various decision support models are employed aiding the BPSS tool selection. The currently established decision support models are either proprietary or comprise only a limited subset of criteria, which affects their accuracy. Addressing this issue, this paper proposes a new hierarchical decision support model for ranking of BPSS based on their technical characteristics by employing DEX and qualitative to quantitative (QQ) methodology. Consequently, the decision expert feeds the required information in a systematic and user friendly manner. There are three significant contributions of the proposed approach. Firstly, the proposed hierarchical model is easily extendible for adding new criteria in the hierarchical structure. Secondly, a fully operational decision support system (DSS) tool that implements the proposed hierarchical model is presented. Finally, the effectiveness of the proposed hierarchical model is assessed by comparing the resulting rankings of BPSS with respect to currently available results.     

Towards optimal distance functions for stochastic substitution models

Distance based reconstruction methods of phylogenetic trees consist of two independent parts: first, inter-species distances are inferred assuming some stochastic model of sequence evolution; then the inferred distances are used to construct a tree. In this paper we concentrate on the task of inter-species distance estimation. Specifically, we characterize the family of valid distance functions for the assumed substitution model and show that deliberate selection of distance function significantly improves the accuracy of distance estimates and, consequently, also improves the accuracy of the reconstructed tree.Our contribution consists of three parts: first, we present a general framework for constructing families of additive distance functions for stochastic evolutionary models. Then, we present a method for selecting (near) optimal distance functions, and we conclude by presenting simulation results which support our theoretical analysis.     

Variable selection – A review and recommendations for the practicing statistician

Statistical models support medical research by facilitating individualized outcome prognostication conditional on independent variables or by estimating effects of risk factors adjusted for covariates. Theory of statistical models is well‐established if the set of independent variables to consider is fixed and small. Hence, we can assume that effect estimates are unbiased and the usual methods for confidence interval estimation are valid. In routine work, however, it is not known a priori which covariates should be included in a model, and often we are confronted with the number of candidate variables in the range 10–30. This number is often too large to be considered in a statistical model. We provide an overview of various available variable selection methods that are based on significance or information criteria, penalized likelihood, the change‐in‐estimate criterion, background knowledge, or combinations thereof. These methods were usually developed in the context of a linear regression model and then transferred to more generalized linear models or models for censored survival data. Variable selection, in particular if used in explanatory modeling where effect estimates are of central interest, can compromise stability of a final model, unbiasedness of regression coefficients, and validity of p‐values or confidence intervals. Therefore, we give pragmatic recommendations for the practicing statistician on application of variable selection methods in general (low‐dimensional) modeling problems and on performing stability investigations and inference. We also propose some quantities based on resampling the entire variable selection process to be routinely reported by software packages offering automated variable selection algorithms.     

Likelihood methods for detecting temporal shifts in diversification rates

Maximum likelihood is a potentially powerful approach for investigating the tempo of diversification using molecular phylogenetic data. Likelihood methods distinguish between rate-constant and rate-variable models of diversification by fitting birth-death models to phylogenetic data. Because model selection in this context is a test of the null hypothesis that diversification rates have been constant over time, strategies for selecting best-fit models must minimize Type I error rates while retaining power to detect rate variation when it is present. Here I examine model selection, parameter estimation, and power to reject the null hypothesis using likelihood models based on the birth-death process. The Akaike information criterion (AIC) has often been used to select among diversification models; however, I find that selecting models based on the lowest AIC score leads to a dramatic inflation of the Type I error rate. When appropriately corrected to reduce Type I error rates, the birth-death likelihood approach performs as well or better than the widely used gamma statistic, at least when diversification rates have shifted abruptly over time. Analyses of datasets simulated under a range of rate-variable diversification scenarios indicate that the birth-death likelihood method has much greater power to detect variation in diversification rates when extinction is present. Furthermore, this method appears to be the only approach available that can distinguish between a temporal increase in diversification rates and a rate-constant model with nonzero extinction. I illustrate use of the method by analyzing a published phylogeny for Australian agamid lizards.