The importance of assessing parameter sensitivity when using biophysical models: a case study using plethodontid salamanders

Landscapes are continually changing due to numerous assaults, including habitat alteration, anthropogenic disturbances, and climate change. Understanding how species will respond to these changes is of critical importance for conservation and management. Mechanistic models, such as biophysical models (BPMs), are an increasingly popular tool to predict how local population dynamics or species’ distributions may be altered in response to environmental and climate changes. By mechanistically modeling relationships between environmental conditions, physiology and behavior, it is possible to make accurate predictions about how species may respond. However, BPMs are often difficult to implement due to lack of appropriate, species-specific data that is biologically realistic or relevant. In this study, we present a BPM for the salamander Plethodon jordani and assess how adding more biological realism has potential to alter model predictions about annual energy budgets. Additionally, we conducted local and global sensitivity analyses to evaluate the importance of accurately specifying model parameter values and functional relationships. We found that the addition of biological realism resulted in greater model complexity as well as substantially different estimates of energy balance. Correct parameterization of biophysical models is also critical, as small changes in parameter values can result in disproportionately large changes in downstream model estimates. Our model highlights the overall importance of using ecologically relevant and specific data for input parameters, as well as careful assessment of parameter sensitivity. We encourage researchers to be aware of the data they are using to parameterize BPMs, and urge the collection of system-specific data that is relevant in spatial and temporal scale. We also recommend greater and more transparent use of sensitivity analyses to provide a better understanding of the model, as well as greater confidence in model predictions.     

Robustness and state-space structure of Boolean gene regulatory models

Robustness to perturbation is an important characteristic of genetic regulatory systems, but the relationship between robustness and model dynamics has not been clearly quantified. We propose a method for quantifying both robustness and dynamics in terms of state-space structures, for Boolean models of genetic regulatory systems. By investigating existing models of the Drosophila melanogaster segment polarity network and the Saccharomyces cerevisiae cell-cycle network, we show that the structure of attractor basins can yield insight into the underlying decision making required of the system, and also the way in which the system maximises its robustness. In particular, gene networks implementing decisions based on a few genes have simple state-space structures, and their attractors are robust by virtue of their simplicity. Gene networks with decisions that involve many interacting genes have correspondingly more complicated state-space structures, and robustness cannot be achieved through the structure of the attractor basins, but is achieved by larger attractor basins that dominate the state space. These different types of robustness are demonstrated by the two models: the D. melanogaster segment polarity network is robust due to simple attractor basins that implement decisions based on spatial signals; the S. cerevisiae cell-cycle network has a complicated state-space structure, and is robust only due to a giant attractor basin that dominates the state space.     

General methodology for nonlinear modeling of neural systems with Poisson point-process inputs

This paper presents a general methodological framework for the practical modeling of neural systems with point-process inputs (sequences of action potentials or, more broadly, identical events) based on the Volterra and Wiener theories of functional expansions and system identification. The paper clarifies the distinctions between Volterra and Wiener kernels obtained from Poisson point-process inputs. It shows that only the Wiener kernels can be estimated via cross-correlation, but must be defined as zero along the diagonals. The Volterra kernels can be estimated far more accurately (and from shorter data-records) by use of the Laguerre expansion technique adapted to point-process inputs, and they are independent of the mean rate of stimulation (unlike their P-W counterparts that depend on it). The Volterra kernels can also be estimated for broadband point-process inputs that are not Poisson. Useful applications of this modeling approach include cases where we seek to determine (model) the transfer characteristics between one neuronal axon (a point-process 'input') and another axon (a point-process 'output') or some other measure of neuronal activity (a continuous 'output', such as population activity) with which a causal link exists.     

Lattice models in ecology and social sciences

Random matching between individuals, or the complete-mixing model, is often assumed in analyzing evolutionary or population dynamics in ecology and game theory or other models in social sciences. Making and analyzing a model is not difficult under this simple assumption. However spatial- or network-structured populations, including the lattice model and the power-law network, are more realistic for many ecological and social phenomena than the complete-mixing model. In this review, I will show first that a lattice model can be useful in investigating the effect of neighborhood interactions on the dynamics, not only of plants and forests, but also of animal and human societies. Second, the lattice model promotes the evolution of spiteful behavior, even though it is well-known that the lattice model promotes the evolution of cooperation. Finally, different social networks result in traits, such as social norms, spreading at different speeds.     

Pair-edge approximation for heterogeneous lattice population models

To increase the analytical tractability of lattice stochastic spatial population models, several approximations have been developed. The pair-edge approximation is a moment-closure method that is effective in predicting persistence criteria and invasion speeds on a homogeneous lattice. Here we evaluate the effectiveness of the pair-edge approximation on a spatially heterogeneous lattice in which some sites are unoccupiable, or "dead". This model has several possible interpretations, including a spatial SIS epidemic model, in which some sites are occupied by immobile host-species individuals while others are empty. We find that, as in the homogeneous model, the pair-edge approximation is significantly more accurate than the ordinary pair approximation in determining conditions for persistence. However, habitat heterogeneity decreases invasion speed more than is predicted by the pair-edge approximation, and the discrepancy increases with greater clustering of "dead" sites. The accuracy of the approximation validates the underlying heuristic picture of population spread and therefore provides qualitative insight into the dynamics of lattice models. Conversely, the situations where the approximation is less accurate reveals limitations of pair approximation in the presence of spatial heterogeneity.     

Analytical methods for predicting the behaviour of population models with general spatial interactions

Many biologists use population models that are spatial, stochastic and individual based. Analytical methods that describe the behaviour of these models approximately are attracting increasing interest as an alternative to expensive computer simulation. The methods can be employed for both prediction and fitting models to data. Recent work has extended existing (mean field) methods with the aim of accounting for the development of spatial correlations. A common feature is the use of closure approximations for truncating the set of evolution equations for summary statistics. We investigate an analytical approach for spatial and stochastic models where individuals interact according to a generic function of their distance; this extends previous methods for lattice models with interactions between close neighbours, such as the pair approximation. Our study also complements work by Bolker and Pacala (BP) [Theor. Pop. Biol. 52 (1997) 179; Am. Naturalist 153 (1999) 575]: it treats individuals as being spatially discrete (defined on a lattice) rather than as a continuous mass distribution; it tests the accuracy of different closure approximations over parameter space, including the additive moment closure (MC) used by BP and the Kirkwood approximation. The study is done in the context of an susceptible-infected-susceptible epidemic model with primary infection and with secondary infection represented by power-law interactions. MC is numerically unstable or inaccurate in parameter regions with low primary infection (or density-independent birth rates). A modified Kirkwood approximation gives stable and generally accurate transient and long-term solutions; we argue it can be applied to lattice and to continuous-space models as a substitute for MC. We derive a generalisation of the basic reproduction ratio, R(0), for spatial models.     

Epidemic predictions in an imperfect world: modelling disease spread with partial data

‘Big-data’ epidemic models are being increasingly used to influence government policy to help with control and eradication of infectious diseases. In the case of livestock, detailed movement records have been used to parametrize realistic transmission models. While livestock movement data are readily available in the UK and other countries in the EU, in many countries around the world, such detailed data are not available. By using a comprehensive database of the UK cattle trade network, we implement various sampling strategies to determine the quantity of network data required to give accurate epidemiological predictions. It is found that by targeting nodes with the highest number of movements, accurate predictions on the size and spatial spread of epidemics can be made. This work has implications for countries such as the USA, where access to data is limited, and developing countries that may lack the resources to collect a full dataset on livestock movements.     

Dynamic occupancy models for analyzing species' range dynamics across large geographic scales

Large‐scale biodiversity data are needed to predict species' responses to global change and to address basic questions in macroecology. While such data are increasingly becoming available, their analysis is challenging because of the typically large heterogeneity in spatial sampling intensity and the need to account for observation processes. Two further challenges are accounting for spatial effects that are not explained by covariates, and drawing inference on dynamics at these large spatial scales. We developed dynamic occupancy models to analyze large‐scale atlas data. In addition to occupancy, these models estimate local colonization and persistence probabilities. We accounted for spatial autocorrelation using conditional autoregressive models and autologistic models. We fitted the models to detection/nondetection data collected on a quarter‐degree grid across southern Africa during two atlas projects, using the hadeda ibis (Bostrychia hagedash) as an example. The model accurately reproduced the range expansion between the first (SABAP1: 1987–1992) and second (SABAP2: 2007–2012) Southern African Bird Atlas Project into the drier parts of interior South Africa. Grid cells occupied during SABAP1 generally remained occupied, but colonization of unoccupied grid cells was strongly dependent on the number of occupied grid cells in the neighborhood. The detection probability strongly varied across space due to variation in effort, observer identity, seasonality, and unexplained spatial effects. We present a flexible hierarchical approach for analyzing grid‐based atlas data using dynamical occupancy models. Our model is similar to a species' distribution model obtained using generalized additive models but has a number of advantages. Our model accounts for the heterogeneous sampling process, spatial correlation, and perhaps most importantly, allows us to examine dynamic aspects of species ranges.     

Accounting for robustness in modeling signal transduction responses

Abstract

A classical question in systems biology is to find a Boolean model which is able to predict the observed responses of a signaling network. It has been previously shown that such models can be tailored based on experimental data. While fitting a minimum-size network to the experimentally observed data is a natural assumption, it can potentially result in a network which is not so robust against the noises in the training dataset. Indeed, it is widely accepted now that biological systems are generally evolved to be very robust. Therefore, in the present work, we extended the classical formulation of Boolean network construction in order to put weight on the robustness of the created network. We show that our method results generally in more relevant networks. Consequently, considering robustness as a design principle of biological networks can result in more realistic models.     

Comparison of models for nucleotide substitution used in maximum- likelihood phylogenetic estimation

Using real sequence data, we evaluate the adequacy of assumptions made in evolutionary models of nucleotide substitution and the effects that these assumptions have on estimation of evolutionary trees. Two aspects of the assumptions are evaluated. The first concerns the pattern of nucleotide substitution, including equilibrium base frequencies and the transition/transversion-rate ratio. The second concerns the variation of substitution rates over sites. The maximum-likelihood estimate of tree topology appears quite robust to both these aspects of the assumptions of the models, but evaluation of the reliability of the estimated tree by using simpler, less realistic models can be misleading. Branch lengths are underestimated when simpler models of substitution are used, but the underestimation caused by ignoring rate variation over nucleotide sites is much more serious. The goodness of fit of a model is reduced by ignoring spatial rate variation, but unrealistic assumptions about the pattern of nucleotide substitution can lead to an extraordinary reduction in the likelihood. It seems that evolutionary biologists can obtain accurate estimates of certain evolutionary parameters even with an incorrect phylogeny, while systematists cannot get the right tree with confidence even when a realistic, and more complex, model of evolution is assumed.      

Diffusion versus network models as descriptions for the spread of prion diseases in the brain

In this paper we will discuss different modeling approaches for the spread of prion diseases in the brain. Firstly, we will compare reaction-diffusion models with models of epidemic diseases on networks. The solutions of the resulting reaction-diffusion equations exhibit traveling wave behavior on a one-dimensional domain, and the wave speed can be estimated. The models can be tested for diffusion-driven (Turing) instability, which could present a possible mechanism for the formation of plaques. We also show that the reaction-diffusion systems are capable of reproducing experimental data on prion spread in the mouse visual system. Secondly, we study classical epidemic models on networks, and use these models to study the influence of the network topology on the disease progression.     

The relationship between real-time and discrete-generation models of epidemic spread

Many important results in stochastic epidemic modelling are based on the Reed-Frost model or on other similar models that are characterised by unrealistic temporal dynamics. Nevertheless, they can be extended to many other more realistic models thanks to an argument first provided by Ludwig [Final size distributions for epidemics, Math. Biosci. 23 (1975) 33-46], that states that, for a disease leading to permanent immunity after recovery, under suitable conditions, a continuous-time infectious process has the same final size distribution as another more tractable discrete-generation contact process; in other words, the temporal dynamics of the epidemic can be neglected without affecting the final size distribution. Despite the importance of such an argument, its presence behind many results is often not clearly stated or hidden in references to previous results. In this paper, we reanalyse Ludwig’s result, highlighting some of the conditions under which it does not hold and providing a general framework to examine the differences between the continuous-time and the discrete-generation process.     

The importance of individual developmental variation in stage‐structured population models

Population stage structure is fundamental to ecology, and models of this structure have proven useful in many different systems. Many ecological variables other than stage, such as habitat type, site occupancy and metapopulation status are also modelled using transitions among discrete states. Transitions among life stages can be characterised by the distribution of time spent in each stage, including the mean and variance of each stage duration and within‐individual correlations among multiple stage durations. Three modelling traditions represent stage durations differently. Matrix models can be derived as a long‐run approximation from any distribution of stage durations, but they are often interpreted directly as a Markov model for stage transitions. Statistical stage‐duration distribution models accommodate the variation typical of cohort development data, but such realism has rarely been incorporated in population theory or statistical population models. Delay‐differential equation models include lags but no variation, except in limited cases. We synthesise these models in one framework and illustrate how individual variation and correlations in development can impact population growth. Furthermore, different development models can yield the same long‐term matrix transition rates but different sensitivities and elasticities. Finally, we discuss future directions for estimating realistic stage duration models from data.     

Validation of species–climate impact models under climate change

Increasing concern over the implications of climate change for biodiversity has led to the use of species–climate envelope models to project species extinction risk under climate‐change scenarios. However, recent studies have demonstrated significant variability in model predictions and there remains a pressing need to validate models and to reduce uncertainties. Model validation is problematic as predictions are made for events that have not yet occurred. Resubstituition and data partitioning of present‐day data sets are, therefore, commonly used to test the predictive performance of models. However, these approaches suffer from the problems of spatial and temporal autocorrelation in the calibration and validation sets. Using observed distribution shifts among 116 British breeding‐bird species over the past ～20 years, we are able to provide a first independent validation of four envelope modelling techniques under climate change. Results showed good to fair predictive performance on independent validation, although rules used to assess model performance are difficult to interpret in a decision‐planning context. We also showed that measures of performance on nonindependent data provided optimistic estimates of models' predictive ability on independent data. Artificial neural networks and generalized additive models provided generally more accurate predictions of species range shifts than generalized linear models or classification tree analysis. Data for independent model validation and replication of this study are rare and we argue that perfect validation may not in fact be conceptually possible. We also note that usefulness of models is contingent on both the questions being asked and the techniques used. Implementations of species–climate envelope models for testing hypotheses and predicting future events may prove wrong, while being potentially useful if put into appropriate context.     

Analytic approximation of spatial epidemic models of foot and mouth disease

The effect of spatial heterogeneity in epidemic models has improved with computational advances, yet far less progress has been made in developing analytical tools for understanding such systems. Here, we develop two classes of second-order moment closure methods for approximating the dynamics of a stochastic spatial model of the spread of foot and mouth disease. We consider the performance of such ‘pseudo-spatial’ models as a function of R₀, the locality in disease transmission, farm distribution and geographically-targeted control when an arbitrary number of spatial kernels are incorporated. One advantage of mapping complex spatial models onto simpler deterministic approximations lies in the ability to potentially obtain a better analytical understanding of disease dynamics and the effects of control. We exploit this tractability by deriving analytical results in the invasion stages of an FMD outbreak, highlighting key principles underlying epidemic spread on contact networks and the effect of spatial correlations.     

When mechanism matters: Bayesian forecasting using models of ecological diffusion

Ecological diffusion is a theory that can be used to understand and forecast spatio‐temporal processes such as dispersal, invasion, and the spread of disease. Hierarchical Bayesian modelling provides a framework to make statistical inference and probabilistic forecasts, using mechanistic ecological models. To illustrate, we show how hierarchical Bayesian models of ecological diffusion can be implemented for large data sets that are distributed densely across space and time. The hierarchical Bayesian approach is used to understand and forecast the growth and geographic spread in the prevalence of chronic wasting disease in white‐tailed deer (Odocoileus virginianus). We compare statistical inference and forecasts from our hierarchical Bayesian model to phenomenological regression‐based methods that are commonly used to analyse spatial occurrence data. The mechanistic statistical model based on ecological diffusion led to important ecological insights, obviated a commonly ignored type of collinearity, and was the most accurate method for forecasting.     

Effect of species rarity on the accuracy of species distribution models for reptiles and amphibians in southern California

Aim Several studies have found that more accurate predictive models of species’ occurrences can be developed for rarer species; however, one recent study found the relationship between range size and model performance to be an artefact of sample prevalence, that is, the proportion of presence versus absence observations in the data used to train the model. We examined the effect of model type, species rarity class, species’ survey frequency, detectability and manipulated sample prevalence on the accuracy of distribution models developed for 30 reptile and amphibian species. Location Coastal southern California, USA. Methods Classification trees, generalized additive models and generalized linear models were developed using species presence and absence data from 420 locations. Model performance was measured using sensitivity, specificity and the area under the curve (AUC) of the receiver‐operating characteristic (ROC) plot based on twofold cross‐validation, or on bootstrapping. Predictors included climate, terrain, soil and vegetation variables. Species were assigned to rarity classes by experts. The data were sampled to generate subsets with varying ratios of presences and absences to test for the effect of sample prevalence. Join count statistics were used to characterize spatial dependence in the prediction errors. Results Species in classes with higher rarity were more accurately predicted than common species, and this effect was independent of sample prevalence. Although positive spatial autocorrelation remained in the prediction errors, it was weaker than was observed in the species occurrence data. The differences in accuracy among model types were slight. Main conclusions Using a variety of modelling methods, more accurate species distribution models were developed for rarer than for more common species. This was presumably because it is difficult to discriminate suitable from unsuitable habitat for habitat generalists, and not as an artefact of the effect of sample prevalence on model estimation.     

From Markovian to pairwise epidemic models and the performance of moment closure approximations

Many if not all models of disease transmission on networks can be linked to the exact state-based Markovian formulation. However the large number of equations for any system of realistic size limits their applicability to small populations. As a result, most modelling work relies on simulation and pairwise models. In this paper, for a simple SIS dynamics on an arbitrary network, we formalise the link between a well known pairwise model and the exact Markovian formulation. This involves the rigorous derivation of the exact ODE model at the level of pairs in terms of the expected number of pairs and triples. The exact system is then closed using two different closures, one well established and one that has been recently proposed. A new interpretation of both closures is presented, which explains several of their previously observed properties. The closed dynamical systems are solved numerically and the results are compared to output from individual-based stochastic simulations. This is done for a range of networks with the same average degree and clustering coefficient but generated using different algorithms. It is shown that the ability of the pairwise system to accurately model an epidemic is fundamentally dependent on the underlying large-scale network structure. We show that the existing pairwise models are a good fit for certain types of network but have to be used with caution as higher-order network structures may compromise their effectiveness.     

Predictive models in ecology: Comparison of performances and assessment of applicability

Ecological systems are governed by complex interactions which are mainly nonlinear. In order to capture the inherent complexity and nonlinearity of ecological, and in general biological systems, empirical models recently gained popularity. However, although these models, particularly connectionist approaches such as multilayered backpropagation networks, are commonly applied as predictive models in ecology to a wide variety of ecosystems and questions, there are no studies to date aiming to assess the performance, both in terms of data fitting and generalizability, and applicability of empirical models in ecology. Our aim is hence to provide an overview for nature of the wide range of the data sets and predictive variables, from both aquatic and terrestrial ecosystems with different scales of time-dependent dynamics, and the applicability and robustness of predictive modeling methods on such data sets by comparing different empirical modeling approaches. The models used in this study range from predicting the occurrence of submerged plants in shallow lakes to predicting nest occurrence of bird species from environmental variables and satellite images. The methods considered include k-nearest neighbor (k-NN), linear and quadratic discriminant analysis (LDA and QDA), generalized linear models (GLM) feedforward multilayer backpropagation networks and pseudo-supervised network ARTMAP.Our results show that the predictive performances of the models on training data could be misleading, and one should consider the predictive performance of a given model on an independent test set for assessing its predictive power. Moreover, our results suggest that for ecosystems involving time-dependent dynamics and periodicities whose frequency are possibly less than the time scale of the data considered, GLM and connectionist neural network models appear to be most suitable and robust, provided that a predictive variable reflecting these time-dependent dynamics included in the model either implicitly or explicitly. For spatial data, which does not include any time-dependence comparable to the time scale covered by the data, on the other hand, neighborhood based methods such as k-NN and ARTMAP proved to be more robust than other methods considered in this study. In addition, for predictive modeling purposes, first a suitable, computationally inexpensive method should be applied to the problem at hand a good predictive performance of which would render the computational cost and efforts associated with complex variants unnecessary.