Determining the parametric structure of models

In this paper we develop a comprehensive approach to determining the parametric structure of models. This involves considering whether a model is parameter redundant or not and investigating model identifiability. The approach adopted makes use of exhaustive summaries, quantities that uniquely define the model. We review and generalise previous work on evaluating the symbolic rank of an appropriate derivative matrix to detect parameter redundancy, and then develop further tools for use within this framework, based on a matrix decomposition. Complex models, where the symbolic rank is difficult to calculate, may be simplified structurally using reparameterisation and by finding a reduced-form exhaustive summary. The approach of the paper is illustrated using examples from ecology, compartment modelling and Bayes networks. This work is topical as models in the biosciences and elsewhere are becoming increasingly complex.     

Does your species have memory? Analyzing capture–recapture data with memory models

We examine memory models for multisite capture–recapture data. This is an important topic, as animals may exhibit behavior that is more complex than simple first‐order Markov movement between sites, when it is necessary to devise and fit appropriate models to data. We consider the Arnason–Schwarz model for multisite capture–recapture data, which incorporates just first‐order Markov movement, and also two alternative models that allow for memory, the Brownie model and the Pradel model. We use simulation to compare two alternative tests which may be undertaken to determine whether models for multisite capture–recapture data need to incorporate memory. Increasing the complexity of models runs the risk of introducing parameters that cannot be estimated, irrespective of how much data are collected, a feature which is known as parameter redundancy. Rouan et al. (JABES, 2009, pp 338–355) suggest a constraint that may be applied to overcome parameter redundancy when it is present in multisite memory models. For this case, we apply symbolic methods to derive a simpler constraint, which allows more parameters to be estimated, and give general results not limited to a particular configuration. We also consider the effect sparse data can have on parameter redundancy and recommend minimum sample sizes. Memory models for multisite capture–recapture data can be highly complex and difficult to fit to data. We emphasize the importance of a structured approach to modeling such data, by considering a priori which parameters can be estimated, which constraints are needed in order for estimation to take place, and how much data need to be collected. We also give guidance on the amount of data needed to use two alternative families of tests for whether models for multisite capture–recapture data need to incorporate memory.     

An evaluation of behavior inferences from Bayesian state‐space models: A case study with the Pacific walrus

State‐space models offer researchers an objective approach to modeling complex animal location data sets, and state‐space model behavior classifications are often assumed to have a link to animal behavior. In this study, we evaluated the behavioral classification accuracy of a Bayesian state‐space model in Pacific walruses using Argos satellite tags with sensors to detect animal behavior in real time. We fit a two‐state discrete‐time continuous‐space Bayesian state‐space model to data from 306 Pacific walruses tagged in the Chukchi Sea. We matched predicted locations and behaviors from the state‐space model (resident, transient behavior) to true animal behavior (foraging, swimming, hauled out) and evaluated classification accuracy with kappa statistics (κ) and root mean square error (RMSE). In addition, we compared biased random bridge utilization distributions generated with resident behavior locations to true foraging behavior locations to evaluate differences in space use patterns. Results indicated that the two‐state model fairly classified true animal behavior (0.06 ≤ κ ≤ 0.26, 0.49 ≤ RMSE ≤ 0.59). Kernel overlap metrics indicated utilization distributions generated with resident behavior locations were generally smaller than utilization distributions generated with true foraging behavior locations. Consequently, we encourage researchers to carefully examine parameters and priors associated with behaviors in state‐space models, and reconcile these parameters with the study species and its expected behaviors.     

Parameter redundancy in mark‐recovery models

We provide a definitive guide to parameter redundancy in mark‐recovery models, indicating, for a wide range of models, in which all the parameters are estimable, and in which models they are not. For these parameter‐redundant models, we identify the parameter combinations that can be estimated. Simple, general results are obtained, which hold irrespective of the duration of the studies. We also examine the effect real data have on whether or not models are parameter redundant, and show that results can be robust even with very sparse data. Covariates, as well as time‐ or age‐varying trends, can be added to models to overcome redundancy problems. We show how to determine, without further calculation, whether or not parameter‐redundant models are still parameter redundant after the addition of covariates or trends.     

An exhaustive analysis of heuristic methods for variable selection in ecological niche modeling and species distribution modeling

Ecological niche models and species distribution models are used in many fields of science. Despite their popularity, only recently have important aspects of the modeling process like model selection been developed. Choosing environmental variables with which to create these models is another critical part of the process, but methods currently in use are not consistent in their results and no comprehensive approach exists by which to perform this step. Here, we compared seven heuristic methods of variable selection against a novel approach that proposes to select best sets of variables by evaluating performance of models created with all combinations of variables and distinct parameter settings of the algorithm in concert. Our results were that—except for the jackknife method for one of the 12 species and fluctuation index for two of the 12 species—none of the heuristic methods for variable selection coincided with the exhaustive one. Performance decreased in models created using variables selected with heuristic methods and both underfitting and overfitting were detected when comparing their geographic projections with the ones of models created with variables selected with the exhaustive method. Using the exhaustive approach could be time consuming, so a two-step exercise may be necessary. However, using this method identifies adequate variable sets and parameter settings in concert that are associated with increased model performance.     

Parameter Redundancy in Multistate Capture‐Recapture Models

Multistate capture‐recapture models are a powerful tool to address a variety of biological questions concerning dispersal and/or individual variability in wild animal populations. However, biologically meaningful models are often over‐parameterized and consequently some parameters cannot be estimated separately. Identifying which quantities are separately estimable is crucial for proper model selection based upon likelihood tests or information criteria and for the interpretation of the estimates obtained. We show how to investigate parameter redundancy in multistate capture‐recapture models, based on formal methods initially proposed by Catchpole and his associates for exponential family distributions (Catchpole, Freeman and Morgan, 1996. Journal of the Royal Statistical Society Series B 58, 763–774). We apply their approach to three models of increasing complexity.     

Model‐based identifiable parameter determination applied to a simultaneous saccharification and fermentation process model for bio‐ethanol production

In this work, a methodology for the model‐based identifiable parameter determination (MBIPD) is presented. This systematic approach is proposed to be used for structure and parameter identification of nonlinear models of biological reaction networks. Usually, this kind of problems are over‐parameterized with large correlations between parameters. Hence, the related inverse problems for parameter determination and analysis are mathematically ill‐posed and numerically difficult to solve. The proposed MBIPD methodology comprises several tasks: (i) model selection, (ii) tracking of an adequate initial guess, and (iii) an iterative parameter estimation step which includes an identifiable parameter subset selection (SsS) algorithm and accuracy analysis of the estimated parameters. The SsS algorithm is based on the analysis of the sensitivity matrix by rank revealing factorization methods. Using this, a reduction of the parameter search space to a reasonable subset, which can be reliably and efficiently estimated from available measurements, is achieved. The simultaneous saccharification and fermentation (SSF) process for bio‐ethanol production from cellulosic material is used as case study for testing the methodology. The successful application of MBIPD to the SSF process demonstrates a relatively large reduction in the identified parameter space. It is shown by a cross‐validation that using the identified parameters (even though the reduction of the search space), the model is still able to predict the experimental data properly. Moreover, it is shown that the model is easily and efficiently adapted to new process conditions by solving reduced and well conditioned problems. © 2013 American Institute of Chemical Engineers Biotechnol. Prog., 29:1064–1082, 2013     

Sensitivity functions in the estimation of parameters of cellular exchange

The analysis of multiple-indicator dilution curves to estimate the rates of transport of ions and substrates across the sarcolemma of myocardial cells requires the formulation of models for the blood-interstitial fluid-cell exchanges. The fitting of models to the sets of experimental data is dependent on acquiring a large enough data set, in one physiological state, that there is at least as much information in the data as there are unknown model parameters to be determined. Inasmuch as data are necessarily noisy, redundancy of data and overdetermination of the unknowns are highly desirable. Sensitivity functions are useful in demonstrating which portions of the data relate to which unknown parameters. They are also useful in adjusting model parameters to fit the model to the data, and therefore in parameter evaluation.     

Mixed‐Effects State‐Space Models for Analysis of Longitudinal Dynamic Systems

Summary The rapid development of new biotechnologies allows us to deeply understand biomedical dynamic systems in more detail and at a cellular level. Many of the subject‐specific biomedical systems can be described by a set of differential or difference equations that are similar to engineering dynamic systems. In this article, motivated by HIV dynamic studies, we propose a class of mixed‐effects state‐space models based on the longitudinal feature of dynamic systems. State‐space models with mixed‐effects components are very flexible in modeling the serial correlation of within‐subject observations and between‐subject variations. The Bayesian approach and the maximum likelihood method for standard mixed‐effects models and state‐space models are modified and investigated for estimating unknown parameters in the proposed models. In the Bayesian approach, full conditional distributions are derived and the Gibbs sampler is constructed to explore the posterior distributions. For the maximum likelihood method, we develop a Monte Carlo EM algorithm with a Gibbs sampler step to approximate the conditional expectations in the E‐step. Simulation studies are conducted to compare the two proposed methods. We apply the mixed‐effects state‐space model to a data set from an AIDS clinical trial to illustrate the proposed methodologies. The proposed models and methods may also have potential applications in other biomedical system analyses such as tumor dynamics in cancer research and genetic regulatory network modeling.     

Parameter space boundaries for unidentifiable compartmental models

Methods for dealing with unidentifiable compartmental models are first reviewed, emphasizing the parameter interval analysis and exhaustive modeling approaches. More general methods are presented for generating the set of all nonnegative parameter solutions that localize the parameters within bounded regions of parameter space and extend previously published parameter bounding strategies. Each point of these regions is an equivalent solution of the parameter identification problem. If a point on the boundary is selected, at least one of the parameters vanishes and an equivalent submodel is obtained. This property shows the close relationship between the exhaustive modeling and parameter interval analysis approaches.     

Using functional traits to predict species growth trajectories,and cross‐validation to evaluate these models for ecological prediction

Modeling plant growth using functional traits is important for understanding the mechanisms that underpin growth and for predicting new situations. We use three data sets on plant height over time and two validation methods—in‐sample model fit and leave‐one‐species‐out cross‐validation—to evaluate non‐linear growth model predictive performance based on functional traits. In‐sample measures of model fit differed substantially from out‐of‐sample model predictive performance; the best fitting models were rarely the best predictive models. Careful selection of predictor variables reduced the bias in parameter estimates, and there was no single best model across our three data sets. Testing and comparing multiple model forms is important. We developed an R package with a formula interface for straightforward fitting and validation of hierarchical, non‐linear growth models. Our intent is to encourage thorough testing of multiple growth model forms and an increased emphasis on assessing model fit relative to a model's purpose.     

Analytical methods for predicting the behaviour of population models with general spatial interactions

Many biologists use population models that are spatial, stochastic and individual based. Analytical methods that describe the behaviour of these models approximately are attracting increasing interest as an alternative to expensive computer simulation. The methods can be employed for both prediction and fitting models to data. Recent work has extended existing (mean field) methods with the aim of accounting for the development of spatial correlations. A common feature is the use of closure approximations for truncating the set of evolution equations for summary statistics. We investigate an analytical approach for spatial and stochastic models where individuals interact according to a generic function of their distance; this extends previous methods for lattice models with interactions between close neighbours, such as the pair approximation. Our study also complements work by Bolker and Pacala (BP) [Theor. Pop. Biol. 52 (1997) 179; Am. Naturalist 153 (1999) 575]: it treats individuals as being spatially discrete (defined on a lattice) rather than as a continuous mass distribution; it tests the accuracy of different closure approximations over parameter space, including the additive moment closure (MC) used by BP and the Kirkwood approximation. The study is done in the context of an susceptible-infected-susceptible epidemic model with primary infection and with secondary infection represented by power-law interactions. MC is numerically unstable or inaccurate in parameter regions with low primary infection (or density-independent birth rates). A modified Kirkwood approximation gives stable and generally accurate transient and long-term solutions; we argue it can be applied to lattice and to continuous-space models as a substitute for MC. We derive a generalisation of the basic reproduction ratio, R(0), for spatial models.     

Weighted parsimony outperforms other methods of phylogenetic inference under models appropriate for morphology

One of the lasting controversies in phylogenetic inference is the degree to which specific evolutionary models should influence the choice of methods. Model‐based approaches to phylogenetic inference (likelihood, Bayesian) are defended on the premise that without explicit statistical models there is no science, and parsimony is defended on the grounds that it provides the best rationalization of the data, while refraining from assigning specific probabilities to trees or character‐state reconstructions. Authors who favour model‐based approaches often focus on the statistical properties of the methods and models themselves, but this is of only limited use in deciding the best method for phylogenetic inference—such decision also requires considering the conditions of evolution that prevail in nature. Another approach is to compare the performance of parsimony and model‐based methods in simulations, which traditionally have been used to defend the use of models of evolution for DNA sequences. Some recent papers, however, have promoted the use of model‐based approaches to phylogenetic inference for discrete morphological data as well. These papers simulated data under models already known to be unfavourable to parsimony, and modelled morphological evolution as if it evolved just like DNA, with probabilities of change for all characters changing in concert along tree branches. The present paper discusses these issues, showing that under reasonable and less restrictive models of evolution for discrete characters, equally weighted parsimony performs as well or better than model‐based methods, and that parsimony under implied weights clearly outperforms all other methods.     

Choosing the right sigmoid growth function using the unified‐models approach

Growth of the young is an important part of the life history in birds. However, modelling methods have paid little attention to the choice of regression model used to describe its pattern. The aim of this study was to evaluate whether a single sigmoid model with an upper asymptote could describe avian growth adequately. We compared unified versions of five growth models of the Richards family (the four‐parameter U‐Richards and the three‐parameter U‐logistic, U‐Gompertz, U‐Bertalanffy and U4‐models) for three traits (body mass, tarsus‐length and wing‐length) for 50 passerine species, including species with varied morphologies and life histories. The U‐family models exhibit a unified set of parameters for all models. The four‐parameter U‐Richards model proved a good choice for fitting growth curves to various traits – its extra d‐parameter allows for a flexible placement of the inflection point. Which of the three‐parameter U‐models was the best performing varied greatly between species and between traits, as each three‐parameter model had a different fixed relative inflection value (fraction of the upper asymptote), implying a different growth pattern. Fixing the asymptotes to averages for adult trait value generally shifted the model preference towards one with lower relative inflection values. Our results illustrate an overlooked difficulty in the analysis of organismal growth, namely, that a single traditional three‐parameter model does not suit all growth data. This is mostly due to differences in inflection placement. Moreover, some biometric traits require more attention when estimating growth rates and other growth‐curve characteristics. We recommend fitting either several three‐parameter models from the U‐family, where the parameters are comparable between models, or only the U‐Richards model.     

Evaluating the Use of Global Sensitivity Analysis in Dynamic MFA

Dynamic material flow analysis (MFA) provides information about material usage over time and consequent changes in material stocks and flows. In order to understand the effect of limited data quality and model assumptions on MFA results, the use of sensitivity analysis methods in dynamic MFA studies has been on the increase. So far, sensitivity analysis in dynamic MFA has been conducted by means of a one‐at‐a‐time method, which tests parameter perturbations individually and observes the outcomes on output. In contrast to that, variance‐based global sensitivity analysis decomposes the variance of the model output into fractions caused by the uncertainty or variability of input parameters. The present study investigates interaction and time‐delay effects of uncertain parameters on the output of an archetypal input‐driven dynamic material flow model using variance‐based global sensitivity analysis. The results show that determining the main (first‐order) effects of parameter variations is often sufficient in dynamic MFA because substantial effects attributed to the simultaneous variation of several parameters (higher‐order effects) do not appear for classical setups of dynamic material flow models. For models with time‐varying parameters, time‐delay effects of parameter variation on model outputs need to be considered, potentially boosting the computational cost of global sensitivity analysis. Finally, the implications of exploring the sensitivities of model outputs with respect to parameter variations in the archetypical model are used to derive model‐ and goal‐specific recommendations on choosing appropriate sensitivity analysis methods in dynamic MFA.     

On observation distributions for state space models of population survey data

1.?State space models are starting to replace more simple time series models in analyses of temporal dynamics of populations that are not perfectly censused. By simultaneously modelling both the dynamics and the observations, consistent estimates of population dynamical parameters may be obtained. For many data sets, the distribution of observation errors is unknown and error models typically chosen in an ad-hoc manner. 2.?To investigate the influence of the choice of observation error on inferences, we analyse the dynamics of a replicated time series of red kangaroo surveys using a state space model with linear state dynamics. Surveys were performed through aerial counts and Poisson, overdispersed Poisson, normal and log-normal distributions may all be adequate for modelling observation errors for the data. We fit each of these to the data and compare them using AIC. 3.?The state space models were fitted with maximum likelihood methods using a recent importance sampling technique that relies on the Kalman filter. The method relaxes the assumption of Gaussian observation errors required by the basic Kalman filter. Matlab code for fitting linear state space models with Poisson observations is provided. 4.?The ability of AIC to identify the correct observation model was investigated in a small simulation study. For the parameter values used in the study, without replicated observations, the correct observation distribution could sometimes be identified but model selection was prone to misclassification. On the other hand, when observations were replicated, the correct distribution could typically be identified. 5.?Our results illustrate that inferences may differ markedly depending on the observation distributions used, suggesting that choosing an adequate observation model can be critical. Model selection and simulations show that for the models and parameter values in this study, a suitable observation model can typically be identified if observations are replicated. Model selection and replication of observations, therefore, provide a potential solution when the observation distribution is unknown.     

Uninformative Parameters and Model Selection Using Akaike's Information Criterion

Abstract: As use of Akaike's Information Criterion (AIC) for model selection has become increasingly common, so has a mistake involving interpretation of models that are within 2 AIC units (ΔAIC ≤ 2) of the top-supported model. Such models are <2 ΔAIC units because the penalty for one additional parameter is +2 AIC units, but model deviance is not reduced by an amount sufficient to overcome the 2-unit penalty and, hence, the additional parameter provides no net reduction in AIC. Simply put, the uninformative parameter does not explain enough variation to justify its inclusion in the model and it should not be interpreted as having any ecological effect. Models with uninformative parameters are frequently presented as being competitive in the Journal of Wildlife Management, including 72% of all AIC-based papers in 2008, and authors and readers need to be more aware of this problem and take appropriate steps to eliminate misinterpretation. I reviewed 5 potential solutions to this problem: 1) report all models but ignore or dismiss those with uninformative parameters, 2) use model averaging to ameliorate the effect of uninformative parameters, 3) use 95% confidence intervals to identify uninformative parameters, 4) perform all-possible subsets regression and use weight-of-evidence approaches to discriminate useful from uninformative parameters, or 5) adopt a methodological approach that allows models containing uninformative parameters to be culled from reported model sets. The first approach is preferable for small sets of a priori models, whereas the last 2 approaches should be used for large model sets or exploratory modeling.     

Exploring Approximate Bayesian Computation for inferring recent demographic history with genomic markers in nonmodel species

Approximate Bayesian computation (ABC) is widely used to infer demographic history of populations and species using DNA markers. Genomic markers can now be developed for nonmodel species using reduced representation library (RRL) sequencing methods that select a fraction of the genome using targeted sequence capture or restriction enzymes (genotyping‐by‐sequencing, GBS). We explored the influence of marker number and length, knowledge of gametic phase, and tradeoffs between sample size and sequencing depth on the quality of demographic inferences performed with ABC. We focused on two‐population models of recent spatial expansion with varying numbers of unknown parameters. Performing ABC on simulated data sets with known parameter values, we found that the timing of a recent spatial expansion event could be precisely estimated in a three‐parameter model. Taking into account uncertainty in parameters such as initial population size and migration rate collectively decreased the precision of inferences dramatically. Phasing haplotypes did not improve results, regardless of sequence length. Numerous short sequences were as valuable as fewer, longer sequences, and performed best when a large sample size was sequenced at low individual depth, even when sequencing errors were added. ABC results were similar to results obtained with an alternative method based on the site frequency spectrum (SFS) when performed with unphased GBS‐type markers. We conclude that unphased GBS‐type data sets can be sufficient to precisely infer simple demographic models, and discuss possible improvements for the use of ABC with genomic data.