Model-checking techniques based on cumulative residuals

Residuals have long been used for graphical and numerical examinations of the adequacy of regression models. Conventional residual analysis based on the plots of raw residuals or their smoothed curves is highly subjective, whereas most numerical goodness-of-fit tests provide little information about the nature of model misspecification. In this paper, we develop objective and informative model-checking techniques by taking the cumulative sums of residuals over certain coordinates (e.g., covariates or fitted values) or by considering some related aggregates of residuals, such as moving sums and moving averages. For a variety of statistical models and data structures, including generalized linear models with independent or dependent observations, the distributions of these stochastic processes tinder the assumed model can be approximated by the distributions of certain zero-mean Gaussian processes whose realizations can be easily generated by computer simulation. Each observed process can then be compared, both graphically and numerically, with a number of realizations from the Gaussian process. Such comparisons enable one to assess objectively whether a trend seen in a residual plot reflects model misspecification or natural variation. The proposed techniques are particularly useful in checking the functional form of a covariate and the link function. Illustrations with several medical studies are provided.     

From inverse problems in mathematical physiology to quantitative differential diagnoses

The improved capacity to acquire quantitative data in a clinical setting has generally failed to improve outcomes in acutely ill patients, suggesting a need for advances in computer-supported data interpretation and decision making. In particular, the application of mathematical models of experimentally elucidated physiological mechanisms could augment the interpretation of quantitative, patient-specific information and help to better target therapy. Yet, such models are typically complex and nonlinear, a reality that often precludes the identification of unique parameters and states of the model that best represent available data. Hypothesizing that this non-uniqueness can convey useful information, we implemented a simplified simulation of a common differential diagnostic process (hypotension in an acute care setting), using a combination of a mathematical model of the cardiovascular system, a stochastic measurement model, and Bayesian inference techniques to quantify parameter and state uncertainty. The output of this procedure is a probability density function on the space of model parameters and initial conditions for a particular patient, based on prior population information together with patient-specific clinical observations. We show that multimodal posterior probability density functions arise naturally, even when unimodal and uninformative priors are used. The peaks of these densities correspond to clinically relevant differential diagnoses and can, in the simplified simulation setting, be constrained to a single diagnosis by assimilating additional observations from dynamical interventions (e.g., fluid challenge). We conclude that the ill-posedness of the inverse problem in quantitative physiology is not merely a technical obstacle, but rather reflects clinical reality and, when addressed adequately in the solution process, provides a novel link between mathematically described physiological knowledge and the clinical concept of differential diagnoses. We outline possible steps toward translating this computational approach to the bedside, to supplement today's evidence-based medicine with a quantitatively founded model-based medicine that integrates mechanistic knowledge with patient-specific information.     

The Embedding Problem for Markov Models of Nucleotide Substitution

Continuous-time Markov processes are often used to model the complex natural phenomenon of sequence evolution. To make the process of sequence evolution tractable, simplifying assumptions are often made about the sequence properties and the underlying process. The validity of one such assumption, time-homogeneity, has never been explored. Violations of this assumption can be found by identifying non-embeddability. A process is non-embeddable if it can not be embedded in a continuous time-homogeneous Markov process. In this study, non-embeddability was demonstrated to exist when modelling sequence evolution with Markov models. Evidence of non-embeddability was found primarily at the third codon position, possibly resulting from changes in mutation rate over time. Outgroup edges and those with a deeper time depth were found to have an increased probability of the underlying process being non-embeddable. Overall, low levels of non-embeddability were detected when examining individual edges of triads across a diverse set of alignments. Subsequent phylogenetic reconstruction analyses demonstrated that non-embeddability could impact on the correct prediction of phylogenies, but at extremely low levels. Despite the existence of non-embeddability, there is minimal evidence of violations of the local time homogeneity assumption and consequently the impact is likely to be minor.     

Continuous evolution of statistical estimators for optimal decision-making

In many everyday situations, humans must make precise decisions in the presence of uncertain sensory information. For example, when asked to combine information from multiple sources we often assign greater weight to the more reliable information. It has been proposed that statistical-optimality often observed in human perception and decision-making requires that humans have access to the uncertainty of both their senses and their decisions. However, the mechanisms underlying the processes of uncertainty estimation remain largely unexplored. In this paper we introduce a novel visual tracking experiment that requires subjects to continuously report their evolving perception of the mean and uncertainty of noisy visual cues over time. We show that subjects accumulate sensory information over the course of a trial to form a continuous estimate of the mean, hindered only by natural kinematic constraints (sensorimotor latency etc.). Furthermore, subjects have access to a measure of their continuous objective uncertainty, rapidly acquired from sensory information available within a trial, but limited by natural kinematic constraints and a conservative margin for error. Our results provide the first direct evidence of the continuous mean and uncertainty estimation mechanisms in humans that may underlie optimal decision making.     

Distribution models of estuarine fish species: The effect of sampling bias,species ecology and threshold selection on models' accuracy

Species distribution models (SDMs) relate presence/absence data to environmental variables, allowing to predict species environmental requirements and potential distribution. They have been increasingly used in fields such as ecology, biogeography and evolution, and often support conservation priorities and strategies. Thus, it becomes crucial to understand how trustworthy and reliable their predictions are. Different approaches, such as using ensemble methods (combining forecasts of different single models), or applying the most suitable threshold to transform continuous probability maps into species presences or absences, have been used to reduce model-based uncertainty. Taking into account the influence of biased sampling imprecision in species location, small datasets and species ecological characteristics, may also help to detect and compensate for uncertainty in the model building process. To investigate the effect of applying an ensemble approach, several threshold selection criteria and different datasets representing seasonal and spatial sampling bias, on models' accuracy, SDMs were built for four estuarine fish species with distinct use of the estuarine systems. Overall, predictions obtained with the ensemble approach were more accurate. Variability in accuracy metrics obtained with the nine threshold selection criteria applied was more pronounced for species with low prevalence and when sensitivity was calculated. Higher values of accuracy measures were registered with the threshold that maximizes the sum of sensitivity and specificity, and the threshold where the predicted prevalence equals the observed, whereas the 0.5 cut-off was unreliable, originating the lowest values for these metrics. Accuracy of models created from a spatially biased sampling was overall higher than accuracy of models created with a seasonally biased sampling or with the multi-year database created and this pattern was consistently obtained for marine migrant species, which use estuaries as nursery areas, presenting a seasonally and regular use of these ecosystems. The ecological dependence between these fish species and estuaries may add difficulties in the model building process, and needs to be taken into account, to improve their accuracy. The present study highlights the need for a thorough analysis of the critical underlying issues of the complete model building process to predict the distribution of estuarine fish species, due to the particular and dynamic nature of these ecosystems.     

The n = 1 constraint in population genomics

A key objective of population genomics is to identify portions of the genome that have been shaped by natural selection rather than by neutral divergence. A previously recognized but underappreciated challenge to this objective is that observations of allele frequencies across genomes in natural populations often correspond to a single, unreplicated instance of the outcome of evolution. This is because the composition of each individual genomic region and population is expected to be the outcome of a unique array of evolutionary processes. Given a single observation, inference of the evolutionary processes that led to the observed state of a locus is associated with considerable uncertainty. This constraint on inference can be ameliorated by utilizing multi-allelic (e.g. DNA haplotypes) rather than bi-allelic markers, by analysing two or more populations with certain models and by utilizing studies of replicated experimental evolution. Future progress in population genomics will follow from research that recognizes the 'n = 1 constraint' and that utilizes appropriate and explicit evolutionary models for analysis.     

A transformation‐based approach to Gaussian mixture density estimation for bounded data

Finite mixture of Gaussian distributions provide a flexible semiparametric methodology for density estimation when the continuous variables under investigation have no boundaries. However, in practical applications, variables may be partially bounded (e.g., taking nonnegative values) or completely bounded (e.g., taking values in the unit interval). In this case, the standard Gaussian finite mixture model assigns nonzero densities to any possible values, even to those outside the ranges where the variables are defined, hence resulting in potentially severe bias. In this paper, we propose a transformation‐based approach for Gaussian mixture modeling in case of bounded variables. The basic idea is to carry out density estimation not on the original data but on appropriately transformed data. Then, the density for the original data can be obtained by a change of variables. Both the transformation parameters and the parameters of the Gaussian mixture are jointly estimated by the expectation‐maximization (EM) algorithm. The methodology for partially and completely bounded data is illustrated using both simulated data and real data applications.     

Preservation of dynamic properties in qualitative modeling frameworks for gene regulatory networks

Mathematical modeling often helps to provide a systems perspective on gene regulatory networks. In particular, qualitative approaches are useful when detailed kinetic information is lacking. Multiple methods have been developed that implement qualitative information in different ways, e.g., in purely discrete or hybrid discrete/continuous models. In this paper, we compare the discrete asynchronous logical modeling formalism for gene regulatory networks due to R. Thomas with piecewise affine differential equation models. We provide a local characterization of the qualitative dynamics of a piecewise affine differential equation model using the discrete dynamics of a corresponding Thomas model. Based on this result, we investigate the consistency of higher-level dynamical properties such as attractor characteristics and reachability. We show that although the two approaches are based on equivalent information, the resulting qualitative dynamics are different. In particular, the dynamics of the piecewise affine differential equation model is not a simple refinement of the dynamics of the Thomas model     

Discrete event, multi-level simulation of metabolite channeling

Typically differential equations are employed to simulate cellular dynamics. To develop a valid continuous model based on differential equations requires accurate parameter estimations; an accuracy which is often difficult to achieve, due to the lack of data. In addition, processes in metabolic pathways, e.g. metabolite channeling, seem to be of a rather qualitative and discrete nature. With respect to the available data and to the perception of the underlying system, a discrete rather than a continuous approach to modeling and simulation seems more adequate. A discrete approach does not necessarily imply a more abstract view on the system. If we move from macro to micro and multi-level modeling, aspects of subsystems and their interactions, which have been only implicitly represented, become an explicit part of the model. To start exploring discrete event phenomena within metabolite channeling we choose the tryptophan synthase. Based on a continuous macro model, a discrete event, multi-level model is developed which allows us to analyze the interrelation between structural and functional characteristics of the enzymes.     

The ideal free distribution as an evolutionarily stable strategy

We examine the evolutionary stability of strategies for dispersal in heterogeneous patchy environments or for switching between discrete states (e.g. defended and undefended) in the context of models for population dynamics or species interactions in either continuous or discrete time. There have been a number of theoretical studies that support the view that in spatially heterogeneous but temporally constant environments there will be selection against unconditional, i.e. random, dispersal, but there may be selection for certain types of dispersal that are conditional in the sense that dispersal rates depend on environmental factors. A particular type of dispersal strategy that has been shown to be evolutionarily stable in some settings is balanced dispersal, in which the equilibrium densities of organisms on each patch are the same whether there is dispersal or not. Balanced dispersal leads to a population distribution that is ideal free in the sense that at equilibrium all individuals have the same fitness and there is no net movement of individuals between patches or states. We find that under rather general assumptions about the underlying population dynamics or species interactions, only such ideal free strategies can be evolutionarily stable. Under somewhat more restrictive assumptions (but still in considerable generality), we show that ideal free strategies are indeed evolutionarily stable. Our main mathematical approach is invasibility analysis using methods from the theory of ordinary differential equations and nonnegative matrices. Our analysis unifies and extends previous results on the evolutionary stability of dispersal or state-switching strategies.     

Spatial Uncertainty and Ecological Models

Applied ecological models that are used to understand and manage natural systems often rely on spatial data as input. Spatial uncertainty in these data can propagate into model predictions. Uncertainty analysis, sensitivity analysis, error analysis, error budget analysis, spatial decision analysis, and hypothesis testing using neutral models are all techniques designed to explore the relationship between variation in model inputs and variation in model predictions. Although similar methods can be used to answer them, these approaches address different questions. These approaches differ in (a) whether the focus is forward or backward (forward to evaluate the magnitude of variation in model predictions propagated or backward to rank input parameters by their influence); (b) whether the question involves model robustness to large variations in spatial pattern or to small deviations from a reference map; and (c) whether processes that generate input uncertainty (for example, cartographic error) are of interest. In this commentary, we propose a taxonomy of approaches, all of which clarify the relationship between spatial uncertainty and the predictions of ecological models. We describe existing techniques and indicate a few areas where research is needed.     

Gaussian processes for machine learning

Gaussian processes (GPs) are natural generalisations of multivariate Gaussian random variables to infinite (countably or continuous) index sets. GPs have been applied in a large number of fields to a diverse range of ends, and very many deep theoretical analyses of various properties are available. This paper gives an introduction to Gaussian processes on a fairly elementary level with special emphasis on characteristics relevant in machine learning. It draws explicit connections to branches such as spline smoothing models and support vector machines in which similar ideas have been investigated. Gaussian process models are routinely used to solve hard machine learning problems. They are attractive because of their flexible non-parametric nature and computational simplicity. Treated within a Bayesian framework, very powerful statistical methods can be implemented which offer valid estimates of uncertainties in our predictions and generic model selection procedures cast as nonlinear optimization problems. Their main drawback of heavy computational scaling has recently been alleviated by the introduction of generic sparse approximations.13,78,31 The mathematical literature on GPs is large and often uses deep concepts which are not required to fully understand most machine learning applications. In this tutorial paper, we aim to present characteristics of GPs relevant to machine learning and to show up precise connections to other "kernel machines" popular in the community. Our focus is on a simple presentation, but references to more detailed sources are provided.     

Global analysis of dynamical decision-making models through local computation around the hidden saddle

Bistable dynamical switches are frequently encountered in mathematical modeling of biological systems because binary decisions are at the core of many cellular processes. Bistable switches present two stable steady-states, each of them corresponding to a distinct decision. In response to a transient signal, the system can flip back and forth between these two stable steady-states, switching between both decisions. Understanding which parameters and states affect this switch between stable states may shed light on the mechanisms underlying the decision-making process. Yet, answering such a question involves analyzing the global dynamical (i.e., transient) behavior of a nonlinear, possibly high dimensional model. In this paper, we show how a local analysis at a particular equilibrium point of bistable systems is highly relevant to understand the global properties of the switching system. The local analysis is performed at the saddle point, an often disregarded equilibrium point of bistable models but which is shown to be a key ruler of the decision-making process. Results are illustrated on three previously published models of biological switches: two models of apoptosis, the programmed cell death and one model of long-term potentiation, a phenomenon underlying synaptic plasticity.     

From molecular networks to qualitative cell behavior

Adaptation and behavior are characteristics of life which are fundamentally dynamic. If we want to model the living cell we have to describe it as a dynamic system. Typical dynamic models are based on quantitative differential equations requiring very detailed kinetic knowledge. Alternative modeling techniques for less fine-grained information are better suited to available functional genomics data. As such, constraint-based techniques and qualitative modeling have proven themselves to be valid approaches in cell biology. These approaches offer formal support to check the consistency of molecular networks against phenotypic observations in the light of dynamic systems.     

Decision Support Methods for Finding Phenotype — Disorder Associations in the Bone Dysplasia Domain

A lack of mature domain knowledge and well established guidelines makes the medical diagnosis of skeletal dysplasias (a group of rare genetic disorders) a very complex process. Machine learning techniques can facilitate objective interpretation of medical observations for the purposes of decision support. However, building decision support models using such techniques is highly problematic in the context of rare genetic disorders, because it depends on access to mature domain knowledge. This paper describes an approach for developing a decision support model in medical domains that are underpinned by relatively sparse knowledge bases. We propose a solution that combines association rule mining with the Dempster-Shafer theory (DST) to compute probabilistic associations between sets of clinical features and disorders, which can then serve as support for medical decision making (e.g., diagnosis). We show, via experimental results, that our approach is able to provide meaningful outcomes even on small datasets with sparse distributions, in addition to outperforming other Machine Learning techniques and behaving slightly better than an initial diagnosis by a clinician.     

The ideal free distribution as an evolutionarily stable strategy

We examine the evolutionary stability of strategies for dispersal in heterogeneous patchy environments or for switching between discrete states (e.g. defended and undefended) in the context of models for population dynamics or species interactions in either continuous or discrete time. There have been a number of theoretical studies that support the view that in spatially heterogeneous but temporally constant environments there will be selection against unconditional, i.e. random, dispersal, but there may be selection for certain types of dispersal that are conditional in the sense that dispersal rates depend on environmental factors. A particular type of dispersal strategy that has been shown to be evolutionarily stable in some settings is balanced dispersal, in which the equilibrium densities of organisms on each patch are the same whether there is dispersal or not. Balanced dispersal leads to a population distribution that is ideal free in the sense that at equilibrium all individuals have the same fitness and there is no net movement of individuals between patches or states. We find that under rather general assumptions about the underlying population dynamics or species interactions, only such ideal free strategies can be evolutionarily stable. Under somewhat more restrictive assumptions (but still in considerable generality), we show that ideal free strategies are indeed evolutionarily stable. Our main mathematical approach is invasibility analysis using methods from the theory of ordinary differential equations and nonnegative matrices. Our analysis unifies and extends previous results on the evolutionary stability of dispersal or state-switching strategies.     

Fuzzy Stochastic Petri Nets for Modeling Biological Systems with Uncertain Kinetic Parameters

Stochastic Petri nets (SPNs) have been widely used to model randomness which is an inherent feature of biological systems. However, for many biological systems, some kinetic parameters may be uncertain due to incomplete, vague or missing kinetic data (often called fuzzy uncertainty), or naturally vary, e.g., between different individuals, experimental conditions, etc. (often called variability), which has prevented a wider application of SPNs that require accurate parameters. Considering the strength of fuzzy sets to deal with uncertain information, we apply a specific type of stochastic Petri nets, fuzzy stochastic Petri nets (FSPNs), to model and analyze biological systems with uncertain kinetic parameters. FSPNs combine SPNs and fuzzy sets, thereby taking into account both randomness and fuzziness of biological systems. For a biological system, SPNs model the randomness, while fuzzy sets model kinetic parameters with fuzzy uncertainty or variability by associating each parameter with a fuzzy number instead of a crisp real value. We introduce a simulation-based analysis method for FSPNs to explore the uncertainties of outputs resulting from the uncertainties associated with input parameters, which works equally well for bounded and unbounded models. We illustrate our approach using a yeast polarization model having an infinite state space, which shows the appropriateness of FSPNs in combination with simulation-based analysis for modeling and analyzing biological systems with uncertain information.     

A non-stochastic approach for modeling uncertainty in population dynamics

We developed a non-stochastic methodology to deal with the uncertainty in models of population dynamics. This approach assumed that noise is bounded; it led to models based on differential inclusions rather than stochastic processes, and avoided stochastic calculus. Examples of estimations of extinction times for exponential and logistic population growth with environmental and demographic noise are presented.