Estimating parameters for generalized mass action models using constraint propagation

As modern molecular biology moves towards the analysis of biological systems as opposed to their individual components, the need for appropriate mathematical and computational techniques for understanding the dynamics and structure of such systems is becoming more pressing. For example, the modeling of biochemical systems using ordinary differential equations (ODEs) based on high-throughput, time-dense profiles is becoming more common-place, which is necessitating the development of improved techniques to estimate model parameters from such data. Due to the high dimensionality of this estimation problem, straight-forward optimization strategies rarely produce correct parameter values, and hence current methods tend to utilize genetic/evolutionary algorithms to perform non-linear parameter fitting. Here, we describe a completely deterministic approach, which is based on interval analysis. This allows us to examine entire sets of parameters, and thus to exhaust the global search within a finite number of steps. In particular, we show how our method may be applied to a generic class of ODEs used for modeling biochemical systems called Generalized Mass Action Models (GMAs). In addition, we show that for GMAs our method is amenable to the technique in interval arithmetic called constraint propagation, which allows great improvement of its efficiency. To illustrate the applicability of our method we apply it to some networks of biochemical reactions appearing in the literature, showing in particular that, in addition to estimating system parameters in the absence of noise, our method may also be used to recover the topology of these networks.     

Practical limits for reverse engineering of dynamical systems: a statistical analysis of sensitivity and parameter inferability in systems biology models

The size and complexity of cellular systems make building predictive models an extremely difficult task. In principle dynamical time-course data can be used to elucidate the structure of the underlying molecular mechanisms, but a central and recurring problem is that many and very different models can be fitted to experimental data, especially when the latter are limited and subject to noise. Even given a model, estimating its parameters remains challenging in real-world systems. Here we present a comprehensive analysis of 180 systems biology models, which allows us to classify the parameters with respect to their contribution to the overall dynamical behaviour of the different systems. Our results reveal candidate elements of control in biochemical pathways that differentially contribute to dynamics. We introduce sensitivity profiles that concisely characterize parameter sensitivity and demonstrate how this can be connected to variability in data. Systematically linking data and model sloppiness allows us to extract features of dynamical systems that determine how well parameters can be estimated from time-course measurements, and associates the extent of data required for parameter inference with the model structure, and also with the global dynamical state of the system. The comprehensive analysis of so many systems biology models reaffirms the inability to estimate precisely most model or kinetic parameters as a generic feature of dynamical systems, and provides safe guidelines for performing better inferences and model predictions in the context of reverse engineering of mathematical models for biological systems.     

A method for solution of the multi-objective inverse problems under uncertainty

We describe a method to solve multi-objective inverse problems under uncertainty. The method was tested on non-linear models of dynamic series and population dynamics, as well as on the spatiotemporal model of gene expression in terms of non-linear differential equations. We consider how to identify model parameters when experimental data contain additive noise and measurements are performed in discrete time points. We formulate the multi-objective problem of optimization under uncertainty. In addition to a criterion of least squares difference we applied a criterion which is based on the integral of trajectories of the system spatiotemporal dynamics, as well as a heuristic criterion CHAOS based on the decision tree method. The optimization problem is formulated using a fuzzy statement and is constrained by penalty functions based on the normalized membership functions of a fuzzy set of model solutions. This allows us to reconstruct the expression pattern of hairy gene in Drosophila even-skipped mutants that is in good agreement with experimental data. The reproducibility of obtained results is confirmed by solution of inverse problems using different global optimization methods with heuristic strategies.     

Can We Identify Non-Stationary Dynamics of Trial-to-Trial Variability?

Identifying sources of the apparent variability in non-stationary scenarios is a fundamental problem in many biological data analysis settings. For instance, neurophysiological responses to the same task often vary from each repetition of the same experiment (trial) to the next. The origin and functional role of this observed variability is one of the fundamental questions in neuroscience. The nature of such trial-to-trial dynamics however remains largely elusive to current data analysis approaches. A range of strategies have been proposed in modalities such as electro-encephalography but gaining a fundamental insight into latent sources of trial-to-trial variability in neural recordings is still a major challenge. In this paper, we present a proof-of-concept study to the analysis of trial-to-trial variability dynamics founded on non-autonomous dynamical systems. At this initial stage, we evaluate the capacity of a simple statistic based on the behaviour of trajectories in classification settings, the trajectory coherence, in order to identify trial-to-trial dynamics. First, we derive the conditions leading to observable changes in datasets generated by a compact dynamical system (the Duffing equation). This canonical system plays the role of a ubiquitous model of non-stationary supervised classification problems. Second, we estimate the coherence of class-trajectories in empirically reconstructed space of system states. We show how this analysis can discern variations attributable to non-autonomous deterministic processes from stochastic fluctuations. The analyses are benchmarked using simulated and two different real datasets which have been shown to exhibit attractor dynamics. As an illustrative example, we focused on the analysis of the rat''s frontal cortex ensemble dynamics during a decision-making task. Results suggest that, in line with recent hypotheses, rather than internal noise, it is the deterministic trend which most likely underlies the observed trial-to-trial variability. Thus, the empirical tool developed within this study potentially allows us to infer the source of variability in in-vivo neural recordings.     

Foreseeing the future of mutualistic communities beyond collapse

Changing conditions may lead to sudden shifts in the state of ecosystems when critical thresholds are passed. Some well‐studied drivers of such transitions lead to predictable outcomes such as a turbid lake or a degraded landscape. Many ecosystems are, however, complex systems of many interacting species. While detecting upcoming transitions in such systems is challenging, predicting what comes after a critical transition is terra incognita altogether. The problem is that complex ecosystems may shift to many different, alternative states. Whether an impending transition has minor, positive or catastrophic effects is thus unclear. Some systems may, however, behave more predictably than others. The dynamics of mutualistic communities can be expected to be relatively simple, because delayed negative feedbacks leading to oscillatory or other complex dynamics are weak. Here, we address the question of whether this relative simplicity allows us to foresee a community's future state. As a case study, we use a model of a bipartite mutualistic network and show that a network's post‐transition state is indicated by the way in which a system recovers from minor disturbances. Similar results obtained with a unipartite model of facilitation suggest that our results are of relevance to a wide range of mutualistic systems.     

Noise enhanced hormonal signal transduction through intracellular calcium oscillations.

In a wide range of non-linear dynamical systems, noise may enhance the detection of weak deterministic input signals. Here, we demonstrate this phenomenon for transmembrane signaling in a hormonal model system of intracellular Ca(2+) oscillations. Adding Gaussian noise to a subthreshold extracellular pulsatile stimulus increased the sensitivity in the dose-response relation of the Ca(2+) oscillations compared to the same noise signal added as a constant mean level. These findings may have important physiological consequences for the operation of hormonal and other physiological signal transduction systems close to the threshold level.     

Studying User Income through Language,Behaviour and Affect in Social Media

Automatically inferring user demographics from social media posts is useful for both social science research and a range of downstream applications in marketing and politics. We present the first extensive study where user behaviour on Twitter is used to build a predictive model of income. We apply non-linear methods for regression, i.e. Gaussian Processes, achieving strong correlation between predicted and actual user income. This allows us to shed light on the factors that characterise income on Twitter and analyse their interplay with user emotions and sentiment, perceived psycho-demographics and language use expressed through the topics of their posts. Our analysis uncovers correlations between different feature categories and income, some of which reflect common belief e.g. higher perceived education and intelligence indicates higher earnings, known differences e.g. gender and age differences, however, others show novel findings e.g. higher income users express more fear and anger, whereas lower income users express more of the time emotion and opinions.     

Proliferation and competition in discrete biological systems

We study the emergence of collective spatio-temporal objects in biological systems by representing individually the elementary interactions between their microscopic components. We use the immune system as a prototype for such interactions. The results of this detailed explicit analysis are compared with the traditional procedure of representing the collective dynamics in terms of densities that obey partial differential equations. The simulations show even for very simple elementary reactions the spontaneous emergence of localized complex structures, from microscopic noise. In turn the effective dynamics of these structures affects the average behaviour of the system in a very decisive way: systems which would according to the differential equations approximation die, display in reality a very lively behaviour. As the optimal modelling method we propose a mixture of microscopic simulation systems describing each reaction separately, and continuous methods describing the average behaviour of the agents.     

Exact analytical results for integrate-and-fire neurons driven by excitatory shot noise

A neuron receives input from other neurons via electrical pulses, so-called spikes. The pulse-like nature of the input is frequently neglected in analytical studies; instead, the input is usually approximated to be Gaussian. Recent experimental studies have shown, however, that an assumption underlying this approximation is often not met: Individual presynaptic spikes can have a significant effect on a neuron’s dynamics. It is thus desirable to explicitly account for the pulse-like nature of neural input, i.e. consider neurons driven by a shot noise – a long-standing problem that is mathematically challenging. In this work, we exploit the fact that excitatory shot noise with exponentially distributed weights can be obtained as a limit case of dichotomous noise, a Markovian two-state process. This allows us to obtain novel exact expressions for the stationary voltage density and the moments of the interspike-interval density of general integrate-and-fire neurons driven by such an input. For the special case of leaky integrate-and-fire neurons, we also give expressions for the power spectrum and the linear response to a signal. We verify and illustrate our expressions by comparison to simulations of leaky-, quadratic- and exponential integrate-and-fire neurons.     

Reconstructing the dynamics of unobserved variables in spatially extended systems

Attractor reconstruction using embedding techniques is a widely used tool when analysing data from real systems. It allows reconstruction of the system dynamics from only one observable and is thus extremely powerful. We show here that this reconstruction is also possible from spatially coupled systems. We use a common host–parasitoid model as an example as ecological systems are virtually always spatially extended. Additionally, data from ecological systems has often only one observable, e.g. population density, from a potentially much higher-dimensional system. Singular value decomposition is used to show the existence of a functional relationship mapping the time delayed coordinates of one variable to the full spatially coupled system. We investigate the effects of noise and indicate two important spatial scales. Finally, we illustrate that a reconstruction can be obtained from a system that is only partially sampled.     

Quantitative analysis and model simplification of an epidemic model with primary and secondary infection

Models of particular epidemiological systems can rapidly become complicated by biological detail which can obscure their essential features and behaviour. In general, we wish to retain only those components and processes that contribute to the dynamics of the system. In this paper, we apply asymptotic techniques to an SEI-type model with primary and secondary infection in order to reduce it to a much simpler form. This allows the identification of parameter groupings discriminating between regions of contrasting dynamics and leads to simple approximations for the model’s transient behaviour. These can be used to follow the evolution of the developing infection process. The techniques examined in this paper will be applicable to a large number of similar models.     

A rigorous approach to investigating common assumptions about disease transmission

Changing scale, for example, the ability to move seamlessly from an individual-based model to a population-based model, is an important problem in many fields. In this paper, we introduce process algebra as a novel solution to this problem in the context of models of infectious disease spread. Process algebra allows us to describe a system in terms of the stochastic behaviour of individuals, and is a technique from computer science. We review the use of process algebra in biological systems, and the variety of quantitative and qualitative analysis techniques available. The analysis illustrated here solves the changing scale problem: from the individual behaviour we can rigorously derive equations to describe the mean behaviour of the system at the level of the population. The biological problem investigated is the transmission of infection, and how this relates to individual interactions.     

Trophic structure and dynamical complexity in simple ecological models

We study the dynamical complexity of five non-linear deterministic predator–prey model systems. These simple systems were selected to represent a diversity of trophic structures and ecological interactions in the real world while still preserving reasonable tractability. We find that these systems can dramatically change attractor types, and the switching among different attractors is dependent on system parameters. While dynamical complexity depends on the nature (e.g., inter-specific competition versus predation) and degree (e.g., number of interacting components) of trophic structure present in the system, these systems all evolve principally on intrinsically noisy limit cycles. Our results support the common observation of cycling and rare observation of chaos in natural populations. Our study also allows us to speculate on the functional role of specialist versus generalist predators in food web modeling.     

Quantifying Dynamic Regulation in Metabolic Pathways with Nonparametric Flux Inference

One of the central tasks in systems biology is to understand how cells regulate their metabolism. Hierarchical regulation analysis is a powerful tool to study this regulation at the metabolic, gene-expression, and signaling levels. It has been widely applied to study steady-state regulation, but analysis of the metabolic dynamics remains challenging because it is difficult to measure time-dependent metabolic flux. Here, we develop a nonparametric method that uses Gaussian processes to accurately infer the dynamics of a metabolic pathway based only on metabolite measurements; from this, we then go on to obtain a dynamical view of the hierarchical regulation processes invoked over time to control the activity in a pathway. Our approach allows us to use hierarchical regulation analysis in a dynamic setting but without the need for explicitly time-dependent flux measurements.