Gene network inference using continuous time Bayesian networks: a comparative study and application to Th17 cell differentiation

Background

Dynamic aspects of gene regulatory networks are typically investigated by measuring system variables at multiple time points. Current state-of-the-art computational approaches for reconstructing gene networks directly build on such data, making a strong assumption that the system evolves in a synchronous fashion at fixed points in time. However, nowadays omics data are being generated with increasing time course granularity. Thus, modellers now have the possibility to represent the system as evolving in continuous time and to improve the models’ expressiveness.

Results

Continuous time Bayesian networks are proposed as a new approach for gene network reconstruction from time course expression data. Their performance was compared to two state-of-the-art methods: dynamic Bayesian networks and Granger causality analysis. On simulated data, the methods comparison was carried out for networks of increasing size, for measurements taken at different time granularity densities and for measurements unevenly spaced over time. Continuous time Bayesian networks outperformed the other methods in terms of the accuracy of regulatory interactions learnt from data for all network sizes. Furthermore, their performance degraded smoothly as the size of the network increased. Continuous time Bayesian networks were significantly better than dynamic Bayesian networks for all time granularities tested and better than Granger causality for dense time series. Both continuous time Bayesian networks and Granger causality performed robustly for unevenly spaced time series, with no significant loss of performance compared to the evenly spaced case, while the same did not hold true for dynamic Bayesian networks. The comparison included the IRMA experimental datasets which confirmed the effectiveness of the proposed method. Continuous time Bayesian networks were then applied to elucidate the regulatory mechanisms controlling murine T helper 17 (Th17) cell differentiation and were found to be effective in discovering well-known regulatory mechanisms, as well as new plausible biological insights.

Conclusions

Continuous time Bayesian networks were effective on networks of both small and large size and were particularly feasible when the measurements were not evenly distributed over time. Reconstruction of the murine Th17 cell differentiation network using continuous time Bayesian networks revealed several autocrine loops, suggesting that Th17 cells may be auto regulating their own differentiation process.     

Continuous-index hidden Markov modelling of array CGH copy number data

MOTIVATION: In recent years, a range of techniques for analysis and segmentation of array comparative genomic hybridization (aCGH) data have been proposed. For array designs in which clones are of unequal lengths, are unevenly spaced or overlap, the discrete-index view typically adopted by such methods may be questionable or improved. RESULTS: We describe a continuous-index hidden Markov model for aCGH data as well as a Monte Carlo EM algorithm to estimate its parameters. It is shown that for a dataset from the BT-474 cell line analysed on 32K BAC tiling microarrays, this model yields considerably better model fit in terms of lag-1 residual autocorrelations compared to a discrete-index HMM, and it is also shown how to use the model for e.g. estimation of change points on the base-pair scale and for estimation of conditional state probabilities across the genome. In addition, the model is applied to the Glioblastoma Multiforme data used in the comparative study by Lai et al. (Lai,W.R. et al. (2005) Comparative analysis of algorithms for identifying amplifications and deletions in array CGH data. Bioinformatics, 21, 3763-3370.) giving result similar to theirs but with certain features highlighted in the continuous-index setting.     

Bayesian Finite Markov Mixture Model for Temporal Multi‐Tissue Polygenic Patterns

Finite mixture models can provide the insights about behavioral patterns as a source of heterogeneity of the various dynamics of time course gene expression data by reducing the high dimensionality and making clear the major components of the underlying structure of the data in terms of the unobservable latent variables. The latent structure of the dynamic transition process of gene expression changes over time can be represented by Markov processes. This paper addresses key problems in the analysis of large gene expression data sets that describe systemic temporal response cascades and dynamic changes to therapeutic doses in multiple tissues, such as liver, skeletal muscle, and kidney from the same animals. Bayesian Finite Markov Mixture Model with a Dirichlet Prior is developed for the identifications of differentially expressed time related genes and dynamic clusters. Deviance information criterion is applied to determine the number of components for model comparisons and selections. The proposed Bayesian models are applied to multiple tissue polygenetic temporal gene expression data and compared to a Bayesian model‐based clustering method, named CAGED. Results show that our proposed Bayesian Finite Markov Mixture model can well capture the dynamic changes and patterns for irregular complex temporal data (© 2009 WILEY‐VCH Verlag GmbH & Co. KGaA, Weinheim)     

Detecting periodic patterns in unevenly spaced gene expression time series using Lomb-Scargle periodograms

MOTIVATION: Periodic patterns in time series resulting from biological experiments are of great interest. The commonly used Fast Fourier Transform (FFT) algorithm is applicable only when data are evenly spaced and when no values are missing, which is not always the case in high-throughput measurements. The choice of statistic to evaluate the significance of the periodic patterns for unevenly spaced gene expression time series has not been well substantiated. METHODS: The Lomb-Scargle periodogram approach is used to search time series of gene expression to quantify the periodic behavior of every gene represented on the DNA array. The Lomb-Scargle periodogram analysis provides a direct method to treat missing values and unevenly spaced time points. We propose the combination of a Lomb-Scargle test statistic for periodicity and a multiple hypothesis testing procedure with controlled false discovery rate to detect significant periodic gene expression patterns. RESULTS: We analyzed the Plasmodium falciparum gene expression dataset. In the Quality Control Dataset of 5080 expression patterns, we found 4112 periodic probes. In addition, we identified 243 probes with periodic expression in the Complete Dataset, which could not be examined in the original study by the FFT analysis due to an excessive number of missing values. While most periodic genes had a period of 48 h, some had a period close to 24 h. Our approach should be applicable for detection and quantification of periodic patterns in any unevenly spaced gene expression time-series data.     

Active inference and agency: optimal control without cost functions

This paper describes a variational free-energy formulation of (partially observable) Markov decision problems in decision making under uncertainty. We show that optimal control can be cast as active inference. In active inference, both action and posterior beliefs about hidden states minimise a free energy bound on the negative log-likelihood of observed states, under a generative model. In this setting, reward or cost functions are absorbed into prior beliefs about state transitions and terminal states. Effectively, this converts optimal control into a pure inference problem, enabling the application of standard Bayesian filtering techniques. We then consider optimal trajectories that rest on posterior beliefs about hidden states in the future. Crucially, this entails modelling control as a hidden state that endows the generative model with a representation of agency. This leads to a distinction between models with and without inference on hidden control states; namely, agency-free and agency-based models, respectively.     

A Bayesian autoregressive three-state hidden Markov model for identifying switching monotonic regimes in microarray time course data

When modeling time course microarray data special interest may reside in identifying time frames in which gene expression levels follow a monotonic (increasing or decreasing) trend. A trajectory may change its regime because of the reaction to treatment or of a natural developmental phase, as in our motivating example about identification of genes involved in embryo development of mice with the 22q11 deletion. To this aim we propose a new flexible Bayesian autoregressive hidden Markov model based on three latent states, corresponding to stationarity, to an increasing and to a decreasing trend. In order to select a list of genes, we propose decision criteria based on the posterior distribution of the parameters of interest, taking into account the uncertainty in parameter estimates. We also compare the proposed model with two simpler models based on constrained formulations of the probability transition matrix.     

Bayesian Spatiotemporal Inference in Functional Magnetic Resonance Imaging

Mapping of the human brain by means of functional magnetic resonance imaging (fMRI) is an emerging field in cognitive and clinical neuroscience. Current techniques to detect activated areas of the brain mostly proceed in two steps. First, conventional methods of correlation, regression, and time series analysis are used to assess activation by a separate, pixelwise comparison of the fMRI signal time courses to the reference function of a presented stimulus. Spatial aspects caused by correlations between neighboring pixels are considered in a separate second step, if at all. The aim of this article is to present hierarchical Bayesian approaches that allow one to simultaneously incorporate temporal and spatial dependencies between pixels directly in the model formulation. For reasons of computational feasibility, models have to be comparatively parsimonious, without oversimplifying. We introduce parametric and semiparametric spatial and spatiotemporal models that proved appropriate and illustrate their performance applied to visual fMRI data.     

An evaluation of behavior inferences from Bayesian state‐space models: A case study with the Pacific walrus

State‐space models offer researchers an objective approach to modeling complex animal location data sets, and state‐space model behavior classifications are often assumed to have a link to animal behavior. In this study, we evaluated the behavioral classification accuracy of a Bayesian state‐space model in Pacific walruses using Argos satellite tags with sensors to detect animal behavior in real time. We fit a two‐state discrete‐time continuous‐space Bayesian state‐space model to data from 306 Pacific walruses tagged in the Chukchi Sea. We matched predicted locations and behaviors from the state‐space model (resident, transient behavior) to true animal behavior (foraging, swimming, hauled out) and evaluated classification accuracy with kappa statistics (κ) and root mean square error (RMSE). In addition, we compared biased random bridge utilization distributions generated with resident behavior locations to true foraging behavior locations to evaluate differences in space use patterns. Results indicated that the two‐state model fairly classified true animal behavior (0.06 ≤ κ ≤ 0.26, 0.49 ≤ RMSE ≤ 0.59). Kernel overlap metrics indicated utilization distributions generated with resident behavior locations were generally smaller than utilization distributions generated with true foraging behavior locations. Consequently, we encourage researchers to carefully examine parameters and priors associated with behaviors in state‐space models, and reconcile these parameters with the study species and its expected behaviors.     

Discovery of multi-dimensional modules by integrative analysis of cancer genomic data

Recent technology has made it possible to simultaneously perform multi-platform genomic profiling (e.g. DNA methylation (DM) and gene expression (GE)) of biological samples, resulting in so-called ‘multi-dimensional genomic data’. Such data provide unique opportunities to study the coordination between regulatory mechanisms on multiple levels. However, integrative analysis of multi-dimensional genomics data for the discovery of combinatorial patterns is currently lacking. Here, we adopt a joint matrix factorization technique to address this challenge. This method projects multiple types of genomic data onto a common coordinate system, in which heterogeneous variables weighted highly in the same projected direction form a multi-dimensional module (md-module). Genomic variables in such modules are characterized by significant correlations and likely functional associations. We applied this method to the DM, GE, and microRNA expression data of 385 ovarian cancer samples from the The Cancer Genome Atlas project. These md-modules revealed perturbed pathways that would have been overlooked with only a single type of data, uncovered associations between different layers of cellular activities and allowed the identification of clinically distinct patient subgroups. Our study provides an useful protocol for uncovering hidden patterns and their biological implications in multi-dimensional ‘omic’ data.     

Bayesian change-point analyses in ecology

Ecological and biological processes can change from one state to another once a threshold has been crossed in space or time. Threshold responses to incremental changes in underlying variables can characterize diverse processes from climate change to the desertification of arid lands from overgrazing. Simultaneously estimating the location of thresholds and associated ecological parameters can be difficult: ecological data are often 'noisy', which can make the identification of the locations of ecological thresholds challenging. We illustrate this problem using two ecological examples and apply a class of statistical models well-suited to addressing this problem. We first consider the case of estimating allometric relationships between tree diameter and height when the trees have distinctly different growth modes across life-history stages. We next estimate the effects of canopy gaps and dense understory vegetation on tree recruitment in transects that transverse both canopy and gap conditions. The Bayesian change-point models that we present estimate both threshold locations and the slope or level of ecological quantities of interest, while incorporating uncertainty in the change-point location into these estimates. This class of models is suitable for problems with multiple thresholds and can account for spatial or temporal autocorrelation.     

A novel approach for pilot error detection using Dynamic Bayesian Networks

In the last decade Dynamic Bayesian Networks (DBNs) have become one type of the most attractive probabilistic modelling framework extensions of Bayesian Networks (BNs) for working under uncertainties from a temporal perspective. Despite this popularity not many researchers have attempted to study the use of these networks in anomaly detection or the implications of data anomalies on the outcome of such models. An abnormal change in the modelled environment’s data at a given time, will cause a trailing chain effect on data of all related environment variables in current and consecutive time slices. Albeit this effect fades with time, it still can have an ill effect on the outcome of such models. In this paper we propose an algorithm for pilot error detection, using DBNs as the modelling framework for learning and detecting anomalous data. We base our experiments on the actions of an aircraft pilot, and a flight simulator is created for running the experiments. The proposed anomaly detection algorithm has achieved good results in detecting pilot errors and effects on the whole system.     

Modelling regulatory pathways in E. coli from time series expression profiles

MOTIVATION: Cells continuously reprogram their gene expression network as they move through the cell cycle or sense changes in their environment. In order to understand the regulation of cells, time series expression profiles provide a more complete picture than single time point expression profiles. Few analysis techniques, however, are well suited to modelling such time series data. RESULTS: We describe an approach that naturally handles time series data with the capabilities of modelling causality, feedback loops, and environmental or hidden variables using a Dynamic Bayesian network. We also present a novel way of combining prior biological knowledge and current observations to improve the quality of analysis and to model interactions between sets of genes rather than individual genes. Our approach is evaluated on time series expression data measured in response to physiological changes that affect tryptophan metabolism in E. coli. Results indicate that this approach is capable of finding correlations between sets of related genes.     

Bayesian restoration of a hidden Markov chain with applications to DNA sequencing.

Hidden Markov models (HMMs) are a class of stochastic models that have proven to be powerful tools for the analysis of molecular sequence data. A hidden Markov model can be viewed as a black box that generates sequences of observations. The unobservable internal state of the box is stochastic and is determined by a finite state Markov chain. The observable output is stochastic with distribution determined by the state of the hidden Markov chain. We present a Bayesian solution to the problem of restoring the sequence of states visited by the hidden Markov chain from a given sequence of observed outputs. Our approach is based on a Monte Carlo Markov chain algorithm that allows us to draw samples from the full posterior distribution of the hidden Markov chain paths. The problem of estimating the probability of individual paths and the associated Monte Carlo error of these estimates is addressed. The method is illustrated by considering a problem of DNA sequence multiple alignment. The special structure for the hidden Markov model used in the sequence alignment problem is considered in detail. In conclusion, we discuss certain interesting aspects of biological sequence alignments that become accessible through the Bayesian approach to HMM restoration.     

A Probabilistic Treatment of Phylogeny and Sequence Alignment

Carrying out simultaneous tree-building and alignment of sequence data is a difficult computational task, and the methods currently available are either limited to a few sequences or restricted to highly simplified models of alignment and phylogeny. A method is given here for overcoming these limitations by Bayesian sampling of trees and alignments simultaneously. The method uses a standard substitution matrix model for residues together with a hidden Markov model structure that allows affine gap penalties. It escapes the heavy computational burdens of other models by using an approximation called the ``*' rule, which replaces missing data by a sum over all possible values of variables. The behavior of the model is demonstrated on test sets of globins. Received: 25 May 1998 / Accepted: 8 December 1998     

Performance of a new dynamic model for predicting diurnal time courses of stomatal conductance at the leaf level

Under natural conditions, plants are subjected to continuous changes of irradiance that drive variations of stomatal conductance to water vapour (g_s). We propose a dynamic model to predict the temporal response of g_s at the leaf level using an asymmetric sigmoid function with a unique parameter describing time constants for increasing and decreasing g_s. The model parameters were adjusted to observed data using Approximate Bayesian Computation. We tested the model performance for (1) instant changes of irradiance; or (2) continuous and controlled variations of irradiance simulating diurnal time courses. Compared with the two mostly used steady‐state models, our dynamic model described daily time courses of g_s with a higher accuracy. In particular, it was able to describe the hysteresis of g_s responses to increasing/decreasing irradiance and the resulting rapid variations of intrinsic water‐use efficiency. Compared to the mechanistic model of temporal responses of g_s by Kirschbaum, Gross & Pearcy, for which time constants were estimated with a large variance, our model estimated time constants with a higher precision. It is expected to improve predictions of water loss and water‐use efficiency in higher scale models by using a small number of parameters.     

Calibration of multivariate scatter plots for exploratory analysis of relations within and between sets of variables in genomic research

The scatter plot is a well known and easily applicable graphical tool to explore relationships between two quantitative variables. For the exploration of relations between multiple variables, generalisations of the scatter plot are useful. We present an overview of multivariate scatter plots focussing on the following situations. Firstly, we look at a scatter plot for portraying relations between quantitative variables within one data matrix. Secondly, we discuss a similar plot for the case of qualitative variables. Thirdly, we describe scatter plots for the relationships between two sets of variables where we focus on correlations. Finally, we treat plots of the relationships between multiple response and predictor variables, focussing on the matrix of regression coefficients. We will present both known and new results, where an important original contribution concerns a procedure for the inclusion of scales for the variables in multivariate scatter plots. We provide software for drawing such scales. We illustrate the construction and interpretation of the plots by means of examples on data collected in a genomic research program on taste in tomato.     

CLIMTREG: Detecting temporal changes in climate–growth reactions – A computer program using intra-annual daily and yearly moving time intervals of variable width

Analysing climate–growth relationships is one of the key areas in dendrochronological research. One problem however remained unsolved – the discrepancy between the trees continuous growth over the vegetation period and the mean climate variables stretching over much longer periods. Here we present the possibility to calculate climate–growth correlations based on daily climate data using variable temporal width together with moving correlations to accommodate for short term as well as long term influences on tree growth. For the first time this offers the opportunity to acknowledge annual changes in the growing seasonal length and effects of short extreme events. Numerous outputs in data- and graphic-files allows a comparison of varying periods with significant correlations between climate and tree growth. Furthermore the use of climate scenarios is an optional tool for growth predictions.