More discussions for granger causality and new causality measures

Granger causality (GC) has been widely applied in economics and neuroscience to reveal causality influence of time series. In our previous paper (Hu et al., in IEEE Trans on Neural Netw, 22(6), pp. 829–844, 2011), we proposed new causalities in time and frequency domains and particularly focused on new causality in frequency domain by pointing out the shortcomings/limitations of GC or Granger-alike causality metrics and the advantages of new causality. In this paper we continue our previous discussions and focus on new causality and GC or Granger-alike causality metrics in time domain. Although one strong motivation was introduced in our previous paper (Hu et al., in IEEE Trans on Neural Netw, 22(6), pp. 829–844, 2011) we here present additional motivation for the proposed new causality metric and restate the previous motivation for completeness. We point out one property of conditional GC in time domain and the shortcomings/limitations of conditional GC which cannot reveal the real strength of the directional causality among three time series. We also show the shortcomings/limitations of directed causality (DC) or normalize DC for multivariate time series and demonstrate it cannot reveal real causality at all. By calculating GC and new causality values for an example we demonstrate the influence of one of the time series on the other is linearly increased as the coupling strength is linearly increased. This fact further supports reasonability of new causality metric. We point out that larger instantaneous correlation does not necessarily mean larger true causality (e.g., GC and new causality), or vice versa. Finally we conduct analysis of statistical test for significance and asymptotic distribution property of new causality metric by illustrative examples.     

Causality between pillars of sustainable development: Global stylized facts or regional phenomena?

Since the Brundtland report, most governments have committed themselves to sustainable development. For this purpose, different global and national organizations and institutions have tried to find the relationships, and especially causalities, between sustainability pillars, which are interesting for them from a policy point of view. With respect to their findings, some questions need to be answered before appropriate policies can be formulated. Are causalities between sustainability pillars global stylized facts or regional phenomena? Can countries with different characteristics follow the same rules, or are causalities between the pillars sensitive to the regional and intrinsic features of countries? Using principal component analysis for the construction of sustainability indicators and the Granger causality model (GMM approach) for testing the causalities between sustainability pillars in different samples, this study finds that causal patterns among the pillars of sustainability are completely sensitive to the characteristics of the countries that are grouped. Therefore, it is recommended that researchers concentrate more on homogeneous case studies and avoid generalizations of causal relationships between sets of heterogeneous countries.     

A graphical approach for evaluating effective connectivity in neural systems

The identification of effective connectivity from time-series data such as electroencephalogram (EEG) or time-resolved function magnetic resonance imaging (fMRI) recordings is an important problem in brain imaging. One commonly used approach to inference effective connectivity is based on vector autoregressive models and the concept of Granger causality. However, this probabilistic concept of causality can lead to spurious causalities in the presence of latent variables. Recently, graphical models have been used to discuss problems of causal inference for multivariate data. In this paper, we extend these concepts to the case of time-series and present a graphical approach for discussing Granger-causal relationships among multiple time-series. In particular, we propose a new graphical representation that allows the characterization of spurious causality and, thus, can be used to investigate spurious causality. The method is demonstrated with concurrent EEG and fMRI recordings which are used to investigate the interrelations between the alpha rhythm in the EEG and blood oxygenation level dependent (BOLD) responses in the fMRI. The results confirm previous findings on the location of the source of the EEG alpha rhythm.     

Causal measures of structure and plasticity in simulated and living neural networks

A major goal of neuroscience is to understand the relationship between neural structures and their function. Recording of neural activity with arrays of electrodes is a primary tool employed toward this goal. However, the relationships among the neural activity recorded by these arrays are often highly complex making it problematic to accurately quantify a network's structural information and then relate that structure to its function. Current statistical methods including cross correlation and coherence have achieved only modest success in characterizing the structural connectivity. Over the last decade an alternative technique known as Granger causality is emerging within neuroscience. This technique, borrowed from the field of economics, provides a strong mathematical foundation based on linear auto-regression to detect and quantify "causal" relationships among different time series. This paper presents a combination of three Granger based analytical methods that can quickly provide a relatively complete representation of the causal structure within a neural network. These are a simple pairwise Granger causality metric, a conditional metric, and a little known computationally inexpensive subtractive conditional method. Each causal metric is first described and evaluated in a series of biologically plausible neural simulations. We then demonstrate how Granger causality can detect and quantify changes in the strength of those relationships during plasticity using 60 channel spike train data from an in vitro cortical network measured on a microelectrode array. We show that these metrics can not only detect the presence of causal relationships, they also provide crucial information about the strength and direction of that relationship, particularly when that relationship maybe changing during plasticity. Although we focus on the analysis of multichannel spike train data the metrics we describe are applicable to any stationary time series in which causal relationships among multiple measures is desired. These techniques can be especially useful when the interactions among those measures are highly complex, difficult to untangle, and maybe changing over time.     

收敛交叉映射方法及其在生态学中的应用

收敛交叉映射(CCM)是一种分析非线性系统中时间序列变量间因果关系的方法。其不同于传统的线性系统分析方法,是通过对变量进行状态空间重构来获取变量的历史信息,随着时间序列不断增长,当其估计性能呈现收敛的性质时,可以判断因果关系的存在。本文介绍了CCM的发展史及其较传统的格兰杰因果检验的优点,详细阐明了CCM的原理、算法过程和实现途径。CCM作为一种针对变量间具有弱到中等强度耦合关系的系统分析方法,可以用来有效地解决非线性生态系统多变量间复杂的因果关系问题。将该方法应用于具有空间信息的多点位时间序列变量间因果分析时,应充分考虑点位间的空间自相关性,与可以去除变量及序列间空间相关性的方法相结合,从而确保CCM对变量因果关系的分析更加准确,结果也更具有信服力。     

Akaike causality in state space

We present a new approach of explaining instantaneous causality in multivariate fMRI time series by a state space model. A given single time series can be divided into two noise-driven processes, a common process shared among multivariate time series and a specific process refining the common process. By assuming that noises are independent, a causality map is drawn using Akaike noise contribution ratio theory. The method is illustrated by an application to fMRI data recorded under visual stimulation.     

Detecting effective connectivity in networks of coupled neuronal oscillators

The application of data-driven time series analysis techniques such as Granger causality, partial directed coherence and phase dynamics modeling to estimate effective connectivity in brain networks has recently gained significant prominence in the neuroscience community. While these techniques have been useful in determining causal interactions among different regions of brain networks, a thorough analysis of the comparative accuracy and robustness of these methods in identifying patterns of effective connectivity among brain networks is still lacking. In this paper, we systematically address this issue within the context of simple networks of coupled spiking neurons. Specifically, we develop a method to assess the ability of various effective connectivity measures to accurately determine the true effective connectivity of a given neuronal network. Our method is based on decision tree classifiers which are trained using several time series features that can be observed solely from experimentally recorded data. We show that the classifiers constructed in this work provide a general framework for determining whether a particular effective connectivity measure is likely to produce incorrect results when applied to a dataset.     

BOLD Granger Causality Reflects Vascular Anatomy

A number of studies have tried to exploit subtle phase differences in BOLD time series to resolve the order of sequential activation of brain regions, or more generally the ability of signal in one region to predict subsequent signal in another region. More recently, such lag-based measures have been applied to investigate directed functional connectivity, although this application has been controversial. We attempted to use large publicly available datasets (FCON 1000, ADHD 200, Human Connectome Project) to determine whether consistent spatial patterns of Granger Causality are observed in typical fMRI data. For BOLD datasets from 1,240 typically developing subjects ages 7–40, we measured Granger causality between time series for every pair of 7,266 spherical ROIs covering the gray matter and 264 seed ROIs at hubs of the brain’s functional network architecture. Granger causality estimates were strongly reproducible for connections in a test and replication sample (n=620 subjects for each group), as well as in data from a single subject scanned repeatedly, both during resting and passive video viewing. The same effect was even stronger in high temporal resolution fMRI data from the Human Connectome Project, and was observed independently in data collected during performance of 7 task paradigms. The spatial distribution of Granger causality reflected vascular anatomy with a progression from Granger causality sources, in Circle of Willis arterial inflow distributions, to sinks, near large venous vascular structures such as dural venous sinuses and at the periphery of the brain. Attempts to resolve BOLD phase differences with Granger causality should consider the possibility of reproducible vascular confounds, a problem that is independent of the known regional variability of the hemodynamic response.     

Aligning gene expression time series with time warping algorithms

motivation: Increasingly, biological processes are being studied through time series of RNA expression data collected for large numbers of genes. Because common processes may unfold at varying rates in different experiments or individuals, methods are needed that will allow corresponding expression states in different time series to be mapped to one another. Results: We present implementations of time warping algorithms applicable to RNA and protein expression data and demonstrate their application to published yeast RNA expression time series. Programs executing two warping algorithms are described, a simple warping algorithm and an interpolative algorithm, along with programs that generate graphics that visually present alignment information. We show time warping to be superior to simple clustering at mapping corresponding time states. We document the impact of statistical measurement noise and sample size on the quality of time alignments, and present issues related to statistical assessment of alignment quality through alignment scores. We also discuss directions for algorithm improvement including development of multiple time series alignments and possible applications to causality searches and non-temporal processes ('concentration warping').     

Modelling regulatory pathways in E. coli from time series expression profiles

MOTIVATION: Cells continuously reprogram their gene expression network as they move through the cell cycle or sense changes in their environment. In order to understand the regulation of cells, time series expression profiles provide a more complete picture than single time point expression profiles. Few analysis techniques, however, are well suited to modelling such time series data. RESULTS: We describe an approach that naturally handles time series data with the capabilities of modelling causality, feedback loops, and environmental or hidden variables using a Dynamic Bayesian network. We also present a novel way of combining prior biological knowledge and current observations to improve the quality of analysis and to model interactions between sets of genes rather than individual genes. Our approach is evaluated on time series expression data measured in response to physiological changes that affect tryptophan metabolism in E. coli. Results indicate that this approach is capable of finding correlations between sets of related genes.     

Inferring qualitative relations in genetic networks and metabolic pathways

MOTIVATION: Inferring genetic network architecture from time series data of gene expression patterns is an important topic in bioinformatics. Although inference algorithms based on the Boolean network were proposed, the Boolean network was not sufficient as a model of a genetic network. RESULTS: First, a Boolean network model with noise is proposed, together with an inference algorithm for it. Next, a qualitative network model is proposed, in which regulation rules are represented as qualitative rules and embedded in the network structure. Algorithms are also presented for inferring qualitative relations from time series data. Then, an algorithm for inferring S-systems (synergistic and saturable systems) from time series data is presented, where S-systems are based on a particular kind of nonlinear differential equation and have been applied to the analysis of various biological systems. Theoretical results are shown for Boolean networks with noises and simple qualitative networks. Computational results are shown for Boolean networks with noises and S-systems, where real data are not used because the proposed models are still conceptual and the quantity and quality of currently available data are not enough for the application of the proposed methods.     

Extended causal modeling to assess Partial Directed Coherence in multiple time series with significant instantaneous interactions

The Partial Directed Coherence (PDC) and its generalized formulation (gPDC) are popular tools for investigating, in the frequency domain, the concept of Granger causality among multivariate (MV) time series. PDC and gPDC are formalized in terms of the coefficients of an MV autoregressive (MVAR) model which describes only the lagged effects among the time series and forsakes instantaneous effects. However, instantaneous effects are known to affect linear parametric modeling, and are likely to occur in experimental time series. In this study, we investigate the impact on the assessment of frequency domain causality of excluding instantaneous effects from the model underlying PDC evaluation. Moreover, we propose the utilization of an extended MVAR model including both instantaneous and lagged effects. This model is used to assess PDC either in accordance with the definition of Granger causality when considering only lagged effects (iPDC), or with an extended form of causality, when we consider both instantaneous and lagged effects (ePDC). The approach is first evaluated on three theoretical examples of MVAR processes, which show that the presence of instantaneous correlations may produce misleading profiles of PDC and gPDC, while ePDC and iPDC derived from the extended model provide here a correct interpretation of extended and lagged causality. It is then applied to representative examples of cardiorespiratory and EEG MV time series. They suggest that ePDC and iPDC are better interpretable than PDC and gPDC in terms of the known cardiovascular and neural physiologies.     

Inference of genetic regulatory networks with recurrent neural network models using particle swarm optimization

Genetic regulatory network inference is critically important for revealing fundamental cellular processes, investigating gene functions, and understanding their relations. The availability of time series gene expression data makes it possible to investigate the gene activities of whole genomes, rather than those of only a pair of genes or among several genes. However, current computational methods do not sufficiently consider the temporal behavior of this type of data and lack the capability to capture the complex nonlinear system dynamics. We propose a recurrent neural network (RNN) and particle swarm optimization (PSO) approach to infer genetic regulatory networks from time series gene expression data. Under this framework, gene interaction is explained through a connection weight matrix. Based on the fact that the measured time points are limited and the assumption that the genetic networks are usually sparsely connected, we present a PSO-based search algorithm to unveil potential genetic network constructions that fit well with the time series data and explore possible gene interactions. Furthermore, PSO is used to train the RNN and determine the network parameters. Our approach has been applied to both synthetic and real data sets. The results demonstrate that the RNN/PSO can provide meaningful insights in understanding the nonlinear dynamics of the gene expression time series and revealing potential regulatory interactions between genes.     

Analyzing time series gene expression data

MOTIVATION: Time series expression experiments are an increasingly popular method for studying a wide range of biological systems. However, when analyzing these experiments researchers face many new computational challenges. Algorithms that are specifically designed for time series experiments are required so that we can take advantage of their unique features (such as the ability to infer causality from the temporal response pattern) and address the unique problems they raise (e.g. handling the different non-uniform sampling rates). RESULTS: We present a comprehensive review of the current research in time series expression data analysis. We divide the computational challenges into four analysis levels: experimental design, data analysis, pattern recognition and networks. For each of these levels, we discuss computational and biological problems at that level and point out some of the methods that have been proposed to deal with these issues. Many open problems in all these levels are discussed. This review is intended to serve as both, a point of reference for experimental biologists looking for practical solutions for analyzing their data, and a starting point for computer scientists interested in working on the computational problems related to time series expression analysis.     

Parametric construction of episode networks from pseudoperiodic time series based on mutual information

Recently, the construction of networks from time series data has gained widespread interest. In this paper, we develop this area further by introducing a network construction procedure for pseudoperiodic time series. We call such networks episode networks, in which an episode corresponds to a temporal interval of a time series, and which defines a node in the network. Our model includes a number of features which distinguish it from current methods. First, the proposed construction procedure is a parametric model which allows it to adapt to the characteristics of the data; the length of an episode being the parameter. As a direct consequence, networks of minimal size containing the maximal information about the time series can be obtained. In this paper, we provide an algorithm to determine the optimal value of this parameter. Second, we employ estimates of mutual information values to define the connectivity structure among the nodes in the network to exploit efficiently the nonlinearities in the time series. Finally, we apply our method to data from electroencephalogram (EEG) experiments and demonstrate that the constructed episode networks capture discriminative information from the underlying time series that may be useful for diagnostic purposes.     

Detecting Causality by Combined Use of Multiple Methods: Climate and Brain Examples

Identifying causal relations from time series is the first step to understanding the behavior of complex systems. Although many methods have been proposed, few papers have applied multiple methods together to detect causal relations based on time series generated from coupled nonlinear systems with some unobserved parts. Here we propose the combined use of three methods and a majority vote to infer causality under such circumstances. Two of these methods are proposed here for the first time, and all of the three methods can be applied even if the underlying dynamics is nonlinear and there are hidden common causes. We test our methods with coupled logistic maps, coupled Rössler models, and coupled Lorenz models. In addition, we show from ice core data how the causal relations among the temperature, the CH₄ level, and the CO₂ level in the atmosphere changed in the last 800,000 years, a conclusion also supported by irregularly sampled data analysis. Moreover, these methods show how three regions of the brain interact with each other during the visually cued, two-choice arm reaching task. Especially, we demonstrate that this is due to bottom up influences at the beginning of the task, while there exist mutual influences between the posterior medial prefrontal cortex and the presupplementary motor area. Based on our results, we conclude that identifying causality with an appropriate ensemble of multiple methods ensures the validity of the obtained results more firmly.     

Time Delayed Causal Gene Regulatory Network Inference with Hidden Common Causes

Inferring the gene regulatory network (GRN) is crucial to understanding the working of the cell. Many computational methods attempt to infer the GRN from time series expression data, instead of through expensive and time-consuming experiments. However, existing methods make the convenient but unrealistic assumption of causal sufficiency, i.e. all the relevant factors in the causal network have been observed and there are no unobserved common cause. In principle, in the real world, it is impossible to be certain that all relevant factors or common causes have been observed, because some factors may not have been conceived of, and therefore are impossible to measure. In view of this, we have developed a novel algorithm named HCC-CLINDE to infer an GRN from time series data allowing the presence of hidden common cause(s). We assume there is a sparse causal graph (possibly with cycles) of interest, where the variables are continuous and each causal link has a delay (possibly more than one time step). A small but unknown number of variables are not observed. Each unobserved variable has only observed variables as children and parents, with at least two children, and the children are not linked to each other. Since it is difficult to obtain very long time series, our algorithm is also capable of utilizing multiple short time series, which is more realistic. To our knowledge, our algorithm is far less restrictive than previous works. We have performed extensive experiments using synthetic data on GRNs of size up to 100, with up to 10 hidden nodes. The results show that our algorithm can adequately recover the true causal GRN and is robust to slight deviation from Gaussian distribution in the error terms. We have also demonstrated the potential of our algorithm on small YEASTRACT subnetworks using limited real data.     

Gene network inference using continuous time Bayesian networks: a comparative study and application to Th17 cell differentiation

Background

Dynamic aspects of gene regulatory networks are typically investigated by measuring system variables at multiple time points. Current state-of-the-art computational approaches for reconstructing gene networks directly build on such data, making a strong assumption that the system evolves in a synchronous fashion at fixed points in time. However, nowadays omics data are being generated with increasing time course granularity. Thus, modellers now have the possibility to represent the system as evolving in continuous time and to improve the models’ expressiveness.

Results

Continuous time Bayesian networks are proposed as a new approach for gene network reconstruction from time course expression data. Their performance was compared to two state-of-the-art methods: dynamic Bayesian networks and Granger causality analysis. On simulated data, the methods comparison was carried out for networks of increasing size, for measurements taken at different time granularity densities and for measurements unevenly spaced over time. Continuous time Bayesian networks outperformed the other methods in terms of the accuracy of regulatory interactions learnt from data for all network sizes. Furthermore, their performance degraded smoothly as the size of the network increased. Continuous time Bayesian networks were significantly better than dynamic Bayesian networks for all time granularities tested and better than Granger causality for dense time series. Both continuous time Bayesian networks and Granger causality performed robustly for unevenly spaced time series, with no significant loss of performance compared to the evenly spaced case, while the same did not hold true for dynamic Bayesian networks. The comparison included the IRMA experimental datasets which confirmed the effectiveness of the proposed method. Continuous time Bayesian networks were then applied to elucidate the regulatory mechanisms controlling murine T helper 17 (Th17) cell differentiation and were found to be effective in discovering well-known regulatory mechanisms, as well as new plausible biological insights.

Conclusions

Continuous time Bayesian networks were effective on networks of both small and large size and were particularly feasible when the measurements were not evenly distributed over time. Reconstruction of the murine Th17 cell differentiation network using continuous time Bayesian networks revealed several autocrine loops, suggesting that Th17 cells may be auto regulating their own differentiation process.