Independent EEG sources are dipolar

Independent component analysis (ICA) and blind source separation (BSS) methods are increasingly used to separate individual brain and non-brain source signals mixed by volume conduction in electroencephalographic (EEG) and other electrophysiological recordings. We compared results of decomposing thirteen 71-channel human scalp EEG datasets by 22 ICA and BSS algorithms, assessing the pairwise mutual information (PMI) in scalp channel pairs, the remaining PMI in component pairs, the overall mutual information reduction (MIR) effected by each decomposition, and decomposition 'dipolarity' defined as the number of component scalp maps matching the projection of a single equivalent dipole with less than a given residual variance. The least well-performing algorithm was principal component analysis (PCA); best performing were AMICA and other likelihood/mutual information based ICA methods. Though these and other commonly-used decomposition methods returned many similar components, across 18 ICA/BSS algorithms mean dipolarity varied linearly with both MIR and with PMI remaining between the resulting component time courses, a result compatible with an interpretation of many maximally independent EEG components as being volume-conducted projections of partially-synchronous local cortical field activity within single compact cortical domains. To encourage further method comparisons, the data and software used to prepare the results have been made available (http://sccn.ucsd.edu/wiki/BSSComparison).     

Estimating causal interaction between prefrontal cortex and striatum by transfer entropy

Transfer entropy (TE) is an information-theoretic measure for the investigation of causal interaction between two systems without a requirement of pre-specific interaction model (such as: linear or nonlinear). We introduced an efficient algorithm to calculate TE values between two systems based on observed time signals. By this method, we demonstrated that the TE correctly estimated the coupling strength and the direction of information transmission of two nonlinearly coupled systems. We also calculated TE values of real local field potentials (LFPs) recorded simultaneously in the lateral prefrontal cortex (LPFC) and the striatum of the behavioral monkey, and observed that the TE value from the LPFC to the striatum was stronger than that from the striatum to the LPFC, consistent with anatomical structure between the two areas. Moreover, the TE value dynamically varied dependent on behavior stages of the monkey. These results from simulated and real LFPs data suggested that the TE was able to effectively estimate functional connectivity between different brain regions and characterized their dynamical properties.     

Assessing spatial coupling in complex population dynamics using mutual prediction and continuity statistics

A number of important questions in ecology involve the possibility of interactions or "coupling" among potential components of ecological systems. The basic question of whether two components are coupled (exhibit dynamical interdependence) is relevant to investigations of movement of animals over space, population regulation, food webs and trophic interactions, and is also useful in the design of monitoring programs. For example, in spatially extended systems, coupling among populations in different locations implies the existence of redundant information in the system and the possibility of exploiting this redundancy in the development of spatial sampling designs. One approach to the identification of coupling involves study of the purported mechanisms linking system components. Another approach is based on time series of two potential components of the same system and, in previous ecological work, has relied on linear cross-correlation analysis. Here we present two different attractor-based approaches, continuity and mutual prediction, for determining the degree to which two population time series (e.g., at different spatial locations) are coupled. Both approaches are demonstrated on a one-dimensional predator-prey model system exhibiting complex dynamics. Of particular interest is the spatial asymmetry introduced into the model as linearly declining resource for the prey over the domain of the spatial coordinate. Results from these approaches are then compared to the more standard cross-correlation analysis. In contrast to cross-correlation, both continuity and mutual prediction are clearly able to discern the asymmetry in the flow of information through this system.     

A method for the estimation of functional brain connectivity from time-series data

A central issue in cognitive neuroscience is which cortical areas are involved in managing information processing in a cognitive task and to understand their temporal interactions. Since the transfer of information in the form of electrical activity from one cortical region will in turn evoke electrical activity in other regions, the analysis of temporal synchronization provides a tool to understand neuronal information processing between cortical regions. We adopt a method for revealing time-dependent functional connectivity. We apply statistical analyses of phases to recover the information flow and the functional connectivity between cortical regions for high temporal resolution data. We further develop an evaluation method for these techniques based on two kinds of model networks. These networks consist of coupled Rössler attractors or of coupled stochastic Ornstein–Uhlenbeck systems. The implemented time-dependent coupling includes uni- and bi-directional connectivities as well as time delayed feedback. The synchronization dynamics of these networks are analyzed using the mean phase coherence, based on averaging over phase-differences, and the general synchronization index. The latter is based on the Shannon entropy. The combination of these with a parametric time delay forms the basis of a connectivity pattern, which includes the temporal and time lagged dynamics of the synchronization between two sources. We model and discuss potential artifacts. We find that the general phase measures are remarkably stable. They produce highly comparable results for stochastic and periodic systems. Moreover, the methods proves useful for identifying brief periods of phase coupling and delays. Therefore, we propose that the method is useful as a basis for generating potential functional connective models.     

Hopf bifurcation and bursting synchronization in an excitable systems with chemical delayed coupling

In this paper we consider the Hopf bifurcation and synchronization in the two coupled Hindmarsh–Rose excitable systems with chemical coupling and time-delay. We surveyed the conditions for Hopf bifurcations by means of dynamical bifurcation analysis and numerical simulation. The results show that the coupled excitable systems with no delay have supercritical Hopf bifurcation, while the delayed system undergoes Hopf bifurcations at critical time delays when coupling strength lies in a particular region. We also investigated the effect of the delay on the transition of bursting synchronization in the coupled system. The results are helpful for us to better understand the dynamical properties of excitable systems and the biological mechanism of information encoding and cognitive activity.     

基于大脑皮层互信息理论的睡眠分级研究

睡眠的分级研究是睡眠状况分析和睡眠质量评价的前提和基本内容。目前国际通用的睡眠分级方法,是利用脑电信号另加脑功能信号（如肌电图、眼动电流图）,且必须由人工来判别分析的。大脑皮层互信息理论是研究脑功能变化的有力工具。通过动态计算睡眠脑电四个导联之间的互信息时间序列的复杂度,并利用一个三层的人工神经网络进行六个级别的分类,6例720个不同时期的睡眠片段的测试表明,系统睡眠分级与人工分级的总相符率达到90.83％,且实现了睡眠动态自动分级。神经网络的学习功能,可使系统的准确率进一步提高,逐渐接近或达到人工分级的水平。     

A method for the generation of standardized qualitative dynamical systems of regulatory networks

Background  

Modeling of molecular networks is necessary to understand their dynamical properties. While a wealth of information on molecular connectivity is available, there are still relatively few data regarding the precise stoichiometry and kinetics of the biochemical reactions underlying most molecular networks. This imbalance has limited the development of dynamical models of biological networks to a small number of well-characterized systems. To overcome this problem, we wanted to develop a methodology that would systematically create dynamical models of regulatory networks where the flow of information is known but the biochemical reactions are not. There are already diverse methodologies for modeling regulatory networks, but we aimed to create a method that could be completely standardized, i.e. independent of the network under study, so as to use it systematically.     

Coherency and connectivity in oscillating neural networks: linear partialization analysis

 This paper studies the relation between the functional synaptic connections between two artificial neural networks and the correlation of their spiking activities. The model neurons had realistic non-oscillatory dynamic properties and the networks showed oscillatory behavior as a result of their internal synaptic connectivity. We found that both excitation and inhibition cause phase locking of the oscillating activities. When the two networks excite each other the oscillations synchronize with zero phase lag, whereas mutual inhibition between the networks resulted in an anti-phase (half period phase difference) synchronization. Correlations between the activities of the two networks can also be caused by correlated external inputs driving the systems (common input). Our analysis shows that when the networks exhibit oscillatory behavior and the rate of the common input is smaller than a characteristic network oscillator frequency, the cross-correlation functions between the activities of two systems still carry information about the mutual synaptic connectivity. This information can be retrieved with linear partialization, removing the influence of the common input. We further explored the network responses to periodic external input. We found that when the input is of a frequency smaller than a certain threshold, the network responds with bursts at the same frequency as the input. Above the threshold, the network responds with a fraction of the input frequency. This frequency threshold, characterizing the oscillatory properties of the network, is also found to determine the limit to which linear partialization works. Received: 20 October 1995 / Accepted in revised form: 20 May 1996     

Morphogenesis by coupled regulatory networks: Reliable control of positional information and proportion regulation

Based on a non-equilibrium mechanism for spatial pattern formation we study how position information can be controlled by locally coupled discrete dynamical networks, similar to gene regulation networks of cells in a developing multicellular organism. As an example we study the developmental problems of domain formation and proportion regulation in the presence of noise, as well as in the presence of cell flow. We find that networks that solve this task exhibit a hierarchical structure of information processing and are of similar complexity as developmental circuits of living cells. Proportion regulation is scalable with system size and leads to sharp, precisely localized boundaries of gene expression domains, even for large numbers of cells. A detailed analysis of noise-induced dynamics, using a mean-field approximation, shows that noise in gene expression states stabilizes (rather than disrupts) the spatial pattern in the presence of cell movements, both for stationary as well as growing systems. Finally, we discuss how this mechanism could be realized in the highly dynamic environment of growing tissues in multicellular organisms.     

Propagation of kinetic uncertainties through a canonical topology of the TLR4 signaling network in different regions of biochemical reaction space

Background  

Signal transduction networks represent the information processing systems that dictate which dynamical regimes of biochemical activity can be accessible to a cell under certain circumstances. One of the major concerns in molecular systems biology is centered on the elucidation of the robustness properties and information processing capabilities of signal transduction networks. Achieving this goal requires the establishment of causal relations between the design principle of biochemical reaction systems and their emergent dynamical behaviors.     

Robustness of Oscillatory Behavior in Correlated Networks

Understanding network robustness against failures of network units is useful for preventing large-scale breakdowns and damages in real-world networked systems. The tolerance of networked systems whose functions are maintained by collective dynamical behavior of the network units has recently been analyzed in the framework called dynamical robustness of complex networks. The effect of network structure on the dynamical robustness has been examined with various types of network topology, but the role of network assortativity, or degree–degree correlations, is still unclear. Here we study the dynamical robustness of correlated (assortative and disassortative) networks consisting of diffusively coupled oscillators. Numerical analyses for the correlated networks with Poisson and power-law degree distributions show that network assortativity enhances the dynamical robustness of the oscillator networks but the impact of network disassortativity depends on the detailed network connectivity. Furthermore, we theoretically analyze the dynamical robustness of correlated bimodal networks with two-peak degree distributions and show the positive impact of the network assortativity.     

Information flow in heterogeneously interacting systems

Motivated by studies on the dynamics of heterogeneously interacting systems in neocortical neural networks, we studied heterogeneously-coupled chaotic systems. We used information-theoretic measures to investigate directions of information flow in heterogeneously coupled Rössler systems, which we selected as a typical chaotic system. In bi-directionally coupled systems, spontaneous and irregular switchings of the phase difference between two chaotic oscillators were observed. The direction of information transmission spontaneously switched in an intermittent manner, depending on the phase difference between the two systems. When two further oscillatory inputs are added to the coupled systems, this system dynamically selects one of the two inputs by synchronizing, selection depending on the internal phase differences between the two systems. These results indicate that the effective direction of information transmission dynamically changes, induced by a switching of phase differences between the two systems.     

Time-dependent propagators for stochastic models of gene expression: an analytical method

The inherent stochasticity of gene expression in the context of regulatory networks profoundly influences the dynamics of the involved species. Mathematically speaking, the propagators which describe the evolution of such networks in time are typically defined as solutions of the corresponding chemical master equation (CME). However, it is not possible in general to obtain exact solutions to the CME in closed form, which is due largely to its high dimensionality. In the present article, we propose an analytical method for the efficient approximation of these propagators. We illustrate our method on the basis of two categories of stochastic models for gene expression that have been discussed in the literature. The requisite procedure consists of three steps: a probability-generating function is introduced which transforms the CME into (a system of) partial differential equations (PDEs); application of the method of characteristics then yields (a system of) ordinary differential equations (ODEs) which can be solved using dynamical systems techniques, giving closed-form expressions for the generating function; finally, propagator probabilities can be reconstructed numerically from these expressions via the Cauchy integral formula. The resulting ‘library’ of propagators lends itself naturally to implementation in a Bayesian parameter inference scheme, and can be generalised systematically to related categories of stochastic models beyond the ones considered here.

     

Modeling of dynamical systems through deep learning

This review presents a modern perspective on dynamical systems in the context of current goals and open challenges. In particular, our review focuses on the key challenges of discovering dynamics from data and finding data-driven representations that make nonlinear systems amenable to linear analysis. We explore various challenges in modern dynamical systems, along with emerging techniques in data science and machine learning to tackle them. The two chief challenges are (1) nonlinear dynamics and (2) unknown or partially known dynamics. Machine learning is providing new and powerful techniques for both challenges. Dimensionality reduction methods are used for projecting dynamical methods in reduced form, and these methods perform computational efficiency on real-world data. Data-driven models drive to discover the governing equations and give laws of physics. The identification of dynamical systems through deep learning techniques succeeds in inferring physical systems. Machine learning provides advanced new and powerful algorithms for nonlinear dynamics. Advanced deep learning methods like autoencoders, recurrent neural networks, convolutional neural networks, and reinforcement learning are used in modeling of dynamical systems.     

The principle of interval constraints: a generalization of the symmetric Dirichlet distribution.

A structure for representing problems in decision analysis and in expert systems, which reason under uncertainty, is the influence diagram or causal network. A causal network consists of an underlying joint probability distribution and a directed acyclic graph in which a propositional variable that represents a marginal distribution is stored at each vertex in the graph. This paper is concerned with two of the problems in applications that use causal networks. The first problem is the determination of the conditional probabilities of the values of remaining propositional variables in the network given that certain variables are instantiated for particular values. This is called probability propagation. The second problem is the determination of the most probable, second most probable, third most probable, and so on sets of values of a particular set of variables (called the explanation set) given that certain variables are instantiated for particular values. This problem is called abductive inference. There exists a class of causal networks in which each variable has only two parents, for which the time is required, by any known method, for probability propagation is exponential relative to the number of vertices in the network. The determination of a new method that would be efficient for all causal networks appears unlikely, because probability propagation has been shown to be #P-complete. In many medical applications, networks are often large and not sparsely connected. Therefore a method for the exact determination of probability values appears unlikely for such applications, and the development of approximation methods seems to be the best solution. The current approximation methods obtain interval bounds for the probability values. When such intervals are obtained, it is not possible in general to rank the alternatives. In this paper, a method is developed for obtaining expected values for the point probabilities from interval constraints on the probabilities. The method is based on an application of the principle of indifference to the probability values themselves. The distributions obtained with the principle of indifference are a generalization of the symmetric Dirichlet distribution in which prior ignorance is assumed.     

Boolean dynamics of biological networks with multiple coupled feedback loops

Boolean networks have been frequently used to study the dynamics of biological networks. In particular, there have been various studies showing that the network connectivity and the update rule of logical functions affect the dynamics of Boolean networks. There has been, however, relatively little attention paid to the dynamical role of a feedback loop, which is a circular chain of interactions between Boolean variables. We note that such feedback loops are ubiquitously found in various biological systems as multiple coupled structures and they are often the primary cause of complex dynamics. In this article, we investigate the relationship between the multiple coupled feedback loops and the dynamics of Boolean networks. We show that networks have a larger proportion of basins corresponding to fixed-point attractors as they have more coupled positive feedback loops, and a larger proportion of basins for limit-cycle attractors as they have more coupled negative feedback loops.     

Node vulnerability under finite perturbations in complex networks

A measure to quantify vulnerability under perturbations (attacks, failures, large fluctuations) in ensembles (networks) of coupled dynamical systems is proposed. Rather than addressing the issue of how the network properties change upon removal of elements of the graph (the strategy followed by most of the existing methods for studying the vulnerability of a network based on its topology), here a dynamical definition of vulnerability is introduced, referring to the robustness of a collective dynamical state to perturbing events occurring over a fixed topology. In particular, we study how the collective (synchronized) dynamics of a network of chaotic units is disrupted under the action of a finite size perturbation on one of its nodes. Illustrative examples are provided for three systems of identical chaotic oscillators coupled according to three distinct well-known network topologies. A quantitative comparison between the obtained vulnerability rankings and the classical connectivity/centrality rankings is made that yields conclusive results. Possible applications of the proposed strategy and conclusions are also discussed.