Global relationship between anatomical connectivity and activity propagation in the cerebral cortex

Anatomical connectivity is a prerequisite for cooperative interactions between cortical areas, but it has yet to be demonstrated that association fibre networks determine the macroscopical flow of activity in the cerebral cortex. To test this notion, we constructed a large-scale model of cortical areas whose interconnections were based on published anatomical data from tracing studies. Using this model we simulated the propagation of activity in response to activation of individual cortical areas and compared the resulting topographic activation patterns to electrophysiological observations on the global spread of epileptic activity following intracortical stimulation. Here we show that a neural network with connectivity derived from experimental data reproduces cortical propagation of activity significantly better than networks with different types of neighbourhood-based connectivity or random connections. Our results indicate that association fibres and their relative connection strengths are useful predictors of global topographic activation patterns in the cerebral cortex. This global structure-function relationship may open a door to explicit interpretation of cortical activation data in terms of underlying anatomical connectivity.     

The impact of sparsity in low-rank recurrent neural networks

Neural population dynamics are often highly coordinated, allowing task-related computations to be understood as neural trajectories through low-dimensional subspaces. How the network connectivity and input structure give rise to such activity can be investigated with the aid of low-rank recurrent neural networks, a recently-developed class of computational models which offer a rich theoretical framework linking the underlying connectivity structure to emergent low-dimensional dynamics. This framework has so far relied on the assumption of all-to-all connectivity, yet cortical networks are known to be highly sparse. Here we investigate the dynamics of low-rank recurrent networks in which the connections are randomly sparsified, which makes the network connectivity formally full-rank. We first analyse the impact of sparsity on the eigenvalue spectrum of low-rank connectivity matrices, and use this to examine the implications for the dynamics. We find that in the presence of sparsity, the eigenspectra in the complex plane consist of a continuous bulk and isolated outliers, a form analogous to the eigenspectra of connectivity matrices composed of a low-rank and a full-rank random component. This analogy allows us to characterise distinct dynamical regimes of the sparsified low-rank network as a function of key network parameters. Altogether, we find that the low-dimensional dynamics induced by low-rank connectivity structure are preserved even at high levels of sparsity, and can therefore support rich and robust computations even in networks sparsified to a biologically-realistic extent.     

A modular latching chain

Many cognitive tasks involve transitions between distinct mental processes, which may range from discrete states to complex strategies. The ability of cortical networks to combine discrete jumps with continuous glides along ever changing trajectories, dubbed latching dynamics, may be essential for the emergence of the unique cognitive capacities of modern humans. Novel trajectories have to be followed in the multidimensional space of cortical activity for novel behaviours to be produced; yet, not everything changes: several lines of evidence point at recurring patterns in the sequence of activation of cortical areas in a variety of behaviours. To extend a mathematical model of latching dynamics beyond the simple unstructured auto-associative Potts network previously analysed, we introduce delayed structured connectivity and hetero-associative connection weights, and we explore their effects on the dynamics. A modular model in the small-world regime is considered, with modules arranged on a ring. The synaptic weights include a standard auto-associative component, stabilizing distinct patterns of activity, and a hetero-associative component, favoring transitions from one pattern, expressed in one module, to the next, in the next module. We then study, through simulations, how structural parameters, like those regulating rewiring probability, noise and feedback connections, determine sequential association dynamics.     

The Effect of Slow Electrical Stimuli to Achieve Learning in Cultured Networks of Rat Cortical Neurons

Learning, or more generally, plasticity may be studied using cultured networks of rat cortical neurons on multi electrode arrays. Several protocols have been proposed to affect connectivity in such networks. One of these protocols, proposed by Shahaf and Marom, aimed to train the input-output relationship of a selected connection in a network using slow electrical stimuli. Although the results were quite promising, the experiments appeared difficult to repeat and the training protocol did not serve as a basis for wider investigation yet. Here, we repeated their protocol, and compared our ‘learning curves’ to the original results. Although in some experiments the protocol did not seem to work, we found that on average, the protocol showed a significantly improved stimulus response indeed. Furthermore, the protocol always induced functional connectivity changes that were much larger than changes that occurred after a comparable period of random or no stimulation. Finally, our data shows that stimulation at a fixed electrode induces functional connectivity changes of similar magnitude as stimulation through randomly varied sites; both larger than spontaneous connectivity fluctuations. We concluded that slow electrical stimulation always induced functional connectivity changes, although uncontrolled. The magnitude of change increased when we applied the adaptive (closed-loop) training protocol. We hypothesize that networks develop an equilibrium between connectivity and activity. Induced connectivity changes depend on the combination of applied stimulus and initial connectivity. Plain stimuli may drive networks to the nearest equilibrium that accommodates this input, whereas adaptive stimulation may direct the space for exploration and force networks to a new balance, at a larger distance from the initial state.     

Dense Neuron Clustering Explains Connectivity Statistics in Cortical Microcircuits

Local cortical circuits appear highly non-random, but the underlying connectivity rule remains elusive. Here, we analyze experimental data observed in layer 5 of rat neocortex and suggest a model for connectivity from which emerge essential observed non-random features of both wiring and weighting. These features include lognormal distributions of synaptic connection strength, anatomical clustering, and strong correlations between clustering and connection strength. Our model predicts that cortical microcircuits contain large groups of densely connected neurons which we call clusters. We show that such a cluster contains about one fifth of all excitatory neurons of a circuit which are very densely connected with stronger than average synapses. We demonstrate that such clustering plays an important role in the network dynamics, namely, it creates bistable neural spiking in small cortical circuits. Furthermore, introducing local clustering in large-scale networks leads to the emergence of various patterns of persistent local activity in an ongoing network activity. Thus, our results may bridge a gap between anatomical structure and persistent activity observed during working memory and other cognitive processes.     

Nonlinear Dynamics Analysis of a Self-Organizing Recurrent Neural Network: Chaos Waning

Self-organization is thought to play an important role in structuring nervous systems. It frequently arises as a consequence of plasticity mechanisms in neural networks: connectivity determines network dynamics which in turn feed back on network structure through various forms of plasticity. Recently, self-organizing recurrent neural network models (SORNs) have been shown to learn non-trivial structure in their inputs and to reproduce the experimentally observed statistics and fluctuations of synaptic connection strengths in cortex and hippocampus. However, the dynamics in these networks and how they change with network evolution are still poorly understood. Here we investigate the degree of chaos in SORNs by studying how the networks'' self-organization changes their response to small perturbations. We study the effect of perturbations to the excitatory-to-excitatory weight matrix on connection strengths and on unit activities. We find that the network dynamics, characterized by an estimate of the maximum Lyapunov exponent, becomes less chaotic during its self-organization, developing into a regime where only few perturbations become amplified. We also find that due to the mixing of discrete and (quasi-)continuous variables in SORNs, small perturbations to the synaptic weights may become amplified only after a substantial delay, a phenomenon we propose to call deferred chaos.     

Regimes and mechanisms of transient amplification in abstract and biological neural networks

Neuronal networks encode information through patterns of activity that define the networks’ function. The neurons’ activity relies on specific connectivity structures, yet the link between structure and function is not fully understood. Here, we tackle this structure-function problem with a new conceptual approach. Instead of manipulating the connectivity directly, we focus on upper triangular matrices, which represent the network dynamics in a given orthonormal basis obtained by the Schur decomposition. This abstraction allows us to independently manipulate the eigenspectrum and feedforward structures of a connectivity matrix. Using this method, we describe a diverse repertoire of non-normal transient amplification, and to complement the analysis of the dynamical regimes, we quantify the geometry of output trajectories through the effective rank of both the eigenvector and the dynamics matrices. Counter-intuitively, we find that shrinking the eigenspectrum’s imaginary distribution leads to highly amplifying regimes in linear and long-lasting dynamics in nonlinear networks. We also find a trade-off between amplification and dimensionality of neuronal dynamics, i.e., trajectories in neuronal state-space. Networks that can amplify a large number of orthogonal initial conditions produce neuronal trajectories that lie in the same subspace of the neuronal state-space. Finally, we examine networks of excitatory and inhibitory neurons. We find that the strength of global inhibition is directly linked with the amplitude of amplification, such that weakening inhibitory weights also decreases amplification, and that the eigenspectrum’s imaginary distribution grows with an increase in the ratio between excitatory-to-inhibitory and excitatory-to-excitatory connectivity strengths. Consequently, the strength of global inhibition reveals itself as a strong signature for amplification and a potential control mechanism to switch dynamical regimes. Our results shed a light on how biological networks, i.e., networks constrained by Dale’s law, may be optimised for specific dynamical regimes.     

Highly nonrandom features of synaptic connectivity in local cortical circuits

How different is local cortical circuitry from a random network? To answer this question, we probed synaptic connections with several hundred simultaneous quadruple whole-cell recordings from layer 5 pyramidal neurons in the rat visual cortex. Analysis of this dataset revealed several nonrandom features in synaptic connectivity. We confirmed previous reports that bidirectional connections are more common than expected in a random network. We found that several highly clustered three-neuron connectivity patterns are overrepresented, suggesting that connections tend to cluster together. We also analyzed synaptic connection strength as defined by the peak excitatory postsynaptic potential amplitude. We found that the distribution of synaptic connection strength differs significantly from the Poisson distribution and can be fitted by a lognormal distribution. Such a distribution has a heavier tail and implies that synaptic weight is concentrated among few synaptic connections. In addition, the strengths of synaptic connections sharing pre- or postsynaptic neurons are correlated, implying that strong connections are even more clustered than the weak ones. Therefore, the local cortical network structure can be viewed as a skeleton of stronger connections in a sea of weaker ones. Such a skeleton is likely to play an important role in network dynamics and should be investigated further.     

Network Self-Organization Explains the Statistics and Dynamics of Synaptic Connection Strengths in Cortex

The information processing abilities of neural circuits arise from their synaptic connection patterns. Understanding the laws governing these connectivity patterns is essential for understanding brain function. The overall distribution of synaptic strengths of local excitatory connections in cortex and hippocampus is long-tailed, exhibiting a small number of synaptic connections of very large efficacy. At the same time, new synaptic connections are constantly being created and individual synaptic connection strengths show substantial fluctuations across time. It remains unclear through what mechanisms these properties of neural circuits arise and how they contribute to learning and memory. In this study we show that fundamental characteristics of excitatory synaptic connections in cortex and hippocampus can be explained as a consequence of self-organization in a recurrent network combining spike-timing-dependent plasticity (STDP), structural plasticity and different forms of homeostatic plasticity. In the network, associative synaptic plasticity in the form of STDP induces a rich-get-richer dynamics among synapses, while homeostatic mechanisms induce competition. Under distinctly different initial conditions, the ensuing self-organization produces long-tailed synaptic strength distributions matching experimental findings. We show that this self-organization can take place with a purely additive STDP mechanism and that multiplicative weight dynamics emerge as a consequence of network interactions. The observed patterns of fluctuation of synaptic strengths, including elimination and generation of synaptic connections and long-term persistence of strong connections, are consistent with the dynamics of dendritic spines found in rat hippocampus. Beyond this, the model predicts an approximately power-law scaling of the lifetimes of newly established synaptic connection strengths during development. Our results suggest that the combined action of multiple forms of neuronal plasticity plays an essential role in the formation and maintenance of cortical circuits.     

Influence of Wiring Cost on the Large-Scale Architecture of Human Cortical Connectivity

In the past two decades some fundamental properties of cortical connectivity have been discovered: small-world structure, pronounced hierarchical and modular organisation, and strong core and rich-club structures. A common assumption when interpreting results of this kind is that the observed structural properties are present to enable the brain''s function. However, the brain is also embedded into the limited space of the skull and its wiring has associated developmental and metabolic costs. These basic physical and economic aspects place separate, often conflicting, constraints on the brain''s connectivity, which must be characterized in order to understand the true relationship between brain structure and function. To address this challenge, here we ask which, and to what extent, aspects of the structural organisation of the brain are conserved if we preserve specific spatial and topological properties of the brain but otherwise randomise its connectivity. We perform a comparative analysis of a connectivity map of the cortical connectome both on high- and low-resolutions utilising three different types of surrogate networks: spatially unconstrained (‘random’), connection length preserving (‘spatial’), and connection length optimised (‘reduced’) surrogates. We find that unconstrained randomisation markedly diminishes all investigated architectural properties of cortical connectivity. By contrast, spatial and reduced surrogates largely preserve most properties and, interestingly, often more so in the reduced surrogates. Specifically, our results suggest that the cortical network is less tightly integrated than its spatial constraints would allow, but more strongly segregated than its spatial constraints would necessitate. We additionally find that hierarchical organisation and rich-club structure of the cortical connectivity are largely preserved in spatial and reduced surrogates and hence may be partially attributable to cortical wiring constraints. In contrast, the high modularity and strong s-core of the high-resolution cortical network are significantly stronger than in the surrogates, underlining their potential functional relevance in the brain.     

Interplay between Graph Topology and Correlations of Third Order in Spiking Neuronal Networks

The study of processes evolving on networks has recently become a very popular research field, not only because of the rich mathematical theory that underpins it, but also because of its many possible applications, a number of them in the field of biology. Indeed, molecular signaling pathways, gene regulation, predator-prey interactions and the communication between neurons in the brain can be seen as examples of networks with complex dynamics. The properties of such dynamics depend largely on the topology of the underlying network graph. In this work, we want to answer the following question: Knowing network connectivity, what can be said about the level of third-order correlations that will characterize the network dynamics? We consider a linear point process as a model for pulse-coded, or spiking activity in a neuronal network. Using recent results from theory of such processes, we study third-order correlations between spike trains in such a system and explain which features of the network graph (i.e. which topological motifs) are responsible for their emergence. Comparing two different models of network topology—random networks of Erdős-Rényi type and networks with highly interconnected hubs—we find that, in random networks, the average measure of third-order correlations does not depend on the local connectivity properties, but rather on global parameters, such as the connection probability. This, however, ceases to be the case in networks with a geometric out-degree distribution, where topological specificities have a strong impact on average correlations.     

The frustrated brain: from dynamics on motifs to communities and networks

Cognitive function depends on an adaptive balance between flexible dynamics and integrative processes in distributed cortical networks. Patterns of zero-lag synchrony likely underpin numerous perceptual and cognitive functions. Synchronization fulfils integration by reducing entropy, while adaptive function mandates that a broad variety of stable states be readily accessible. Here, we elucidate two complementary influences on patterns of zero-lag synchrony that derive from basic properties of brain networks. First, mutually coupled pairs of neuronal subsystems—resonance pairs—promote stable zero-lag synchrony among the small motifs in which they are embedded, and whose effects can propagate along connected chains. Second, frustrated closed-loop motifs disrupt synchronous dynamics, enabling metastable configurations of zero-lag synchrony to coexist. We document these two complementary influences in small motifs and illustrate how these effects underpin stable versus metastable phase-synchronization patterns in prototypical modular networks and in large-scale cortical networks of the macaque (CoCoMac). We find that the variability of synchronization patterns depends on the inter-node time delay, increases with the network size and is maximized for intermediate coupling strengths. We hypothesize that the dialectic influences of resonance versus frustration may form a dynamic substrate for flexible neuronal integration, an essential platform across diverse cognitive processes.     

A method for the estimation of functional brain connectivity from time-series data

A central issue in cognitive neuroscience is which cortical areas are involved in managing information processing in a cognitive task and to understand their temporal interactions. Since the transfer of information in the form of electrical activity from one cortical region will in turn evoke electrical activity in other regions, the analysis of temporal synchronization provides a tool to understand neuronal information processing between cortical regions. We adopt a method for revealing time-dependent functional connectivity. We apply statistical analyses of phases to recover the information flow and the functional connectivity between cortical regions for high temporal resolution data. We further develop an evaluation method for these techniques based on two kinds of model networks. These networks consist of coupled Rössler attractors or of coupled stochastic Ornstein–Uhlenbeck systems. The implemented time-dependent coupling includes uni- and bi-directional connectivities as well as time delayed feedback. The synchronization dynamics of these networks are analyzed using the mean phase coherence, based on averaging over phase-differences, and the general synchronization index. The latter is based on the Shannon entropy. The combination of these with a parametric time delay forms the basis of a connectivity pattern, which includes the temporal and time lagged dynamics of the synchronization between two sources. We model and discuss potential artifacts. We find that the general phase measures are remarkably stable. They produce highly comparable results for stochastic and periodic systems. Moreover, the methods proves useful for identifying brief periods of phase coupling and delays. Therefore, we propose that the method is useful as a basis for generating potential functional connective models.     

Optimal Information Representation and Criticality in an Adaptive Sensory Recurrent Neuronal Network

Recurrent connections play an important role in cortical function, yet their exact contribution to the network computation remains unknown. The principles guiding the long-term evolution of these connections are poorly understood as well. Therefore, gaining insight into their computational role and into the mechanism shaping their pattern would be of great importance. To that end, we studied the learning dynamics and emergent recurrent connectivity in a sensory network model based on a first-principle information theoretic approach. As a test case, we applied this framework to a model of a hypercolumn in the visual cortex and found that the evolved connections between orientation columns have a "Mexican hat" profile, consistent with empirical data and previous modeling work. Furthermore, we found that optimal information representation is achieved when the network operates near a critical point in its dynamics. Neuronal networks working near such a phase transition are most sensitive to their inputs and are thus optimal in terms of information representation. Nevertheless, a mild change in the pattern of interactions may cause such networks to undergo a transition into a different regime of behavior in which the network activity is dominated by its internal recurrent dynamics and does not reflect the objective input. We discuss several mechanisms by which the pattern of interactions can be driven into this supercritical regime and relate them to various neurological and neuropsychiatric phenomena.     

Identifying disease-associated biomarker network features through conditional graphical model

Biomarkers are often organized into networks, in which the strengths of network connections vary across subjects depending on subject-specific covariates (eg, genetic variants). Variation of network connections, as subject-specific feature variables, has been found to predict disease clinical outcome. In this work, we develop a two-stage method to estimate biomarker networks that account for heterogeneity among subjects and evaluate network's association with disease clinical outcome. In the first stage, we propose a conditional Gaussian graphical model with mean and precision matrix depending on covariates to obtain covariate-dependent networks with connection strengths varying across subjects while assuming homogeneous network structure. In the second stage, we evaluate clinical utility of network measures (connection strengths) estimated from the first stage. The second-stage analysis provides the relative predictive power of between-region network measures on clinical impairment in the context of regional biomarkers and existing disease risk factors. We assess the performance of proposed method by extensive simulation studies and application to a Huntington's disease (HD) study to investigate the effect of HD causal gene on the rate of change in motor symptom through affecting brain subcortical and cortical gray matter atrophy connections. We show that cortical network connections and subcortical volumes, but not subcortical connections are identified to be predictive of clinical motor function deterioration. We validate these findings in an independent HD study. Lastly, highly similar patterns seen in the gray matter connections and a previous white matter connectivity study suggest a shared biological mechanism for HD and support the hypothesis that white matter loss is a direct result of neuronal loss as opposed to the loss of myelin or dysmyelination.     

Changes in Task-Related Functional Connectivity across Multiple Spatial Scales Are Related to Reading Performance

Reading requires the interaction of a distributed set of cortical areas whose distinct patterns give rise to a wide range of individual skill. However, the nature of these neural interactions and their relation to reading performance are still poorly understood. Functional connectivity analyses of fMRI data can be used to characterize the nature of interactivity of distributed brain networks, yet most previous studies have focused on connectivity during task-free (i.e., “resting state”) conditions. Here, we report new methods for assessing task-related functional connectivity using data-driven graph theoretical methods and describe how large-scale patterns of connectivity relate to individual variability in reading performance among children. We found that connectivity patterns of subjects performing a reading task could be decomposed hierarchically into multiple sub-networks, and we observed stronger long-range interaction between sub-networks in subjects with higher task accuracy. Additionally, we found a network of hub regions known to be critical to reading that displays increased short-range synchronization in higher accuracy subjects. These individual differences in task-related functional connectivity reveal that increased interaction between distant regions, coupled with selective local integration within key regions, is associated with better reading performance. Importantly, we show that task-related neuroimaging data contains far more information than usually extracted via standard univariate analyses – information that can meaningfully relate neural connectivity patterns to cognition and task.     

Role of structural inhomogeneities in resting-state brain dynamics

Brain imaging methods allow a non-invasive assessment of both structural and functional connectivity. However, the mechanism of how functional connectivity arises in a structured network of interacting neural populations is as yet poorly understood. Here we use a modeling approach to explore the way in which functional correlations arise from underlying structural connections taking into account inhomogeneities in the interactions between the brain regions of interest. The local dynamics of a neural population is assumed to be of phase-oscillator type. The considered structural connectivity patterns describe long-range anatomical connections between interacting neural elements. We find a dependence of the simulated functional connectivity patterns on the parameters governing the dynamics. We calculate graph-theoretic measures of the functional network topology obtained from numerical simulations. The effect of structural inhomogeneities in the coupling term on the observed network state is quantified by examining the relation between simulated and empirical functional connectivity. Importantly, we show that simulated and empirical functional connectivity agree for a narrow range of coupling strengths. We conclude that identification of functional connectivity during rest requires an analysis of the network dynamics.     

Anatomical connectivity defines the organization of clusters of cortical areas in the macaque monkey and the cat

The number of different cortical structures in mammalian brains and the number of extrinsic fibres linking these regions are both large. As with any complex system, systematic analysis is required to draw reliable conclusions about the organization of the complex neural networks comprising these numerous elements. One aspect of organization that has long been suspected is that cortical networks are organized into 'streams' or 'systems'. Here we report computational analyses capable of showing whether clusters of strongly interconnected areas are aspects of the global organization of cortical systems in macaque and cat. We used two different approaches to analyse compilations of corticocortical connection data from the macaque and the cat. The first approach, optimal set analysis, employed an explicit definition of a neural 'system' or 'stream', which was based on differential connectivity. We defined a two-component cost function that described the cost of the global cluster arrangement of areas in terms of the areas' connectivity within and between candidate clusters. Optimal cluster arrangements of cortical areas were then selected computationally from the very many possible arrangements, using an evolutionary optimization algorithm. The second approach, non-parametric cluster analysis (NPCA), grouped cortical areas on the basis of their proximity in multidimensional scaling representations. We used non-metric multidimensional scaling to represent the cortical connectivity structures metrically in two and five dimensions. NPCA then analysed these representations to determine the nature of the clusters for a wide range of different cluster shape parameters. The results from both approaches largely agreed. They showed that macaque and cat cortices are organized into densely intra-connected clusters of areas, and identified the constituent members of the clusters. These clusters reflected functionally specialized sets of cortical areas, suggesting that structure and function are closely linked at this gross, systems level.     

Dynamics of diluted attractor neural networks with delays

We present an analysis of the attractors of a deterministic dynamics in formal neural networks characterized by binary threshold units and a nonsymmetric connectivity. It is shown that in these networks a stored pattern or a pattern sequence is represented by a cloud of attractors rather than by a single attractor. Dilution, which we describe by a power-law scaling, and delayed couplings are shown to equip this type of network with a dynamic behaviour that is interesting enough for simplified models of biological motor systems. Received: 27 November 1992/Accepted in revised form: 22 September 1993