On how network architecture determines the dominant patterns of spontaneous neural activity

In the absence of sensory stimulation, neocortical circuits display complex patterns of neural activity. These patterns are thought to reflect relevant properties of the network, including anatomical features like its modularity. It is also assumed that the synaptic connections of the network constrain the repertoire of emergent, spontaneous patterns. Although the link between network architecture and network activity has been extensively investigated in the last few years from different perspectives, our understanding of the relationship between the network connectivity and the structure of its spontaneous activity is still incomplete. Using a general mathematical model of neural dynamics we have studied the link between spontaneous activity and the underlying network architecture. In particular, here we show mathematically how the synaptic connections between neurons determine the repertoire of spatial patterns displayed in the spontaneous activity. To test our theoretical result, we have also used the model to simulate spontaneous activity of a neural network, whose architecture is inspired by the patchy organization of horizontal connections between cortical columns in the neocortex of primates and other mammals. The dominant spatial patterns of the spontaneous activity, calculated as its principal components, coincide remarkably well with those patterns predicted from the network connectivity using our theory. The equivalence between the concept of dominant pattern and the concept of attractor of the network dynamics is also demonstrated. This in turn suggests new ways of investigating encoding and storage capabilities of neural networks.     

Variability v.s. synchronicity of neuronal activity in local cortical network models with different wiring topologies

Dynamical behavior of a biological neuronal network depends significantly on the spatial pattern of synaptic connections among neurons. While neuronal network dynamics has extensively been studied with simple wiring patterns, such as all-to-all or random synaptic connections, not much is known about the activity of networks with more complicated wiring topologies. Here, we examined how different wiring topologies may influence the response properties of neuronal networks, paying attention to irregular spike firing, which is known as a characteristic of in vivo cortical neurons, and spike synchronicity. We constructed a recurrent network model of realistic neurons and systematically rewired the recurrent synapses to change the network topology, from a localized regular and a “small-world” network topology to a distributed random network topology. Regular and small-world wiring patterns greatly increased the irregularity or the coefficient of variation (Cv) of output spike trains, whereas such an increase was small in random connectivity patterns. For given strength of recurrent synapses, the firing irregularity exhibited monotonous decreases from the regular to the random network topology. By contrast, the spike coherence between an arbitrary neuron pair exhibited a non-monotonous dependence on the topological wiring pattern. More precisely, the wiring pattern to maximize the spike coherence varied with the strength of recurrent synapses. In a certain range of the synaptic strength, the spike coherence was maximal in the small-world network topology, and the long-range connections introduced in this wiring changed the dependence of spike synchrony on the synaptic strength moderately. However, the effects of this network topology were not really special in other properties of network activity. Action Editor: Xiao-Jing Wang     

Nonoptimal component placement, but short processing paths, due to long-distance projections in neural systems

It has been suggested that neural systems across several scales of organization show optimal component placement, in which any spatial rearrangement of the components would lead to an increase of total wiring. Using extensive connectivity datasets for diverse neural networks combined with spatial coordinates for network nodes, we applied an optimization algorithm to the network layouts, in order to search for wire-saving component rearrangements. We found that optimized component rearrangements could substantially reduce total wiring length in all tested neural networks. Specifically, total wiring among 95 primate (Macaque) cortical areas could be decreased by 32%, and wiring of neuronal networks in the nematode Caenorhabditis elegans could be reduced by 48% on the global level, and by 49% for neurons within frontal ganglia. Wiring length reductions were possible due to the existence of long-distance projections in neural networks. We explored the role of these projections by comparing the original networks with minimally rewired networks of the same size, which possessed only the shortest possible connections. In the minimally rewired networks, the number of processing steps along the shortest paths between components was significantly increased compared to the original networks. Additional benchmark comparisons also indicated that neural networks are more similar to network layouts that minimize the length of processing paths, rather than wiring length. These findings suggest that neural systems are not exclusively optimized for minimal global wiring, but for a variety of factors including the minimization of processing steps.     

Self-Organization of Microcircuits in Networks of Spiking Neurons with Plastic Synapses

The synaptic connectivity of cortical networks features an overrepresentation of certain wiring motifs compared to simple random-network models. This structure is shaped, in part, by synaptic plasticity that promotes or suppresses connections between neurons depending on their joint spiking activity. Frequently, theoretical studies focus on how feedforward inputs drive plasticity to create this network structure. We study the complementary scenario of self-organized structure in a recurrent network, with spike timing-dependent plasticity driven by spontaneous dynamics. We develop a self-consistent theory for the evolution of network structure by combining fast spiking covariance with a slow evolution of synaptic weights. Through a finite-size expansion of network dynamics we obtain a low-dimensional set of nonlinear differential equations for the evolution of two-synapse connectivity motifs. With this theory in hand, we explore how the form of the plasticity rule drives the evolution of microcircuits in cortical networks. When potentiation and depression are in approximate balance, synaptic dynamics depend on weighted divergent, convergent, and chain motifs. For additive, Hebbian STDP these motif interactions create instabilities in synaptic dynamics that either promote or suppress the initial network structure. Our work provides a consistent theoretical framework for studying how spiking activity in recurrent networks interacts with synaptic plasticity to determine network structure.     

Artificial neural network properties associated with wiring patterns in the visual projections of vertebrates and arthropods

We model the functioning of different wiring schemes in visual projections using artificial neural networks and so speculate on selective factors underlying taxonomic variation in neural architecture. We model the high connective overlap of vertebrates (where networks have a dense mesh of connections) and the less overlapping, more modular architecture of arthropods. We also consider natural variation in these basic wiring schemes. Generally, arthropod networks are as efficient or more efficient in functioning compared to vertebrate networks. They do not show the confusion effect (decreasing targeting accuracy with increasing input group size), and they train as well or better. Arthropod networks are, however, generally poorer at reconstructing novel inputs. The ability of vertebrate networks to effectively process novel stimuli could promote behavioral sophistication and drive the evolution of vertebrate wiring schemes. Vertebrate networks with less connective overlap have, surprisingly, similar or superior properties compared to those with high connective overlap. Thus, the partial connective overlap seen in real vertebrate visual projections may be an optimal, evolved solution. Arthropod networks with and without whole-cell neural connections within neural layers have similar properties. This indicates that neural connections mediated by offshoots of single cells (dendrites) may be fundamental to generating the confusion effect.     

Mesoscopic Organization Reveals the Constraints Governing Caenorhabditis elegans Nervous System

One of the biggest challenges in biology is to understand how activity at the cellular level of neurons, as a result of their mutual interactions, leads to the observed behavior of an organism responding to a variety of environmental stimuli. Investigating the intermediate or mesoscopic level of organization in the nervous system is a vital step towards understanding how the integration of micro-level dynamics results in macro-level functioning. The coordination of many different co-occurring processes at this level underlies the command and control of overall network activity. In this paper, we have considered the somatic nervous system of the nematode Caenorhabditis elegans, for which the entire neuronal connectivity diagram is known. We focus on the organization of the system into modules, i.e., neuronal groups having relatively higher connection density compared to that of the overall network. We show that this mesoscopic feature cannot be explained exclusively in terms of considerations such as, optimizing for resource constraints (viz., total wiring cost) and communication efficiency (i.e., network path length). Even including information about the genetic relatedness of the cells cannot account for the observed modular structure. Comparison with other complex networks designed for efficient transport (of signals or resources) implies that neuronal networks form a distinct class. This suggests that the principal function of the network, viz., processing of sensory information resulting in appropriate motor response, may be playing a vital role in determining the connection topology. Using modular spectral analysis we make explicit the intimate relation between function and structure in the nervous system. This is further brought out by identifying functionally critical neurons purely on the basis of patterns of intra- and inter-modular connections. Our study reveals how the design of the nervous system reflects several constraints, including its key functional role as a processor of information.     

Emergence of Metastable State Dynamics in Interconnected Cortical Networks with Propagation Delays

The importance of the large number of thin-diameter and unmyelinated axons that connect different cortical areas is unknown. The pronounced propagation delays in these axons may prevent synchronization of cortical networks and therefore hinder efficient information integration and processing. Yet, such global information integration across cortical areas is vital for higher cognitive function. We hypothesized that delays in communication between cortical areas can disrupt synchronization and therefore enhance the set of activity trajectories and computations interconnected networks can perform. To evaluate this hypothesis, we studied the effect of long-range cortical projections with propagation delays in interconnected large-scale cortical networks that exhibited spontaneous rhythmic activity. Long-range connections with delays caused the emergence of metastable, spatio-temporally distinct activity states between which the networks spontaneously transitioned. Interestingly, the observed activity patterns correspond to macroscopic network dynamics such as globally synchronized activity, propagating wave fronts, and spiral waves that have been previously observed in neurophysiological recordings from humans and animal models. Transient perturbations with simulated transcranial alternating current stimulation (tACS) confirmed the multistability of the interconnected networks by switching the networks between these metastable states. Our model thus proposes that slower long-range connections enrich the landscape of activity states and represent a parsimonious mechanism for the emergence of multistability in cortical networks. These results further provide a mechanistic link between the known deficits in connectivity and cortical state dynamics in neuropsychiatric illnesses such as schizophrenia and autism, as well as suggest non-invasive brain stimulation as an effective treatment for these illnesses.     

The effect of space in plant–animal mutualistic networks: insights from a simulation study

The topology of plant–animal mutualistic networks has the potential to determine the ecological and evolutionary dynamics of interacting species. Many mechanisms have been proposed as explanations of observed network patterns; however, the fact that plant–animal interactions are inherently spatial has so far been ignored. Using a simulation model of frugivorous birds foraging in spatially explicit landscapes we evaluated how plant distribution and the scale of bird movement decisions influenced species interaction probabilities and the resulting network properties. Spatial aggregation and limited animal mobility restricted encounter probabilities, so that the distribution of animal visits per plant deviated strongly from the binomial distribution expected for a well-mixed system. Lack of mixing in turn resulted in a strong decrease in network connectance, a weak decrease in nestedness, stronger interactions, greater strength asymmetry and the unexpected presence/absence of some interactions. Our results suggest that spatial processes may contribute substantially to structure plant–animal mutualistic networks.     

Emergence of Slow-Switching Assemblies in Structured Neuronal Networks

Unraveling the interplay between connectivity and spatio-temporal dynamics in neuronal networks is a key step to advance our understanding of neuronal information processing. Here we investigate how particular features of network connectivity underpin the propensity of neural networks to generate slow-switching assembly (SSA) dynamics, i.e., sustained epochs of increased firing within assemblies of neurons which transition slowly between different assemblies throughout the network. We show that the emergence of SSA activity is linked to spectral properties of the asymmetric synaptic weight matrix. In particular, the leading eigenvalues that dictate the slow dynamics exhibit a gap with respect to the bulk of the spectrum, and the associated Schur vectors exhibit a measure of block-localization on groups of neurons, thus resulting in coherent dynamical activity on those groups. Through simple rate models, we gain analytical understanding of the origin and importance of the spectral gap, and use these insights to develop new network topologies with alternative connectivity paradigms which also display SSA activity. Specifically, SSA dynamics involving excitatory and inhibitory neurons can be achieved by modifying the connectivity patterns between both types of neurons. We also show that SSA activity can occur at multiple timescales reflecting a hierarchy in the connectivity, and demonstrate the emergence of SSA in small-world like networks. Our work provides a step towards understanding how network structure (uncovered through advancements in neuroanatomy and connectomics) can impact on spatio-temporal neural activity and constrain the resulting dynamics.     

Trade-off between Multiple Constraints Enables Simultaneous Formation of Modules and Hubs in Neural Systems

The formation of the complex network architecture of neural systems is subject to multiple structural and functional constraints. Two obvious but apparently contradictory constraints are low wiring cost and high processing efficiency, characterized by short overall wiring length and a small average number of processing steps, respectively. Growing evidence shows that neural networks are results from a trade-off between physical cost and functional value of the topology. However, the relationship between these competing constraints and complex topology is not well understood quantitatively. We explored this relationship systematically by reconstructing two known neural networks, Macaque cortical connectivity and C. elegans neuronal connections, from combinatory optimization of wiring cost and processing efficiency constraints, using a control parameter , and comparing the reconstructed networks to the real networks. We found that in both neural systems, the reconstructed networks derived from the two constraints can reveal some important relations between the spatial layout of nodes and the topological connectivity, and match several properties of the real networks. The reconstructed and real networks had a similar modular organization in a broad range of , resulting from spatial clustering of network nodes. Hubs emerged due to the competition of the two constraints, and their positions were close to, and partly coincided, with the real hubs in a range of values. The degree of nodes was correlated with the density of nodes in their spatial neighborhood in both reconstructed and real networks. Generally, the rebuilt network matched a significant portion of real links, especially short-distant ones. These findings provide clear evidence to support the hypothesis of trade-off between multiple constraints on brain networks. The two constraints of wiring cost and processing efficiency, however, cannot explain all salient features in the real networks. The discrepancy suggests that there are further relevant factors that are not yet captured here.     

Structural properties of the Caenorhabditis elegans neuronal network

Despite recent interest in reconstructing neuronal networks, complete wiring diagrams on the level of individual synapses remain scarce and the insights into function they can provide remain unclear. Even for Caenorhabditis elegans, whose neuronal network is relatively small and stereotypical from animal to animal, published wiring diagrams are neither accurate nor complete and self-consistent. Using materials from White et al. and new electron micrographs we assemble whole, self-consistent gap junction and chemical synapse networks of hermaphrodite C. elegans. We propose a method to visualize the wiring diagram, which reflects network signal flow. We calculate statistical and topological properties of the network, such as degree distributions, synaptic multiplicities, and small-world properties, that help in understanding network signal propagation. We identify neurons that may play central roles in information processing, and network motifs that could serve as functional modules of the network. We explore propagation of neuronal activity in response to sensory or artificial stimulation using linear systems theory and find several activity patterns that could serve as substrates of previously described behaviors. Finally, we analyze the interaction between the gap junction and the chemical synapse networks. Since several statistical properties of the C. elegans network, such as multiplicity and motif distributions are similar to those found in mammalian neocortex, they likely point to general principles of neuronal networks. The wiring diagram reported here can help in understanding the mechanistic basis of behavior by generating predictions about future experiments involving genetic perturbations, laser ablations, or monitoring propagation of neuronal activity in response to stimulation.     

The functional geometry of local and horizontal connections in a model of V1.

A mathematical model of interacting hypercolumns in primary visual cortex (V1) is presented that incorporates details concerning the geometry of local and long-range horizontal connections. Each hypercolumn is modeled as a network of interacting excitatory and inhibitory neural populations with orientation and spatial frequency preferences organized around a pair of pinwheels. The pinwheels are arranged on a planar lattice, reflecting the crystalline-like structure of cortex. Local interactions within a hypercolumn generate orientation and spatial frequency tuning curves, which are modulated by horizontal connections between different hypercolumns on the lattice. The symmetry properties of the local and long-range connections play an important role in determining the types of spontaneous activity patterns that can arise in cortex.     

Reconstructing the three-dimensional GABAergic microcircuit of the striatum

A system's wiring constrains its dynamics, yet modelling of neural structures often overlooks the specific networks formed by their neurons. We developed an approach for constructing anatomically realistic networks and reconstructed the GABAergic microcircuit formed by the medium spiny neurons (MSNs) and fast-spiking interneurons (FSIs) of the adult rat striatum. We grew dendrite and axon models for these neurons and extracted probabilities for the presence of these neurites as a function of distance from the soma. From these, we found the probabilities of intersection between the neurites of two neurons given their inter-somatic distance, and used these to construct three-dimensional striatal networks. The MSN dendrite models predicted that half of all dendritic spines are within 100μm of the soma. The constructed networks predict distributions of gap junctions between FSI dendrites, synaptic contacts between MSNs, and synaptic inputs from FSIs to MSNs that are consistent with current estimates. The models predict that to achieve this, FSIs should be at most 1% of the striatal population. They also show that the striatum is sparsely connected: FSI-MSN and MSN-MSN contacts respectively form 7% and 1.7% of all possible connections. The models predict two striking network properties: the dominant GABAergic input to a MSN arises from neurons with somas at the edge of its dendritic field; and FSIs are inter-connected on two different spatial scales: locally by gap junctions and distally by synapses. We show that both properties influence striatal dynamics: the most potent inhibition of a MSN arises from a region of striatum at the edge of its dendritic field; and the combination of local gap junction and distal synaptic networks between FSIs sets a robust input-output regime for the MSN population. Our models thus intimately link striatal micro-anatomy to its dynamics, providing a biologically grounded platform for further study.     

How structure determines correlations in neuronal networks

Networks are becoming a ubiquitous metaphor for the understanding of complex biological systems, spanning the range between molecular signalling pathways, neural networks in the brain, and interacting species in a food web. In many models, we face an intricate interplay between the topology of the network and the dynamics of the system, which is generally very hard to disentangle. A dynamical feature that has been subject of intense research in various fields are correlations between the noisy activity of nodes in a network. We consider a class of systems, where discrete signals are sent along the links of the network. Such systems are of particular relevance in neuroscience, because they provide models for networks of neurons that use action potentials for communication. We study correlations in dynamic networks with arbitrary topology, assuming linear pulse coupling. With our novel approach, we are able to understand in detail how specific structural motifs affect pairwise correlations. Based on a power series decomposition of the covariance matrix, we describe the conditions under which very indirect interactions will have a pronounced effect on correlations and population dynamics. In random networks, we find that indirect interactions may lead to a broad distribution of activation levels with low average but highly variable correlations. This phenomenon is even more pronounced in networks with distance dependent connectivity. In contrast, networks with highly connected hubs or patchy connections often exhibit strong average correlations. Our results are particularly relevant in view of new experimental techniques that enable the parallel recording of spiking activity from a large number of neurons, an appropriate interpretation of which is hampered by the currently limited understanding of structure-dynamics relations in complex networks.     

The wiring economy principle: connectivity determines anatomy in the human brain

Minimization of the wiring cost of white matter fibers in the human brain appears to be an organizational principle. We investigate this aspect in the human brain using whole brain connectivity networks extracted from high resolution diffusion MRI data of 14 normal volunteers. We specifically address the question of whether brain anatomy determines its connectivity or vice versa. Unlike previous studies we use weighted networks, where connections between cortical nodes are real-valued rather than binary off-on connections. In one set of analyses we found that the connectivity structure of the brain has near optimal wiring cost compared to random networks with the same number of edges, degree distribution and edge weight distribution. A specifically designed minimization routine could not find cheaper wiring without significantly degrading network performance. In another set of analyses we kept the observed brain network topology and connectivity but allowed nodes to freely move on a 3D manifold topologically identical to the brain. An efficient minimization routine was written to find the lowest wiring cost configuration. We found that beginning from any random configuration, the nodes invariably arrange themselves in a configuration with a striking resemblance to the brain. This confirms the widely held but poorly tested claim that wiring economy is a driving principle of the brain. Intriguingly, our results also suggest that the brain mainly optimizes for the most desirable network connectivity, and the observed brain anatomy is merely a result of this optimization.     

Inter-Network Interactions: Impact of Connections between Oscillatory Neuronal Networks on Oscillation Frequency and Pattern

Oscillations in electrical activity are a characteristic feature of many brain networks and display a wide variety of temporal patterns. A network may express a single oscillation frequency, alternate between two or more distinct frequencies, or continually express multiple frequencies. In addition, oscillation amplitude may fluctuate over time. The origin of this complex repertoire of activity remains unclear. Different cortical layers often produce distinct oscillation frequencies. To investigate whether interactions between different networks could contribute to the variety of oscillation patterns, we created two model networks, one generating on its own a relatively slow frequency (20 Hz; slow network) and one generating a fast frequency (32 Hz; fast network). Taking either the slow or the fast network as source network projecting connections to the other, or target, network, we systematically investigated how type and strength of inter-network connections affected target network activity. For high inter-network connection strengths, we found that the slow network was more effective at completely imposing its rhythm on the fast network than the other way around. The strongest entrainment occurred when excitatory cells of the slow network projected to excitatory or inhibitory cells of the fast network. The fast network most strongly imposed its rhythm on the slow network when its excitatory cells projected to excitatory cells of the slow network. Interestingly, for lower inter-network connection strengths, multiple frequencies coexisted in the target network. Just as observed in rat prefrontal cortex, the target network could express multiple frequencies at the same time, alternate between two distinct oscillation frequencies, or express a single frequency with alternating episodes of high and low amplitude. Together, our results suggest that input from other oscillating networks may markedly alter a network''s frequency spectrum and may partly be responsible for the rich repertoire of temporal oscillation patterns observed in the brain.     

Network analysis of protein dynamics

The network paradigm is increasingly used to describe the topology and dynamics of complex systems. Here, we review the results of the topological analysis of protein structures as molecular networks describing their small-world character, and the role of hubs and central network elements in governing enzyme activity, allosteric regulation, protein motor function, signal transduction and protein stability. We summarize available data how central network elements are enriched in active centers and ligand binding sites directing the dynamics of the entire protein. We assess the feasibility of conformational and energy networks to simplify the vast complexity of rugged energy landscapes and to predict protein folding and dynamics. Finally, we suggest that modular analysis, novel centrality measures, hierarchical representation of networks and the analysis of network dynamics will soon lead to an expansion of this field.     

A Signature of Attractor Dynamics in the CA3 Region of the Hippocampus

The notion of attractor networks is the leading hypothesis for how associative memories are stored and recalled. A defining anatomical feature of such networks is excitatory recurrent connections. These “attract” the firing pattern of the network to a stored pattern, even when the external input is incomplete (pattern completion). The CA3 region of the hippocampus has been postulated to be such an attractor network; however, the experimental evidence has been ambiguous, leading to the suggestion that CA3 is not an attractor network. In order to resolve this controversy and to better understand how CA3 functions, we simulated CA3 and its input structures. In our simulation, we could reproduce critical experimental results and establish the criteria for identifying attractor properties. Notably, under conditions in which there is continuous input, the output should be “attracted” to a stored pattern. However, contrary to previous expectations, as a pattern is gradually “morphed” from one stored pattern to another, a sharp transition between output patterns is not expected. The observed firing patterns of CA3 meet these criteria and can be quantitatively accounted for by our model. Notably, as morphing proceeds, the activity pattern in the dentate gyrus changes; in contrast, the activity pattern in the downstream CA3 network is attracted to a stored pattern and thus undergoes little change. We furthermore show that other aspects of the observed firing patterns can be explained by learning that occurs during behavioral testing. The CA3 thus displays both the learning and recall signatures of an attractor network. These observations, taken together with existing anatomical and behavioral evidence, make the strong case that CA3 constructs associative memories based on attractor dynamics.     

Coding and synaptic processing of sensory information in the glomerular layer of the olfactory bulb

Input from olfactory receptor neurons is first organized and processed in the glomerular layer of the olfactory bulb. Olfactory glomeruli serve as functional units in coding olfactory information and contain a complex network of synaptic connections. Odor information has long been thought to be represented by spatial patterns of glomerular activation; recent work has, additionally, shown that these patterns are temporally dynamic. At the same time, recent advances in our understanding of the glomerular network suggest that glomerular processing serves to temporally sharpen these dynamics and to modulate spatial patterns of glomerular activity. We speculate that odor representations and their postsynaptic processing are tuned to and shaped by the sniffing behavior of the animal.     

Encoding time in neural dynamic regimes with distinct computational tradeoffs

Converging evidence suggests the brain encodes time in dynamic patterns of neural activity, including neural sequences, ramping activity, and complex dynamics. Most temporal tasks, however, require more than just encoding time, and can have distinct computational requirements including the need to exhibit temporal scaling, generalize to novel contexts, or robustness to noise. It is not known how neural circuits can encode time and satisfy distinct computational requirements, nor is it known whether similar patterns of neural activity at the population level can exhibit dramatically different computational or generalization properties. To begin to answer these questions, we trained RNNs on two timing tasks based on behavioral studies. The tasks had different input structures but required producing identically timed output patterns. Using a novel framework we quantified whether RNNs encoded two intervals using either of three different timing strategies: scaling, absolute, or stimulus-specific dynamics. We found that similar neural dynamic patterns at the level of single intervals, could exhibit fundamentally different properties, including, generalization, the connectivity structure of the trained networks, and the contribution of excitatory and inhibitory neurons. Critically, depending on the task structure RNNs were better suited for generalization or robustness to noise. Further analysis revealed different connection patterns underlying the different regimes. Our results predict that apparently similar neural dynamic patterns at the population level (e.g., neural sequences) can exhibit fundamentally different computational properties in regards to their ability to generalize to novel stimuli and their robustness to noise—and that these differences are associated with differences in network connectivity and distinct contributions of excitatory and inhibitory neurons. We also predict that the task structure used in different experimental studies accounts for some of the experimentally observed variability in how networks encode time.