Trade-off between Multiple Constraints Enables Simultaneous Formation of Modules and Hubs in Neural Systems

The formation of the complex network architecture of neural systems is subject to multiple structural and functional constraints. Two obvious but apparently contradictory constraints are low wiring cost and high processing efficiency, characterized by short overall wiring length and a small average number of processing steps, respectively. Growing evidence shows that neural networks are results from a trade-off between physical cost and functional value of the topology. However, the relationship between these competing constraints and complex topology is not well understood quantitatively. We explored this relationship systematically by reconstructing two known neural networks, Macaque cortical connectivity and C. elegans neuronal connections, from combinatory optimization of wiring cost and processing efficiency constraints, using a control parameter , and comparing the reconstructed networks to the real networks. We found that in both neural systems, the reconstructed networks derived from the two constraints can reveal some important relations between the spatial layout of nodes and the topological connectivity, and match several properties of the real networks. The reconstructed and real networks had a similar modular organization in a broad range of , resulting from spatial clustering of network nodes. Hubs emerged due to the competition of the two constraints, and their positions were close to, and partly coincided, with the real hubs in a range of values. The degree of nodes was correlated with the density of nodes in their spatial neighborhood in both reconstructed and real networks. Generally, the rebuilt network matched a significant portion of real links, especially short-distant ones. These findings provide clear evidence to support the hypothesis of trade-off between multiple constraints on brain networks. The two constraints of wiring cost and processing efficiency, however, cannot explain all salient features in the real networks. The discrepancy suggests that there are further relevant factors that are not yet captured here.     

From Caenorhabditis elegans to the human connectome: a specific modular organization increases metabolic,functional and developmental efficiency

The connectome, or the entire connectivity of a neural system represented by a network, ranges across various scales from synaptic connections between individual neurons to fibre tract connections between brain regions. Although the modularity they commonly show has been extensively studied, it is unclear whether the connection specificity of such networks can already be fully explained by the modularity alone. To answer this question, we study two networks, the neuronal network of Caenorhabditis elegans and the fibre tract network of human brains obtained through diffusion spectrum imaging. We compare them to their respective benchmark networks with varying modularities, which are generated by link swapping to have desired modularity values. We find several network properties that are specific to the neural networks and cannot be fully explained by the modularity alone. First, the clustering coefficient and the characteristic path length of both C. elegans and human connectomes are higher than those of the benchmark networks with similar modularity. High clustering coefficient indicates efficient local information distribution, and high characteristic path length suggests reduced global integration. Second, the total wiring length is smaller than for the alternative configurations with similar modularity. This is due to lower dispersion of connections, which means each neuron in the C. elegans connectome or each region of interest in the human connectome reaches fewer ganglia or cortical areas, respectively. Third, both neural networks show lower algorithmic entropy compared with the alternative arrangements. This implies that fewer genes are needed to encode for the organization of neural systems. While the first two findings show that the neural topologies are efficient in information processing, this suggests that they are also efficient from a developmental point of view. Together, these results show that neural systems are organized in such a way as to yield efficient features beyond those given by their modularity alone.     

Artificial neural network properties associated with wiring patterns in the visual projections of vertebrates and arthropods

We model the functioning of different wiring schemes in visual projections using artificial neural networks and so speculate on selective factors underlying taxonomic variation in neural architecture. We model the high connective overlap of vertebrates (where networks have a dense mesh of connections) and the less overlapping, more modular architecture of arthropods. We also consider natural variation in these basic wiring schemes. Generally, arthropod networks are as efficient or more efficient in functioning compared to vertebrate networks. They do not show the confusion effect (decreasing targeting accuracy with increasing input group size), and they train as well or better. Arthropod networks are, however, generally poorer at reconstructing novel inputs. The ability of vertebrate networks to effectively process novel stimuli could promote behavioral sophistication and drive the evolution of vertebrate wiring schemes. Vertebrate networks with less connective overlap have, surprisingly, similar or superior properties compared to those with high connective overlap. Thus, the partial connective overlap seen in real vertebrate visual projections may be an optimal, evolved solution. Arthropod networks with and without whole-cell neural connections within neural layers have similar properties. This indicates that neural connections mediated by offshoots of single cells (dendrites) may be fundamental to generating the confusion effect.     

Allosteric Communication Occurs via Networks of Tertiary and Quaternary Motions in Proteins

Allosteric proteins bind an effector molecule at one site resulting in a functional change at a second site. We hypothesize that allosteric communication in proteins relies upon networks of quaternary (collective, rigid-body) and tertiary (residue–residue contact) motions. We argue that cyclic topology of these networks is necessary for allosteric communication. An automated algorithm identifies rigid bodies from the displacement between the inactive and the active structures and constructs “quaternary networks” from these rigid bodies and the substrate and effector ligands. We then integrate quaternary networks with a coarse-grained representation of contact rearrangements to form “global communication networks” (GCNs). The GCN reveals allosteric communication among all substrate and effector sites in 15 of 18 multidomain and multimeric proteins, while tertiary and quaternary networks exhibit such communication in only 4 and 3 of these proteins, respectively. Furthermore, in 7 of the 15 proteins connected by the GCN, 50% or more of the substrate-effector paths via the GCN are “interdependent” paths that do not exist via either the tertiary or the quaternary network. Substrate-effector “pathways” typically are not linear but rather consist of polycyclic networks of rigid bodies and clusters of rearranging residue contacts. These results argue for broad applicability of allosteric communication based on structural changes and demonstrate the utility of the GCN. Global communication networks may inform a variety of experiments on allosteric proteins as well as the design of allostery into non-allosteric proteins.     

Total Wiring Length Minimization of C. elegans Neural Network: A Constrained Optimization Approach

Using the most recent data on the connectivity of the C. elegans neural network, we find optimal two-dimensional positions of interneurons that minimize the total wiring length provided that the positions of motor and sensory neurons are fixed. The rationale behind fixing motor and sensory neurons is the following: while positions of motor and sensory neurons can be influenced by the locations of muscles and sensory organs they are attached to, the main function of interneurons is to connect other neurons, and their placement could try to minimize the wiring length. Solutions for l ¹, l ² and squared l ²–norm were obtained. For the Euclidean norm l ², the relative and absolute difference between the real and optimal total wiring lengths is minimal among these functions of distance. Additional network constraints were discussed such as assignment of different weights to electrical or chemical connections, fixation of “tail” interneurons, minimal interneural distance limitation, and others. These constraints were compared by their influence on the optimal positions of interneurons.     

Random Wiring,Ganglion Cell Mosaics,and the Functional Architecture of the Visual Cortex

The architecture of iso-orientation domains in the primary visual cortex (V1) of placental carnivores and primates apparently follows species invariant quantitative laws. Dynamical optimization models assuming that neurons coordinate their stimulus preferences throughout cortical circuits linking millions of cells specifically predict these invariants. This might indicate that V1’s intrinsic connectome and its functional architecture adhere to a single optimization principle with high precision and robustness. To validate this hypothesis, it is critical to closely examine the quantitative predictions of alternative candidate theories. Random feedforward wiring within the retino-cortical pathway represents a conceptually appealing alternative to dynamical circuit optimization because random dimension-expanding projections are believed to generically exhibit computationally favorable properties for stimulus representations. Here, we ask whether the quantitative invariants of V1 architecture can be explained as a generic emergent property of random wiring. We generalize and examine the stochastic wiring model proposed by Ringach and coworkers, in which iso-orientation domains in the visual cortex arise through random feedforward connections between semi-regular mosaics of retinal ganglion cells (RGCs) and visual cortical neurons. We derive closed-form expressions for cortical receptive fields and domain layouts predicted by the model for perfectly hexagonal RGC mosaics. Including spatial disorder in the RGC positions considerably changes the domain layout properties as a function of disorder parameters such as position scatter and its correlations across the retina. However, independent of parameter choice, we find that the model predictions substantially deviate from the layout laws of iso-orientation domains observed experimentally. Considering random wiring with the currently most realistic model of RGC mosaic layouts, a pairwise interacting point process, the predicted layouts remain distinct from experimental observations and resemble Gaussian random fields. We conclude that V1 layout invariants are specific quantitative signatures of visual cortical optimization, which cannot be explained by generic random feedforward-wiring models.     

Variability v.s. synchronicity of neuronal activity in local cortical network models with different wiring topologies

Dynamical behavior of a biological neuronal network depends significantly on the spatial pattern of synaptic connections among neurons. While neuronal network dynamics has extensively been studied with simple wiring patterns, such as all-to-all or random synaptic connections, not much is known about the activity of networks with more complicated wiring topologies. Here, we examined how different wiring topologies may influence the response properties of neuronal networks, paying attention to irregular spike firing, which is known as a characteristic of in vivo cortical neurons, and spike synchronicity. We constructed a recurrent network model of realistic neurons and systematically rewired the recurrent synapses to change the network topology, from a localized regular and a “small-world” network topology to a distributed random network topology. Regular and small-world wiring patterns greatly increased the irregularity or the coefficient of variation (Cv) of output spike trains, whereas such an increase was small in random connectivity patterns. For given strength of recurrent synapses, the firing irregularity exhibited monotonous decreases from the regular to the random network topology. By contrast, the spike coherence between an arbitrary neuron pair exhibited a non-monotonous dependence on the topological wiring pattern. More precisely, the wiring pattern to maximize the spike coherence varied with the strength of recurrent synapses. In a certain range of the synaptic strength, the spike coherence was maximal in the small-world network topology, and the long-range connections introduced in this wiring changed the dependence of spike synchrony on the synaptic strength moderately. However, the effects of this network topology were not really special in other properties of network activity. Action Editor: Xiao-Jing Wang     

Models of cortical networks with long-range patchy projections

The cortex exhibits an intricate vertical and horizontal architecture, the latter often featuring spatially clustered projection patterns, so-called patches. Many network studies of cortical dynamics ignore such spatial structures and assume purely random wiring. Here, we focus on non-random network structures provided by long-range horizontal (patchy) connections that remain inside the gray matter. We investigate how the spatial arrangement of patchy projections influences global network topology and predict its impact on the activity dynamics of the network. Since neuroanatomical data on horizontal projections is rather sparse, we suggest and compare four candidate scenarios of how patchy connections may be established. To identify a set of characteristic network properties that enables us to pin down the differences between the resulting network models, we employ the framework of stochastic graph theory. We find that patchy projections provide an exceptionally efficient way of wiring, as the resulting networks tend to exhibit small-world properties with significantly reduced wiring costs. Furthermore, the eigenvalue spectra, as well as the structure of common in- and output of the networks suggest that different spatial connectivity patterns support distinct types of activity propagation.     

A biologically inspired neural network for dynamic programming

An artificial neural network with a two-layer feedback topology and generalized recurrent neurons, for solving nonlinear discrete dynamic optimization problems, is developed. A direct method to assign the weights of neural networks is presented. The method is based on Bellmann's Optimality Principle and on the interchange of information which occurs during the synaptic chemical processing among neurons. The neural network based algorithm is an advantageous approach for dynamic programming due to the inherent parallelism of the neural networks; further it reduces the severity of computational problems that can occur in methods like conventional methods. Some illustrative application examples are presented to show how this approach works out including the shortest path and fuzzy decision making problems.     

Interactogeneous: Disease Gene Prioritization Using Heterogeneous Networks and Full Topology Scores

Disease gene prioritization aims to suggest potential implications of genes in disease susceptibility. Often accomplished in a guilt-by-association scheme, promising candidates are sorted according to their relatedness to known disease genes. Network-based methods have been successfully exploiting this concept by capturing the interaction of genes or proteins into a score. Nonetheless, most current approaches yield at least some of the following limitations: (1) networks comprise only curated physical interactions leading to poor genome coverage and density, and bias toward a particular source; (2) scores focus on adjacencies (direct links) or the most direct paths (shortest paths) within a constrained neighborhood around the disease genes, ignoring potentially informative indirect paths; (3) global clustering is widely applied to partition the network in an unsupervised manner, attributing little importance to prior knowledge; (4) confidence weights and their contribution to edge differentiation and ranking reliability are often disregarded. We hypothesize that network-based prioritization related to local clustering on graphs and considering full topology of weighted gene association networks integrating heterogeneous sources should overcome the above challenges. We term such a strategy Interactogeneous. We conducted cross-validation tests to assess the impact of network sources, alternative path inclusion and confidence weights on the prioritization of putative genes for 29 diseases. Heat diffusion ranking proved the best prioritization method overall, increasing the gap to neighborhood and shortest paths scores mostly on single source networks. Heterogeneous associations consistently delivered superior performance over single source data across the majority of methods. Results on the contribution of confidence weights were inconclusive. Finally, the best Interactogeneous strategy, heat diffusion ranking and associations from the STRING database, was used to prioritize genes for Parkinson’s disease. This method effectively recovered known genes and uncovered interesting candidates which could be linked to pathogenic mechanisms of the disease.     

Ligand binding and circular permutation modify residue interaction network in DHFR

Residue interaction networks and loop motions are important for catalysis in dihydrofolate reductase (DHFR). Here, we investigate the effects of ligand binding and chain connectivity on network communication in DHFR. We carry out systematic network analysis and molecular dynamics simulations of the native DHFR and 19 of its circularly permuted variants by breaking the chain connections in ten folding element regions and in nine nonfolding element regions as observed by experiment. Our studies suggest that chain cleavage in folding element areas may deactivate DHFR due to large perturbations in the network properties near the active site. The protein active site is near or coincides with residues through which the shortest paths in the residue interaction network tend to go. Further, our network analysis reveals that ligand binding has “network-bridging effects” on the DHFR structure. Our results suggest that ligand binding leads to a modification, with most of the interaction networks now passing through the cofactor, shortening the average shortest path. Ligand binding at the active site has profound effects on the network centrality, especially the closeness.     

The color phi phenomenon: Not so special,after all?

We show how anomalous time reversal of stimuli and their associated responses can exist in very small connectionist models. These networks are built from dynamical toy model neurons which adhere to a minimal set of biologically plausible properties. The appearance of a “ghost” response, temporally and spatially located in between responses caused by actual stimuli, as in the phi phenomenon, is demonstrated in a similar small network, where it is caused by priming and long-distance feedforward paths. We then demonstrate that the color phi phenomenon can be present in an echo state network, a recurrent neural network, without explicitly training for the presence of the effect, such that it emerges as an artifact of the dynamical processing. Our results suggest that the color phi phenomenon might simply be a feature of the inherent dynamical and nonlinear sensory processing in the brain and in and of itself is not related to consciousness.     

Identification of Phytoplankton from Flow Cytometry Data by Using Radial Basis Function Neural Networks

We describe here the application of a type of artificial neural network, the Gaussian radial basis function (RBF) network, in the identification of a large number of phytoplankton strains from their 11-dimensional flow cytometric characteristics measured by the European Optical Plankton Analyser instrument. The effect of network parameters on optimization is examined. Optimized RBF networks recognized 34 species of marine and freshwater phytoplankton with 91.5% success overall. The relative importance of each measured parameter in discriminating these data and the behavior of RBF networks in response to data from “novel” species (species not present in the training data) were analyzed.     

Cliques and cavities in the human connectome

Encoding brain regions and their connections as a network of nodes and edges captures many of the possible paths along which information can be transmitted as humans process and perform complex behaviors. Because cognitive processes involve large, distributed networks of brain areas, principled examinations of multi-node routes within larger connection patterns can offer fundamental insights into the complexities of brain function. Here, we investigate both densely connected groups of nodes that could perform local computations as well as larger patterns of interactions that would allow for parallel processing. Finding such structures necessitates that we move from considering exclusively pairwise interactions to capturing higher order relations, concepts naturally expressed in the language of algebraic topology. These tools can be used to study mesoscale network structures that arise from the arrangement of densely connected substructures called cliques in otherwise sparsely connected brain networks. We detect cliques (all-to-all connected sets of brain regions) in the average structural connectomes of 8 healthy adults scanned in triplicate and discover the presence of more large cliques than expected in null networks constructed via wiring minimization, providing architecture through which brain network can perform rapid, local processing. We then locate topological cavities of different dimensions, around which information may flow in either diverging or converging patterns. These cavities exist consistently across subjects, differ from those observed in null model networks, and – importantly – link regions of early and late evolutionary origin in long loops, underscoring their unique role in controlling brain function. These results offer a first demonstration that techniques from algebraic topology offer a novel perspective on structural connectomics, highlighting loop-like paths as crucial features in the human brain’s structural architecture.     

The Dynamics of Conjunctive and Disjunctive Boolean Network Models

For many biological networks, the topology of the network constrains its dynamics. In particular, feedback loops play a crucial role. The results in this paper quantify the constraints that (unsigned) feedback loops exert on the dynamics of a class of discrete models for gene regulatory networks. Conjunctive (resp. disjunctive) Boolean networks, obtained by using only the AND (resp. OR) operator, comprise a subclass of networks that consist of canalyzing functions, used to describe many published gene regulation mechanisms. For the study of feedback loops, it is common to decompose the wiring diagram into linked components each of which is strongly connected. It is shown that for conjunctive Boolean networks with strongly connected wiring diagram, the feedback loop structure completely determines the long-term dynamics of the network. A formula is established for the precise number of limit cycles of a given length, and it is determined which limit cycle lengths can appear. For general wiring diagrams, the situation is much more complicated, as feedback loops in one strongly connected component can influence the feedback loops in other components. This paper provides a sharp lower bound and an upper bound on the number of limit cycles of a given length, in terms of properties of the partially ordered set of strongly connected components.     

Adaptive Neural Network Structure Optimization Algorithm Based on Dynamic Nodes

Large-scale artificial neural networks have many redundant structures, making the network fall into the issue of local optimization and extended training time. Moreover, existing neural network topology optimization algorithms have the disadvantage of many calculations and complex network structure modeling. We propose a Dynamic Node-based neural network Structure optimization algorithm (DNS) to handle these issues. DNS consists of two steps: the generation step and the pruning step. In the generation step, the network generates hidden layers layer by layer until accuracy reaches the threshold. Then, the network uses a pruning algorithm based on Hebb’s rule or Pearson’s correlation for adaptation in the pruning step. In addition, we combine genetic algorithm to optimize DNS (GA-DNS). Experimental results show that compared with traditional neural network topology optimization algorithms, GA-DNS can generate neural networks with higher construction efficiency, lower structure complexity, and higher classification accuracy.     

Exploring the Morphospace of Communication Efficiency in Complex Networks

Graph theoretical analysis has played a key role in characterizing global features of the topology of complex networks, describing diverse systems such as protein interactions, food webs, social relations and brain connectivity. How system elements communicate with each other depends not only on the structure of the network, but also on the nature of the system''s dynamics which are constrained by the amount of knowledge and resources available for communication processes. Complementing widely used measures that capture efficiency under the assumption that communication preferentially follows shortest paths across the network (“routing”), we define analytic measures directed at characterizing network communication when signals flow in a random walk process (“diffusion”). The two dimensions of routing and diffusion efficiency define a morphospace for complex networks, with different network topologies characterized by different combinations of efficiency measures and thus occupying different regions of this space. We explore the relation of network topologies and efficiency measures by examining canonical network models, by evolving networks using a multi-objective optimization strategy, and by investigating real-world network data sets. Within the efficiency morphospace, specific aspects of network topology that differentially favor efficient communication for routing and diffusion processes are identified. Charting regions of the morphospace that are occupied by canonical, evolved or real networks allows inferences about the limits of communication efficiency imposed by connectivity and dynamics, as well as the underlying selection pressures that have shaped network topology.     

Structural properties of the Caenorhabditis elegans neuronal network

Despite recent interest in reconstructing neuronal networks, complete wiring diagrams on the level of individual synapses remain scarce and the insights into function they can provide remain unclear. Even for Caenorhabditis elegans, whose neuronal network is relatively small and stereotypical from animal to animal, published wiring diagrams are neither accurate nor complete and self-consistent. Using materials from White et al. and new electron micrographs we assemble whole, self-consistent gap junction and chemical synapse networks of hermaphrodite C. elegans. We propose a method to visualize the wiring diagram, which reflects network signal flow. We calculate statistical and topological properties of the network, such as degree distributions, synaptic multiplicities, and small-world properties, that help in understanding network signal propagation. We identify neurons that may play central roles in information processing, and network motifs that could serve as functional modules of the network. We explore propagation of neuronal activity in response to sensory or artificial stimulation using linear systems theory and find several activity patterns that could serve as substrates of previously described behaviors. Finally, we analyze the interaction between the gap junction and the chemical synapse networks. Since several statistical properties of the C. elegans network, such as multiplicity and motif distributions are similar to those found in mammalian neocortex, they likely point to general principles of neuronal networks. The wiring diagram reported here can help in understanding the mechanistic basis of behavior by generating predictions about future experiments involving genetic perturbations, laser ablations, or monitoring propagation of neuronal activity in response to stimulation.     

Using Pareto optimality to explore the topology and dynamics of the human connectome

Graph theory has provided a key mathematical framework to analyse the architecture of human brain networks. This architecture embodies an inherently complex relationship between connection topology, the spatial arrangement of network elements, and the resulting network cost and functional performance. An exploration of these interacting factors and driving forces may reveal salient network features that are critically important for shaping and constraining the brain''s topological organization and its evolvability. Several studies have pointed to an economic balance between network cost and network efficiency with networks organized in an ‘economical’ small-world favouring high communication efficiency at a low wiring cost. In this study, we define and explore a network morphospace in order to characterize different aspects of communication efficiency in human brain networks. Using a multi-objective evolutionary approach that approximates a Pareto-optimal set within the morphospace, we investigate the capacity of anatomical brain networks to evolve towards topologies that exhibit optimal information processing features while preserving network cost. This approach allows us to investigate network topologies that emerge under specific selection pressures, thus providing some insight into the selectional forces that may have shaped the network architecture of existing human brains.     

Path lengths in protein-protein interaction networks and biological complexity

We investigated the biological significance of path lengths in 12 protein-protein interaction (PPI) networks. We put forward three predictions, based on the idea that biological complexity influences path lengths. First, at the network level, path lengths are generally longer in PPIs than in random networks. Second, this pattern is more pronounced in more complex organisms. Third, within a PPI network, path lengths of individual proteins are biologically significant. We found that in 11 of the 12 species, average path lengths in PPI networks are significantly longer than those in randomly rewired networks. The PPI network of the malaria parasite Plasmodium falciparum, however, does not exhibit deviation from rewired networks. Furthermore, eukaryotic PPIs exhibit significantly greater deviation from randomly rewired networks than prokaryotic PPIs. Thus our study highlights the potentially meaningful variation in path lengths of PPI networks. Moreover, node eccentricity, defined as the longest path from a protein to others, is significantly correlated with the levels of gene expression and dispensability in the yeast PPI network. We conclude that biological complexity influences both global and local properties of path lengths in PPI networks. Investigating variation of path lengths may provide new tools to analyze the evolution of functional modules in biological systems.