Models of cortical networks with long-range patchy projections

The cortex exhibits an intricate vertical and horizontal architecture, the latter often featuring spatially clustered projection patterns, so-called patches. Many network studies of cortical dynamics ignore such spatial structures and assume purely random wiring. Here, we focus on non-random network structures provided by long-range horizontal (patchy) connections that remain inside the gray matter. We investigate how the spatial arrangement of patchy projections influences global network topology and predict its impact on the activity dynamics of the network. Since neuroanatomical data on horizontal projections is rather sparse, we suggest and compare four candidate scenarios of how patchy connections may be established. To identify a set of characteristic network properties that enables us to pin down the differences between the resulting network models, we employ the framework of stochastic graph theory. We find that patchy projections provide an exceptionally efficient way of wiring, as the resulting networks tend to exhibit small-world properties with significantly reduced wiring costs. Furthermore, the eigenvalue spectra, as well as the structure of common in- and output of the networks suggest that different spatial connectivity patterns support distinct types of activity propagation.     

Nonoptimal component placement, but short processing paths, due to long-distance projections in neural systems

It has been suggested that neural systems across several scales of organization show optimal component placement, in which any spatial rearrangement of the components would lead to an increase of total wiring. Using extensive connectivity datasets for diverse neural networks combined with spatial coordinates for network nodes, we applied an optimization algorithm to the network layouts, in order to search for wire-saving component rearrangements. We found that optimized component rearrangements could substantially reduce total wiring length in all tested neural networks. Specifically, total wiring among 95 primate (Macaque) cortical areas could be decreased by 32%, and wiring of neuronal networks in the nematode Caenorhabditis elegans could be reduced by 48% on the global level, and by 49% for neurons within frontal ganglia. Wiring length reductions were possible due to the existence of long-distance projections in neural networks. We explored the role of these projections by comparing the original networks with minimally rewired networks of the same size, which possessed only the shortest possible connections. In the minimally rewired networks, the number of processing steps along the shortest paths between components was significantly increased compared to the original networks. Additional benchmark comparisons also indicated that neural networks are more similar to network layouts that minimize the length of processing paths, rather than wiring length. These findings suggest that neural systems are not exclusively optimized for minimal global wiring, but for a variety of factors including the minimization of processing steps.     

From Caenorhabditis elegans to the human connectome: a specific modular organization increases metabolic,functional and developmental efficiency

The connectome, or the entire connectivity of a neural system represented by a network, ranges across various scales from synaptic connections between individual neurons to fibre tract connections between brain regions. Although the modularity they commonly show has been extensively studied, it is unclear whether the connection specificity of such networks can already be fully explained by the modularity alone. To answer this question, we study two networks, the neuronal network of Caenorhabditis elegans and the fibre tract network of human brains obtained through diffusion spectrum imaging. We compare them to their respective benchmark networks with varying modularities, which are generated by link swapping to have desired modularity values. We find several network properties that are specific to the neural networks and cannot be fully explained by the modularity alone. First, the clustering coefficient and the characteristic path length of both C. elegans and human connectomes are higher than those of the benchmark networks with similar modularity. High clustering coefficient indicates efficient local information distribution, and high characteristic path length suggests reduced global integration. Second, the total wiring length is smaller than for the alternative configurations with similar modularity. This is due to lower dispersion of connections, which means each neuron in the C. elegans connectome or each region of interest in the human connectome reaches fewer ganglia or cortical areas, respectively. Third, both neural networks show lower algorithmic entropy compared with the alternative arrangements. This implies that fewer genes are needed to encode for the organization of neural systems. While the first two findings show that the neural topologies are efficient in information processing, this suggests that they are also efficient from a developmental point of view. Together, these results show that neural systems are organized in such a way as to yield efficient features beyond those given by their modularity alone.     

Corticalization of two divisions of the visual system during vertebrate evolution

Data on the evolution of the visual system in vertebrate phylogeny are described. Visual projections are demonstrated in the telencephalon of cyclostomata (lampreys). The existence of a retino-thalamo-telencephalic pathway is demonstrated in elasmobranchs (skates). Two visual pathways are present in amphibians (frogs) and reptiles (turtles): retino-thalamo-telencephalic and retino-tecto-thalamo-telencephalic, and these overlap partly at the thalamic level in the lateral geniculate nucleus and completely in the telencephalon. In turtles the earliest visual and tectal impulses relay on their way to the telencephalon in the lateral geniculate body, and later impulses relay in the nucleus rotundus. In mammals (rats) visual tecto-cortical connections are seen; judging from the latent period of potentials arising in the visual cortex in response to stimulation of the superior colliculi these connections have one synaptic relay in the thalamus. The much shorter latent periods of visual evoked potentials recorded in the tectum of the monkey than in turtles (under identical chronic experimental conditions) confirm the views of morphologists on the progressive development of the tectal division of the visual system in vertebrate phylogeny. It is concluded that corticalization of both divisions of the visual system, i.e., the existence of telencephalic representation, appears in the early stages of vertebrate evolution.     

Trade-off between Multiple Constraints Enables Simultaneous Formation of Modules and Hubs in Neural Systems

The formation of the complex network architecture of neural systems is subject to multiple structural and functional constraints. Two obvious but apparently contradictory constraints are low wiring cost and high processing efficiency, characterized by short overall wiring length and a small average number of processing steps, respectively. Growing evidence shows that neural networks are results from a trade-off between physical cost and functional value of the topology. However, the relationship between these competing constraints and complex topology is not well understood quantitatively. We explored this relationship systematically by reconstructing two known neural networks, Macaque cortical connectivity and C. elegans neuronal connections, from combinatory optimization of wiring cost and processing efficiency constraints, using a control parameter , and comparing the reconstructed networks to the real networks. We found that in both neural systems, the reconstructed networks derived from the two constraints can reveal some important relations between the spatial layout of nodes and the topological connectivity, and match several properties of the real networks. The reconstructed and real networks had a similar modular organization in a broad range of , resulting from spatial clustering of network nodes. Hubs emerged due to the competition of the two constraints, and their positions were close to, and partly coincided, with the real hubs in a range of values. The degree of nodes was correlated with the density of nodes in their spatial neighborhood in both reconstructed and real networks. Generally, the rebuilt network matched a significant portion of real links, especially short-distant ones. These findings provide clear evidence to support the hypothesis of trade-off between multiple constraints on brain networks. The two constraints of wiring cost and processing efficiency, however, cannot explain all salient features in the real networks. The discrepancy suggests that there are further relevant factors that are not yet captured here.     

Random Wiring,Ganglion Cell Mosaics,and the Functional Architecture of the Visual Cortex

The architecture of iso-orientation domains in the primary visual cortex (V1) of placental carnivores and primates apparently follows species invariant quantitative laws. Dynamical optimization models assuming that neurons coordinate their stimulus preferences throughout cortical circuits linking millions of cells specifically predict these invariants. This might indicate that V1’s intrinsic connectome and its functional architecture adhere to a single optimization principle with high precision and robustness. To validate this hypothesis, it is critical to closely examine the quantitative predictions of alternative candidate theories. Random feedforward wiring within the retino-cortical pathway represents a conceptually appealing alternative to dynamical circuit optimization because random dimension-expanding projections are believed to generically exhibit computationally favorable properties for stimulus representations. Here, we ask whether the quantitative invariants of V1 architecture can be explained as a generic emergent property of random wiring. We generalize and examine the stochastic wiring model proposed by Ringach and coworkers, in which iso-orientation domains in the visual cortex arise through random feedforward connections between semi-regular mosaics of retinal ganglion cells (RGCs) and visual cortical neurons. We derive closed-form expressions for cortical receptive fields and domain layouts predicted by the model for perfectly hexagonal RGC mosaics. Including spatial disorder in the RGC positions considerably changes the domain layout properties as a function of disorder parameters such as position scatter and its correlations across the retina. However, independent of parameter choice, we find that the model predictions substantially deviate from the layout laws of iso-orientation domains observed experimentally. Considering random wiring with the currently most realistic model of RGC mosaic layouts, a pairwise interacting point process, the predicted layouts remain distinct from experimental observations and resemble Gaussian random fields. We conclude that V1 layout invariants are specific quantitative signatures of visual cortical optimization, which cannot be explained by generic random feedforward-wiring models.     

The wiring economy principle: connectivity determines anatomy in the human brain

Minimization of the wiring cost of white matter fibers in the human brain appears to be an organizational principle. We investigate this aspect in the human brain using whole brain connectivity networks extracted from high resolution diffusion MRI data of 14 normal volunteers. We specifically address the question of whether brain anatomy determines its connectivity or vice versa. Unlike previous studies we use weighted networks, where connections between cortical nodes are real-valued rather than binary off-on connections. In one set of analyses we found that the connectivity structure of the brain has near optimal wiring cost compared to random networks with the same number of edges, degree distribution and edge weight distribution. A specifically designed minimization routine could not find cheaper wiring without significantly degrading network performance. In another set of analyses we kept the observed brain network topology and connectivity but allowed nodes to freely move on a 3D manifold topologically identical to the brain. An efficient minimization routine was written to find the lowest wiring cost configuration. We found that beginning from any random configuration, the nodes invariably arrange themselves in a configuration with a striking resemblance to the brain. This confirms the widely held but poorly tested claim that wiring economy is a driving principle of the brain. Intriguingly, our results also suggest that the brain mainly optimizes for the most desirable network connectivity, and the observed brain anatomy is merely a result of this optimization.     

Specificity and randomness: structure-function relationships in neural circuits

A fundamental but unsolved problem in neuroscience is how connections between neurons might underlie information processing in central circuits. Building wiring diagrams of neural networks may accelerate our understanding of how they compute. But even if we had wiring diagrams, it is critical to know what neurons in a circuit are doing: their physiology. In both the retina and cerebral cortex, a great deal is known about topographic specificity, such as lamination and cell-type specificity of connections. Little, however, is known about connections as they relate to function. Here, we review how advances in functional imaging and electron microscopy have recently allowed the examination of relationships between sensory physiology and synaptic connections in cortical and retinal circuits.     

Rule extraction from minimal neural networks for credit card screening

While feedforward neural networks have been widely accepted as effective tools for solving classification problems, the issue of finding the best network architecture remains unresolved, particularly so in real-world problem settings. We address this issue in the context of credit card screening, where it is important to not only find a neural network with good predictive performance but also one that facilitates a clear explanation of how it produces its predictions. We show that minimal neural networks with as few as one hidden unit provide good predictive accuracy, while having the added advantage of making it easier to generate concise and comprehensible classification rules for the user. To further reduce model size, a novel approach is suggested in which network connections from the input units to this hidden unit are removed by a very straightaway pruning procedure. In terms of predictive accuracy, both the minimized neural networks and the rule sets generated from them are shown to compare favorably with other neural network based classifiers. The rules generated from the minimized neural networks are concise and thus easier to validate in a real-life setting.     

The trappist's approach to pathfinding: elucidating brain wiring using secretory-trap mutagenesis

A key problem in using genetics to dissect the wiring of the mammalian brain lies in discovering which of the billions of neural connections have been disrupted by a particular mutation. A novel gene-trap approach targets the genes involved in brain wiring and labels the axons of neurons expressing those genes, enabling the effects of mutations to be observed directly.     

The information theory of developmental pruning: Optimizing global network architectures using local synaptic rules

During development, biological neural networks produce more synapses and neurons than needed. Many of these synapses and neurons are later removed in a process known as neural pruning. Why networks should initially be over-populated, and the processes that determine which synapses and neurons are ultimately pruned, remains unclear. We study the mechanisms and significance of neural pruning in model neural networks. In a deep Boltzmann machine model of sensory encoding, we find that (1) synaptic pruning is necessary to learn efficient network architectures that retain computationally-relevant connections, (2) pruning by synaptic weight alone does not optimize network size and (3) pruning based on a locally-available measure of importance based on Fisher information allows the network to identify structurally important vs. unimportant connections and neurons. This locally-available measure of importance has a biological interpretation in terms of the correlations between presynaptic and postsynaptic neurons, and implies an efficient activity-driven pruning rule. Overall, we show how local activity-dependent synaptic pruning can solve the global problem of optimizing a network architecture. We relate these findings to biology as follows: (I) Synaptic over-production is necessary for activity-dependent connectivity optimization. (II) In networks that have more neurons than needed, cells compete for activity, and only the most important and selective neurons are retained. (III) Cells may also be pruned due to a loss of synapses on their axons. This occurs when the information they convey is not relevant to the target population.     

Variability v.s. synchronicity of neuronal activity in local cortical network models with different wiring topologies

Dynamical behavior of a biological neuronal network depends significantly on the spatial pattern of synaptic connections among neurons. While neuronal network dynamics has extensively been studied with simple wiring patterns, such as all-to-all or random synaptic connections, not much is known about the activity of networks with more complicated wiring topologies. Here, we examined how different wiring topologies may influence the response properties of neuronal networks, paying attention to irregular spike firing, which is known as a characteristic of in vivo cortical neurons, and spike synchronicity. We constructed a recurrent network model of realistic neurons and systematically rewired the recurrent synapses to change the network topology, from a localized regular and a “small-world” network topology to a distributed random network topology. Regular and small-world wiring patterns greatly increased the irregularity or the coefficient of variation (Cv) of output spike trains, whereas such an increase was small in random connectivity patterns. For given strength of recurrent synapses, the firing irregularity exhibited monotonous decreases from the regular to the random network topology. By contrast, the spike coherence between an arbitrary neuron pair exhibited a non-monotonous dependence on the topological wiring pattern. More precisely, the wiring pattern to maximize the spike coherence varied with the strength of recurrent synapses. In a certain range of the synaptic strength, the spike coherence was maximal in the small-world network topology, and the long-range connections introduced in this wiring changed the dependence of spike synchrony on the synaptic strength moderately. However, the effects of this network topology were not really special in other properties of network activity. Action Editor: Xiao-Jing Wang     

Multiple roles of ephrins during the formation of thalamocortical projections: maps and more

The functional architecture of the cerebral cortex is based on intrinsic connections that precisely link neurons from distinct cortical laminae as well as layer-specific afferent and efferent projections. Experimental strategies using in vitro assays originally developed by Friedrich Bonhoeffer have suggested that positional cues confined to individual layers regulate the assembly of local cortical circuits and the formation of thalamocortical projections. One of these wiring molecules is ephrinA5, a ligand for Eph receptor tyrosine kinases. EphrinA5 and Eph receptors exhibit highly dynamic expression patterns in distinct regions of the cortex and thalamus during early and late stages of thalamocortical and cortical circuit formation. In vitro assays suggest that ephrinA5 is a multifunctional wiring molecule for different populations of cortical and thalamic axons. Additionally, the expression patterns of ephrinA5 during cortical development are consistent with this molecule regulating, in alternative ways, specific components of thalamic and cortical connectivity. To test this directly, the organization of thalamocortical projections was examined in mice lacking ephrinA5 gene expression. The anatomical studies in ephrinA5 knockout animals revealed a miswiring of limbic thalamic projections and changes in neocortical circuits that were predicted from the expression pattern and the in vitro analysis of ephrinA5 function.     

Implementing Gaussian process inference with neural networks

Gaussian processes compare favourably with backpropagation neural networks as a tool for regression, and Bayesian neural networks have Gaussian process behaviour when the number of hidden neurons tends to infinity. We describe a simple recurrent neural network with connection weights trained by one-shot Hebbian learning. This network amounts to a dynamical system which relaxes to a stable state in which it generates predictions identical to those of Gaussian process regression. In effect an infinite number of hidden units in a feed-forward architecture can be replaced by a merely finite number, together with recurrent connections.     

The relative efficiency of modular and non-modular networks of different size

Most biological networks are modular but previous work with small model networks has indicated that modularity does not necessarily lead to increased functional efficiency. Most biological networks are large, however, and here we examine the relative functional efficiency of modular and non-modular neural networks at a range of sizes. We conduct a detailed analysis of efficiency in networks of two size classes: ‘small’ and ‘large’, and a less detailed analysis across a range of network sizes. The former analysis reveals that while the modular network is less efficient than one of the two non-modular networks considered when networks are small, it is usually equally or more efficient than both non-modular networks when networks are large. The latter analysis shows that in networks of small to intermediate size, modular networks are much more efficient that non-modular networks of the same (low) connective density. If connective density must be kept low to reduce energy needs for example, this could promote modularity. We have shown how relative functionality/performance scales with network size, but the precise nature of evolutionary relationship between network size and prevalence of modularity will depend on the costs of connectivity.     

Flowers and weeds: cell-type specific pruning in the developing visual thalamus

In the first weeks of vertebrate postnatal life, neural networks in the visual thalamus undergo activity-dependent refinement thought to be important for the development of functional vision. This process involves pruning of synaptic connections between retinal ganglion cells and excitatory thalamic neurons that relay signals on to visual areas of the cortex. A recent report in Neural Development shows that this does not occur in inhibitory neurons, questioning our current understanding of the development of mature neural circuits. See research article: http://www.neuraldevelopment.com/content/8/1/24     

Sensory experience and the formation of a computational map of auditory space in the brain.

The basic wiring of the brain is first established before birth by using a variety of molecular guidance cues. These connections are then refined by patterns of neural activity, which are initially generated spontaneously and subsequently driven by sensory experience. In the superior colliculus, a midbrain nucleus involved in the control of orienting behaviour, visual, auditory, and tactile inputs converge to form superimposed maps of sensory space. Maps of visual space and of the body surface arise from spatially ordered projections from the retina and skin, respectively. In contrast, the map of auditory space is computed within the brain by tuning the neurons to different localization cues that result from the acoustical properties of the head and ears. Establishing and maintaining the registration of the maps in the face of individual differences in the size and relative positions of different sense organs is an activity-dependent process in which the synaptic circuits underlying the auditory representation are modified and calibrated under the influence of both auditory and visual experience. BioEssays 1999;21:900-911.     

Altered Anatomical Network in Early Blindness Revealed by Diffusion Tensor Tractography

The topological architecture of the cerebral anatomical network reflects the structural organization of the human brain. Recently, topological measures based on graph theory have provided new approaches for quantifying large-scale anatomical networks. Diffusion MRI studies have revealed the efficient small-world properties and modular structure of the anatomical network in normal subjects. However, no previous study has used diffusion MRI to reveal changes in the brain anatomical network in early blindness. Here, we utilized diffusion tensor imaging to construct binary anatomical networks for 17 early blind subjects and 17 age- and gender-matched sighted controls. We established the existence of structural connections between any pair of the 90 cortical and sub-cortical regions using deterministic tractography. Compared with controls, early blind subjects showed a decreased degree of connectivity, a reduced global efficiency, and an increased characteristic path length in their brain anatomical network, especially in the visual cortex. Moreover, we revealed some regions with motor or somatosensory function have increased connections with other brain regions in the early blind, which suggested experience-dependent compensatory plasticity. This study is the first to show alterations in the topological properties of the anatomical network in early blindness. From the results, we suggest that analyzing the brain''s anatomical network obtained using diffusion MRI data provides new insights into the understanding of the brain''s re-organization in the specific population with early visual deprivation.     

Experience-consistent modeling for radial basis function neural networks

We develop a new approach to the design of neural networks, which utilizes a collaborative framework of knowledge-driven experience. In contrast to the "standard" way of developing neural networks, which explicitly exploits experimental data, this approach incorporates a mechanism of knowledge-driven experience. The essence of the proposed scheme of learning is to take advantage of the parameters (connections) of neural networks built in the past for the same phenomenon (which might also exhibit some variability over time or space) for which are interested to construct the network on a basis of currently available data. We establish a conceptual and algorithmic framework to reconcile these two essential sources of information (data and knowledge) in the process of the development of the network. To make a presentation more focused and come up with a detailed quantification of the resulting architecture, we concentrate on the experience-based design of radial basis function neural networks (RBFNNs). We introduce several performance indexes to quantify an effect of utilization of the knowledge residing within the connections of the networks and establish an optimal level of their use. Experimental results are presented for low-dimensional synthetic data and selected datasets available at the Machine Learning Repository.     

Mapping early brain development in autism

Although the neurobiology of autism has been studied for more than two decades, the majority of these studies have examined brain structure 10, 20, or more years after the onset of clinical symptoms. The pathological biology that causes autism remains unknown, but its signature is likely to be most evident during the first years of life when clinical symptoms are emerging. This review highlights neurobiological findings during the first years of life and emphasizes early brain overgrowth as a key factor in the pathobiology of autism. We speculate that excess neuron numbers may be one possible cause of early brain overgrowth and produce defects in neural patterning and wiring, with exuberant local and short-distance cortical interactions impeding the function of large-scale, long-distance interactions between brain regions. Because large-scale networks underlie socio-emotional and communication functions, such alterations in brain architecture could relate to the early clinical manifestations of autism. As such, autism may additionally provide unique insight into genetic and developmental processes that shape early neural wiring patterns and make possible higher-order social, emotional, and communication functions.