Probabilistic computation by neuromine networks

In this paper, we address the question, can biologically feasible neural nets compute more than can be computed by deterministic polynomial time algorithms? Since we want to maintain a claim of plausibility and reasonableness we restrict ourselves to algorithmically easy to construct nets and we rule out infinite precision in parameters and in any analog parts of the computation. Our approach is to consider the recent advances in randomized algorithms and see if such randomized computations can be described by neural nets. We start with a pair of neurons and show that by connecting them with reciprocal inhibition and some tonic input, then the steady-state will be one neuron ON and one neuron OFF, but which neuron will be ON and which neuron will be OFF will be chosen at random (perhaps, it would be better to say that microscopic noise in the analog computation will be turned into a megascale random bit). We then show that we can build a small network that uses this random bit process to generate repeatedly random bits. This random bit generator can then be connected with a neural net representing the deterministic part of randomized algorithm. We, therefore, demonstrate that these neural nets can carry out probabilistic computation and thus be less limited than classical neural nets.     

Connectomic constraints on computation in feedforward networks of spiking neurons

Several efforts are currently underway to decipher the connectome or parts thereof in a variety of organisms. Ascertaining the detailed physiological properties of all the neurons in these connectomes, however, is out of the scope of such projects. It is therefore unclear to what extent knowledge of the connectome alone will advance a mechanistic understanding of computation occurring in these neural circuits, especially when the high-level function of the said circuit is unknown. We consider, here, the question of how the wiring diagram of neurons imposes constraints on what neural circuits can compute, when we cannot assume detailed information on the physiological response properties of the neurons. We call such constraints—that arise by virtue of the connectome—connectomic constraints on computation. For feedforward networks equipped with neurons that obey a deterministic spiking neuron model which satisfies a small number of properties, we ask if just by knowing the architecture of a network, we can rule out computations that it could be doing, no matter what response properties each of its neurons may have. We show results of this form, for certain classes of network architectures. On the other hand, we also prove that with the limited set of properties assumed for our model neurons, there are fundamental limits to the constraints imposed by network structure. Thus, our theory suggests that while connectomic constraints might restrict the computational ability of certain classes of network architectures, we may require more elaborate information on the properties of neurons in the network, before we can discern such results for other classes of networks.     

Multiple-color optical activation, silencing, and desynchronization of neural activity, with single-spike temporal resolution

The quest to determine how precise neural activity patterns mediate computation, behavior, and pathology would be greatly aided by a set of tools for reliably activating and inactivating genetically targeted neurons, in a temporally precise and rapidly reversible fashion. Having earlier adapted a light-activated cation channel, channelrhodopsin-2 (ChR2), for allowing neurons to be stimulated by blue light, we searched for a complementary tool that would enable optical neuronal inhibition, driven by light of a second color. Here we report that targeting the codon-optimized form of the light-driven chloride pump halorhodopsin from the archaebacterium Natronomas pharaonis (hereafter abbreviated Halo) to genetically-specified neurons enables them to be silenced reliably, and reversibly, by millisecond-timescale pulses of yellow light. We show that trains of yellow and blue light pulses can drive high-fidelity sequences of hyperpolarizations and depolarizations in neurons simultaneously expressing yellow light-driven Halo and blue light-driven ChR2, allowing for the first time manipulations of neural synchrony without perturbation of other parameters such as spiking rates. The Halo/ChR2 system thus constitutes a powerful toolbox for multichannel photoinhibition and photostimulation of virally or transgenically targeted neural circuits without need for exogenous chemicals, enabling systematic analysis and engineering of the brain, and quantitative bioengineering of excitable cells.     

Preferential control of basal dendritic protrusions by EphB2

The flow of information between neurons in many neural circuits is controlled by a highly specialized site of cell-cell contact known as a synapse. A number of molecules have been identified that are involved in central nervous system synapse development, but knowledge is limited regarding whether these cues direct organization of specific synapse types or on particular regions of individual neurons. Glutamate is the primary excitatory neurotransmitter in the brain, and the majority of glutamatergic synapses occur on mushroom-shaped protrusions called dendritic spines. Changes in the morphology of these structures are associated with long-lasting modulation of synaptic strength thought to underlie learning and memory, and can be abnormal in neuropsychiatric disease. Here, we use rat cortical slice cultures to examine how a previously-described synaptogenic molecule, the EphB2 receptor tyrosine kinase, regulates dendritic protrusion morphology in specific regions of the dendritic arbor in cortical pyramidal neurons. We find that alterations in EphB2 signaling can bidirectionally control protrusion length, and knockdown of EphB2 expression levels reduces the number of dendritic spines and filopodia. Expression of wild-type or dominant negative EphB2 reveals that EphB2 preferentially regulates dendritic protrusion structure in basal dendrites. Our findings suggest that EphB2 may act to specify synapse formation in a particular subcellular region of cortical pyramidal neurons.     

Scalability of Asynchronous Networks Is Limited by One-to-One Mapping between Effective Connectivity and Correlations

Network models are routinely downscaled compared to nature in terms of numbers of nodes or edges because of a lack of computational resources, often without explicit mention of the limitations this entails. While reliable methods have long existed to adjust parameters such that the first-order statistics of network dynamics are conserved, here we show that limitations already arise if also second-order statistics are to be maintained. The temporal structure of pairwise averaged correlations in the activity of recurrent networks is determined by the effective population-level connectivity. We first show that in general the converse is also true and explicitly mention degenerate cases when this one-to-one relationship does not hold. The one-to-one correspondence between effective connectivity and the temporal structure of pairwise averaged correlations implies that network scalings should preserve the effective connectivity if pairwise averaged correlations are to be held constant. Changes in effective connectivity can even push a network from a linearly stable to an unstable, oscillatory regime and vice versa. On this basis, we derive conditions for the preservation of both mean population-averaged activities and pairwise averaged correlations under a change in numbers of neurons or synapses in the asynchronous regime typical of cortical networks. We find that mean activities and correlation structure can be maintained by an appropriate scaling of the synaptic weights, but only over a range of numbers of synapses that is limited by the variance of external inputs to the network. Our results therefore show that the reducibility of asynchronous networks is fundamentally limited.     

Exercise‐linked improvement in age‐associated loss of balance is associated with increased vestibular input to motor neurons

Age‐associated loss of muscle function is exacerbated by a concomitant reduction in balance, leading to gait abnormalities and falls. Even though balance defects can be mitigated by exercise, the underlying neural mechanisms are unknown. We now have investigated components of the proprioceptive and vestibular systems in specific motor neuron pools in sedentary and trained old mice, respectively. We observed a strong age‐linked deterioration in both circuits, with a mitigating effect of exercise on vestibular synapse numbers on motor neurons, closely associated with an improvement in gait and balance in old mice. Our results thus describe how the proprioceptive and vestibular systems are modulated by age and exercise, and how these changes affect their input to motor neurons. These findings not only make a strong case for exercise‐based interventions in elderly individuals to improve balance, but could also lead to targeted therapeutic interventions aimed at the respective neuronal circuitry.     

Memory Capacity of Networks with Stochastic Binary Synapses

In standard attractor neural network models, specific patterns of activity are stored in the synaptic matrix, so that they become fixed point attractors of the network dynamics. The storage capacity of such networks has been quantified in two ways: the maximal number of patterns that can be stored, and the stored information measured in bits per synapse. In this paper, we compute both quantities in fully connected networks of N binary neurons with binary synapses, storing patterns with coding level , in the large and sparse coding limits (). We also derive finite-size corrections that accurately reproduce the results of simulations in networks of tens of thousands of neurons. These methods are applied to three different scenarios: (1) the classic Willshaw model, (2) networks with stochastic learning in which patterns are shown only once (one shot learning), (3) networks with stochastic learning in which patterns are shown multiple times. The storage capacities are optimized over network parameters, which allows us to compare the performance of the different models. We show that finite-size effects strongly reduce the capacity, even for networks of realistic sizes. We discuss the implications of these results for memory storage in the hippocampus and cerebral cortex.     

Neuron-astrocyte signaling in the development and plasticity of neural circuits

Emerging evidence indicates that signaling between perisynaptic astrocytes and neurons at the tripartite synapse plays an important role during the critical period when neural circuits are formed and refined. Cross-talk between astrocytes and neurons during development mediates synaptogenesis, synapse elimination and structural plasticity through a variety of secreted and contact-dependent signals. Recent live imaging studies reveal a dynamic and cooperative interplay between astrocytes and neurons at synapses that is guided by a variety of molecular cues. A unifying theme from these recent findings is that astrocytes can promote the development and plasticity of synaptic circuits. Insight into the molecular mechanisms by which astrocytes regulate the wiring of the brain during development could lead to new therapeutic strategies to promote repair and rewiring of neural circuits in the mature brain following CNS injury and neurodegenerative disease.     

Pair recordings reveal all-silent synaptic connections and the postsynaptic expression of long-term potentiation

The activation of silent synapses is a proposed mechanism to account for rapid increases in synaptic efficacy such as long-term potentiation (LTP). Using simultaneous recordings from individual pre- and postsynaptic neurons in organotypic hippocampal slices, we show that two CA3 neurons can be connected entirely by silent synapses. Increasing release probability or application of cyclothiazide does not produce responses from these silent synapses. Direct measurement of NMDAR-mediated postsynaptic responses in all-silent synaptic connections before and after LTP induction show no change in failure rate, amplitude, or area. These data do not support hypotheses that synapse silent results from presynaptic factors or that LTP results from increases in presynaptic glutamate release. LTP is also associated with an increase in postsynaptic responsiveness to exogenous AMPA. We conclude that synapse silence, activation, and expression of LTP are postsynaptic.     

Bifurcations of lurching waves in a thalamic neuronal network

We consider a two-layer, one-dimensional lattice of neurons; one layer consists of excitatory thalamocortical neurons, while the other is comprised of inhibitory reticular thalamic neurons. Such networks are known to support “lurching” waves, for which propagation does not appear smooth, but rather progresses in a saltatory fashion; these waves can be characterized by different spatial widths (different numbers of neurons active at the same time). We show that these lurching waves are fixed points of appropriately defined Poincaré maps, and follow these fixed points as parameters are varied. In this way, we are able to explain observed transitions in behavior, and, in particular, to show how branches with different spatial widths are linked with each other. Our computer-assisted analysis is quite general and could be applied to other spatially extended systems which exhibit this non-trivial form of wave propagation.     

When shared concept cells support associations: Theory of overlapping memory engrams

Assemblies of neurons, called concepts cells, encode acquired concepts in human Medial Temporal Lobe. Those concept cells that are shared between two assemblies have been hypothesized to encode associations between concepts. Here we test this hypothesis in a computational model of attractor neural networks. We find that for concepts encoded in sparse neural assemblies there is a minimal fraction c_min of neurons shared between assemblies below which associations cannot be reliably implemented; and a maximal fraction c_max of shared neurons above which single concepts can no longer be retrieved. In the presence of a periodically modulated background signal, such as hippocampal oscillations, recall takes the form of association chains reminiscent of those postulated by theories of free recall of words. Predictions of an iterative overlap-generating model match experimental data on the number of concepts to which a neuron responds.     

Spatiotemporal Computations of an Excitable and Plastic Brain: Neuronal Plasticity Leads to Noise-Robust and Noise-Constructive Computations

It is a long-established fact that neuronal plasticity occupies the central role in generating neural function and computation. Nevertheless, no unifying account exists of how neurons in a recurrent cortical network learn to compute on temporally and spatially extended stimuli. However, these stimuli constitute the norm, rather than the exception, of the brain''s input. Here, we introduce a geometric theory of learning spatiotemporal computations through neuronal plasticity. To that end, we rigorously formulate the problem of neural representations as a relation in space between stimulus-induced neural activity and the asymptotic dynamics of excitable cortical networks. Backed up by computer simulations and numerical analysis, we show that two canonical and widely spread forms of neuronal plasticity, that is, spike-timing-dependent synaptic plasticity and intrinsic plasticity, are both necessary for creating neural representations, such that these computations become realizable. Interestingly, the effects of these forms of plasticity on the emerging neural code relate to properties necessary for both combating and utilizing noise. The neural dynamics also exhibits features of the most likely stimulus in the network''s spontaneous activity. These properties of the spatiotemporal neural code resulting from plasticity, having their grounding in nature, further consolidate the biological relevance of our findings.     

The Green’s function formalism as a bridge between single- and multi-compartmental modeling

Neurons are spatially extended structures that receive and process inputs on their dendrites. It is generally accepted that neuronal computations arise from the active integration of synaptic inputs along a dendrite between the input location and the location of spike generation in the axon initial segment. However, many application such as simulations of brain networks use point-neurons—neurons without a morphological component—as computational units to keep the conceptual complexity and computational costs low. Inevitably, these applications thus omit a fundamental property of neuronal computation. In this work, we present an approach to model an artificial synapse that mimics dendritic processing without the need to explicitly simulate dendritic dynamics. The model synapse employs an analytic solution for the cable equation to compute the neuron’s membrane potential following dendritic inputs. Green’s function formalism is used to derive the closed version of the cable equation. We show that by using this synapse model, point-neurons can achieve results that were previously limited to the realms of multi-compartmental models. Moreover, a computational advantage is achieved when only a small number of simulated synapses impinge on a morphologically elaborate neuron. Opportunities and limitations are discussed.     

Neural dynamics as sampling: a model for stochastic computation in recurrent networks of spiking neurons

The organization of computations in networks of spiking neurons in the brain is still largely unknown, in particular in view of the inherently stochastic features of their firing activity and the experimentally observed trial-to-trial variability of neural systems in the brain. In principle there exists a powerful computational framework for stochastic computations, probabilistic inference by sampling, which can explain a large number of macroscopic experimental data in neuroscience and cognitive science. But it has turned out to be surprisingly difficult to create a link between these abstract models for stochastic computations and more detailed models of the dynamics of networks of spiking neurons. Here we create such a link and show that under some conditions the stochastic firing activity of networks of spiking neurons can be interpreted as probabilistic inference via Markov chain Monte Carlo (MCMC) sampling. Since common methods for MCMC sampling in distributed systems, such as Gibbs sampling, are inconsistent with the dynamics of spiking neurons, we introduce a different approach based on non-reversible Markov chains that is able to reflect inherent temporal processes of spiking neuronal activity through a suitable choice of random variables. We propose a neural network model and show by a rigorous theoretical analysis that its neural activity implements MCMC sampling of a given distribution, both for the case of discrete and continuous time. This provides a step towards closing the gap between abstract functional models of cortical computation and more detailed models of networks of spiking neurons.     

Computing with neural synchrony

Neurons communicate primarily with spikes, but most theories of neural computation are based on firing rates. Yet, many experimental observations suggest that the temporal coordination of spikes plays a role in sensory processing. Among potential spike-based codes, synchrony appears as a good candidate because neural firing and plasticity are sensitive to fine input correlations. However, it is unclear what role synchrony may play in neural computation, and what functional advantage it may provide. With a theoretical approach, I show that the computational interest of neural synchrony appears when neurons have heterogeneous properties. In this context, the relationship between stimuli and neural synchrony is captured by the concept of synchrony receptive field, the set of stimuli which induce synchronous responses in a group of neurons. In a heterogeneous neural population, it appears that synchrony patterns represent structure or sensory invariants in stimuli, which can then be detected by postsynaptic neurons. The required neural circuitry can spontaneously emerge with spike-timing-dependent plasticity. Using examples in different sensory modalities, I show that this allows simple neural circuits to extract relevant information from realistic sensory stimuli, for example to identify a fluctuating odor in the presence of distractors. This theory of synchrony-based computation shows that relative spike timing may indeed have computational relevance, and suggests new types of neural network models for sensory processing with appealing computational properties.     

Oscillations and oscillatory behavior in small neural circuits

In order to determine the dynamical properties of central pattern generators (CPGs), we have examined the lobster stomatogastric ganglion using the tools of nonlinear dynamics. The lobster pyloric and gastric mill central pattern generators can be analyzed at both the cellular and network levels because they are small, i.e., contain only 25 neurons between them and each neuron and synapse are repeatedly identifiable from animal to animal. We discuss how the biophysical properties of each neuron and synapse in the two circuits act cooperatively to generate two different patterns of sequential activity, how these patterns are altered by neuromodulators and perturbed by noise and sensory inputs. Finally, we show how simplified Hindmarsh–Rose models can be made into analog electronic neurons that mimic the lobster neurons and in addition be incorporated into artificial CPGs with robotic applications.     

Robustness of neuroprosthetic decoding algorithms

We assessed the ability of two algorithms to predict hand kinematics from neural activity as a function of the amount of data used to determine the algorithm parameters. Using chronically implanted intracortical arrays, single- and multineuron discharge was recorded during trained step tracking and slow continuous tracking tasks in macaque monkeys. The effect of increasing the amount of data used to build a neural decoding model on the ability of that model to predict hand kinematics accurately was examined. We evaluated how well a maximum-likelihood model classified discrete reaching directions and how well a linear filter model reconstructed continuous hand positions over time within and across days. For each of these two models we asked two questions: (1) How does classification performance change as the amount of data the model is built upon increases? (2) How does varying the time interval between the data used to build the model and the data used to test the model affect reconstruction? Less than 1 min of data for the discrete task (8 to 13 neurons) and less than 3 min (8 to 18 neurons) for the continuous task were required to build optimal models. Optimal performance was defined by a cost function we derived that reflects both the ability of the model to predict kinematics accurately and the cost of taking more time to build such models. For both the maximum-likelihood classifier and the linear filter model, increasing the duration between the time of building and testing the model within a day did not cause any significant trend of degradation or improvement in performance. Linear filters built on one day and tested on neural data on a subsequent day generated error-measure distributions that were not significantly different from those generated when the linear filters were tested on neural data from the initial day (p<0.05, Kolmogorov-Smirnov test). These data show that only a small amount of data from a limited number of cortical neurons appears to be necessary to construct robust models to predict kinematic parameters for the subsequent hours. Motor-control signals derived from neurons in motor cortex can be reliably acquired for use in neural prosthetic devices. Adequate decoding models can be built rapidly from small numbers of cells and maintained with daily calibration sessions.     

Energy coding in neural network with inhibitory neurons

This paper aimed at assessing and comparing the effects of the inhibitory neurons in the neural network on the neural energy distribution, and the network activities in the absence of the inhibitory neurons to understand the nature of neural energy distribution and neural energy coding. Stimulus, synchronous oscillation has significant difference between neural networks with and without inhibitory neurons, and this difference can be quantitatively evaluated by the characteristic energy distribution. In addition, the synchronous oscillation difference of the neural activity can be quantitatively described by change of the energy distribution if the network parameters are gradually adjusted. Compared with traditional method of correlation coefficient analysis, the quantitative indicators based on nervous energy distribution characteristics are more effective in reflecting the dynamic features of the neural network activities. Meanwhile, this neural coding method from a global perspective of neural activity effectively avoids the current defects of neural encoding and decoding theory and enormous difficulties encountered. Our studies have shown that neural energy coding is a new coding theory with high efficiency and great potential.