Formation of topographic maps and columnar microstructures in nerve fields

Topographic connections are found in many parts of the vertebrate nervous systems, known for example as retinotopy. The self-organizing ability of Hebb type modifiable synapses plays an important role in forming, at least in refining, the topographic connections. We present a mathematical analysis of a revised version of the Willshaw-Malburg model of topographic formation, solving the equations of synaptic self-organization coupled with the field equation of neural excitations. The equilibrium solutions are obtained and their stability is studied. It is proved that two cases exist depending on parameters. In one case, the smooth topographic organization is obtained as a stable equilibrium of the equations. In the other case, this solution becomes unstable, and instead the topographic organization with columnar microstructures appears. This might explain the columnar structures in the cerebrum. The theory is confirmed by computer simulated experiments.     

A neural network model for the mechanism of feature-extraction

We propose a new multilayered neural network model which has the ability of rapid self-organization. This model is a modified version of the cognitron (Fukushima, 1975). It has modifiable inhibitory feedback connections, as well as conventional modifiable excitatory feedforward connections, between the cells of adjoining layers. If a feature-extracting cell in the network is excited by a stimulus which is already familiar to the network, the cell immediately feeds back inhibitory signals to its presynaptic cells in the preceding layer, which suppresses their response. On the other hand, the feature-extracting cell does not respond to an unfamiliar feature, and the responses from its presynaptic cells are therefore not suppressed because they do not receive any feedback inhibition. Modifiable synapses in the new network are reinforced in a way similar to those in the cognitron, and synaptic connections from cells yielding a large sustained output are reinforced. Since familiar stimulus features do not elicit a sustained response from the cells of the network, only circuits which detect novel stimulus features develop. The network therefore quickly acquires favorable pattern-selectivity by the mere repetitive presentation of set of learning patterns.     

Post-tetanic modification of the efficacy of excitatory transmission in neuronal networks with interhemispheric connections

The modifiable reciprocal transcallosal monosynaptic excitatory connections were for the first time detected in vivo experiments in rat motor cortex using multiunit recording and crosscorrelation analysis, It was shown that high-frequency microstimulation (MCS) of a small group of cortical cells of one hemisphere produces long-term changes in the efficacy of transcallosal excitatory connections, and also ipsilateral connections in both hemispheres. The posttetanic changes appear as long-term potentiation (LTP) and long-term depression (LTD). The bursting neurons were found to have more favorable conditions for the induction of LTP of most converging inputs (in contrast to cells with other discharge patterns). Both LTP and LTD could be simultaneously induced in synapses formed by axon collaterals of a callosal cell on several neurons. LTP and LTD could be simultaneously obtained at diverse synapses of the same cell. The number of spontaneously active callosal neurons as well as the number and efficacy of transcallosal connections increased after the MCS, whereas the number and efficacy of ipsilateral connections decreased. Basing on these data we assume that the ipsilateral inhibition is more effective than the transcallosal inhibition. MCS results in the modification of the pattern of initially existing connections between numerous neurons of an ensemble including cells of both hemispheres.     

A self-organizing neural network model for the development of complex cells

On the basis of recent neurophysiological findings on the mammalian visual cortex, a selforganizing neural network model is proposed for the understanding of the development of complex cells. The model is composed of two kinds of connections from LGN cells to a complex cell. One is direct excitatory connections and the other is indirect inhibitory connections via simple cells. Inhibitory synapses between simple cells and complex cells are assumed to be modifiable. The model was simulated on a computer to confirm its behavior.     

A neural network model for the development of direction selectivity in the visual cortex

A neural network model is proposed to explain the development of direction selectivity of cortical cells. The model is constructed under the following three hypotheses that are very plausible from recent neurophysiological findings. (1) Direction selectivity is developed by modifiable inhibitory synapses. (2) It results not from the direct convergence of many excitatory inputs from LGN cells but from cortical neural networks. (3) Direction-selective mechanism is independent of orientation-selective mechanism.—The model was simulated on a computer for a few kinds of inhibitory connections and initial conditions. The results were consistent with neurophysiological facts not only for normal cats but for cats reared in an abnormal visual environment.     

A template matching model for pattern recognition: Self-organization of templates and template matching by a disinhibitory neural network

A template matching model for pattern recognition is proposed. By following a previouslyproposed algorithm for synaptic modification (Hirai, 1980), the template of a stimulus pattern is selforganized as a spatial distribution pattern of matured synapses on the cells receiving modifiable synapses. Template matching is performed by the disinhibitory neural network cascaded beyond the neural layer composed of the cells receiving the modifiable synapses. The performance of the model has been simulated on a digital computer. After repetitive presentations of a stimulus pattern, a cell receiving the modifiable synapses comes to have the template of that pattern. And the cell in the latter layer of the disinhibitory bitory neural network that receives the disinhibitory input from that cell becomes electively sensitive to that pattern. Learning patterns are not restricted by previously learned ones. They can be subset or superset patterns of the ones previously learned. If an unknown pattern is presented to the model, no cell beyond the disinhibitory neural network will respond. However, if previously learned patterns are embedded in that pattern, the cells which have the templates of those patterns respond and are assumed to transmit the information to higher center. The computer simulation also shows that the model can organize a clean template under a noisy environment.     

Modeling of Rhythmogenesis in an Ensemble of Cortical Neurons Connected with Electrical Synapses

Based on our own data on generation of spindle-like field electrical activity in neuronal barrels of the rat somatic cortex and also on the published data on the properties of voltage-dependent channels in the membranes of cortical cells, we developed a model of the ensemble (simple network) of neurons connected by electrical synapses. Such connections were found earlier in neurophysiological and ultramicroscopic studies. Model neurons with membranes having sodium, potassium, and calcium channels described in the literature were capable of generating bursting rhythmic impulse activity under conditions of switching off of synaptic connections between cells (isolation). With switching on of electrical synapses, spiking generated by separate neurons, which initially was nonsynchronous, became synchronized in time. Ipso facto, we demonstrated the ability of pacemaker oscillatory activity to be electrotonically synchronized in ensembles of neurons connected with electrical synapses.     

RNA conformation in rod-shaped viruses: a possible molecular model for RNA in narcissus mosaic virus.

A model for visual adaptation to spatial grating is developed based on the assumption that inhibitory synapses within the visual system may be temporarily modified as a function of recent usage. Specifically, it is hypothesized that inhibitory synaptic weights are altered as a function of the correlation between recent presynaptic and postsynaptic activity. When such modifiable synapses are incorporated into a simple neural network model having the spatial filtering properties of the human visual system, two coupled equations are obtained which may be solved analytically. The model accounts for experimental data on adaptation to sinusoidal gratings, square wave gratings, single bars, and tilted gratings. The relationship of the model to single and multiple channel models of the human visual system is discussed.     

A synaptic model for spatial frequency adaptation

A model for visual adaptation to spatial grating is developed based on the assumption that inhibitory synapses within the visual system may be temporarily modified as a function of recent usage. Specifically, it is hypothesized that inhibitory synaptic weights are altered as a function of the correlation between recent presynaptic and postsynaptic activity. When such modifiable synapses are incorporated into a simple neural network model having the spatial filtering properties of the human visual system, two coupled equations are obtained which may be solved analytically. The model accounts for experimental data on adaptation to sinusoidal gratings, square wave gratings, single bars, and tilted gratings. The relationship of the model to single and multiple channel models of the human visual system is discussed.     

Self-organizing mechanism for the formation of ordered neural mappings

A model for the formation of ordered neural mappings in general, and of retinotectal connections, in particular is given. The main point came from the theory of noise induced transitions, i.e. order may be the result of the interplay between deterministic and random interactions. An activity-dependent self-organizing mechanism is presented in terms of modifiable synapses. Simulation experiments were done not only for the normal ontogenetic development but also for the plastic behaviour of the retinotopic connections.     

The differential regulation of formation of chemical and electrical connections in Helisoma

Novel chemical and electrical connections form between neurons not normally connected in the buccal ganglia of the snail Helisoma. We examined the cellular and environmental conditions required for the formation of each type of connection. Previous work in situ showed that novel electrical connections could form in response to axotomy. We have now found that axotomy can evoke the formation of novel unidirectional chemical connections between neurons B5 and B4 in addition to a novel electrical connection. The novel chemical connections display all of the normal properties of chemical synapses in Helisoma ganglia. These connections, however, are transient in nature and break 4 days following axotomy. Previous work has shown that conjoint outgrowth is required for the formation of electrical connections. In cell culture we have investigated whether conjoint outgrowth is also required for chemical synaptogenesis. Using neurons B5 and B19 we have found that when neuron pairs make contact in cell culture, under conditions of synchronous neurite extension, both electrical and chemical synapses form. However, if one neuron has ceased extension prior to contact by a growing neuron, electrical synapses never form (Hadley et al., 1983, 1985) but chemical synapses do form. Furthermore, the addition of serotonin (10(-6) M) to culture medium to inhibit neurite extension of B19, but not that of B5, selectively prevents the formation of electrical connections while permitting the formation of chemical synapses. Thus, the timing of contact in relation to the state of neurite extension can specify the type of connection a given neuron can form.     

A model of selective synapse formation in sympathetic ganglia

In the sympathetic system, neurons from several spinal segments are mapped onto targets in the periphery in a topographically ordered way by means of selective synaptic connections in the superior cervical ganglion. Experimental evidence points to a crucial role for chemoaffinity in establishing this topographic map. Furthermore, rearrangements of synapses after surgical manipulations indicate that this chemoaffinity is not based on rigid “key-and-lock” markers. Our model is used to study how such nonrigid markers may interact with other regulatory factors, including growth-regulating signals and the growth potential of individual nerons. In the model, these latter factors are limiting, so that an increasing number of synaptic contacts decreases the likelihood of further synapse formation. These factors are combined with chemoaffinity using a linear threshold model. The model is robust to parameter changes and reproduces experimental observations with reasonable detail. Simulation results are used to discuss characteristic experimental results, such as the substantial plasticity of the connections seen after partial denervation. A surprisingly small effect of transient hyperinnervation in the model may help explain why final connectivities are similar in two real situations with high and low degrees of transient hyperinnervation (development and adult reinnervation). It is shown that spatial restrictions on post-synaptic neurons (dendrites) may contribute significantly to the segmentally broad innervation of each ganglion cell. Finally, we discuss potential effects of presynaptic neuronal death in systems with a high degree of plasticity. © 1993 John Wiley & Sons, Inc.     

Formation and Maintenance of Robust Long-Term Information Storage in the Presence of Synaptic Turnover

A long-standing problem is how memories can be stored for very long times despite the volatility of the underlying neural substrate, most notably the high turnover of dendritic spines and synapses. To address this problem, here we are using a generic and simple probabilistic model for the creation and removal of synapses. We show that information can be stored for several months when utilizing the intrinsic dynamics of multi-synapse connections. In such systems, single synapses can still show high turnover, which enables fast learning of new information, but this will not perturb prior stored information (slow forgetting), which is represented by the compound state of the connections. The model matches the time course of recent experimental spine data during learning and memory in mice supporting the assumption of multi-synapse connections as the basis for long-term storage.     

The information theory of developmental pruning: Optimizing global network architectures using local synaptic rules

During development, biological neural networks produce more synapses and neurons than needed. Many of these synapses and neurons are later removed in a process known as neural pruning. Why networks should initially be over-populated, and the processes that determine which synapses and neurons are ultimately pruned, remains unclear. We study the mechanisms and significance of neural pruning in model neural networks. In a deep Boltzmann machine model of sensory encoding, we find that (1) synaptic pruning is necessary to learn efficient network architectures that retain computationally-relevant connections, (2) pruning by synaptic weight alone does not optimize network size and (3) pruning based on a locally-available measure of importance based on Fisher information allows the network to identify structurally important vs. unimportant connections and neurons. This locally-available measure of importance has a biological interpretation in terms of the correlations between presynaptic and postsynaptic neurons, and implies an efficient activity-driven pruning rule. Overall, we show how local activity-dependent synaptic pruning can solve the global problem of optimizing a network architecture. We relate these findings to biology as follows: (I) Synaptic over-production is necessary for activity-dependent connectivity optimization. (II) In networks that have more neurons than needed, cells compete for activity, and only the most important and selective neurons are retained. (III) Cells may also be pruned due to a loss of synapses on their axons. This occurs when the information they convey is not relevant to the target population.     

A model of stimulus-specific neural assemblies in the insect antennal lobe

It has been proposed that synchronized neural assemblies in the antennal lobe of insects encode the identity of olfactory stimuli. In response to an odor, some projection neurons exhibit synchronous firing, phase-locked to the oscillations of the field potential, whereas others do not. Experimental data indicate that neural synchronization and field oscillations are induced by fast GABA(A)-type inhibition, but it remains unclear how desynchronization occurs. We hypothesize that slow inhibition plays a key role in desynchronizing projection neurons. Because synaptic noise is believed to be the dominant factor that limits neuronal reliability, we consider a computational model of the antennal lobe in which a population of oscillatory neurons interact through unreliable GABA(A) and GABA(B) inhibitory synapses. From theoretical analysis and extensive computer simulations, we show that transmission failures at slow GABA(B) synapses make the neural response unpredictable. Depending on the balance between GABA(A) and GABA(B) inputs, particular neurons may either synchronize or desynchronize. These findings suggest a wiring scheme that triggers stimulus-specific synchronized assemblies. Inhibitory connections are set by Hebbian learning and selectively activated by stimulus patterns to form a spiking associative memory whose storage capacity is comparable to that of classical binary-coded models. We conclude that fast inhibition acts in concert with slow inhibition to reformat the glomerular input into odor-specific synchronized neural assemblies.     

Emergence of Connectivity Motifs in Networks of Model Neurons with Short- and Long-Term Plastic Synapses

Recent experimental data from the rodent cerebral cortex and olfactory bulb indicate that specific connectivity motifs are correlated with short-term dynamics of excitatory synaptic transmission. It was observed that neurons with short-term facilitating synapses form predominantly reciprocal pairwise connections, while neurons with short-term depressing synapses form predominantly unidirectional pairwise connections. The cause of these structural differences in excitatory synaptic microcircuits is unknown. We show that these connectivity motifs emerge in networks of model neurons, from the interactions between short-term synaptic dynamics (SD) and long-term spike-timing dependent plasticity (STDP). While the impact of STDP on SD was shown in simultaneous neuronal pair recordings in vitro, the mutual interactions between STDP and SD in large networks are still the subject of intense research. Our approach combines an SD phenomenological model with an STDP model that faithfully captures long-term plasticity dependence on both spike times and frequency. As a proof of concept, we first simulate and analyze recurrent networks of spiking neurons with random initial connection efficacies and where synapses are either all short-term facilitating or all depressing. For identical external inputs to the network, and as a direct consequence of internally generated activity, we find that networks with depressing synapses evolve unidirectional connectivity motifs, while networks with facilitating synapses evolve reciprocal connectivity motifs. We then show that the same results hold for heterogeneous networks, including both facilitating and depressing synapses. This does not contradict a recent theory that proposes that motifs are shaped by external inputs, but rather complements it by examining the role of both the external inputs and the internally generated network activity. Our study highlights the conditions under which SD-STDP might explain the correlation between facilitation and reciprocal connectivity motifs, as well as between depression and unidirectional motifs.     

High-capacity embedding of synfire chains in a cortical network model

Synfire chains, sequences of pools linked by feedforward connections, support the propagation of precisely timed spike sequences, or synfire waves. An important question remains, how synfire chains can efficiently be embedded in cortical architecture. We present a model of synfire chain embedding in a cortical scale recurrent network using conductance-based synapses, balanced chains, and variable transmission delays. The network attains substantially higher embedding capacities than previous spiking neuron models and allows all its connections to be used for embedding. The number of waves in the model is regulated by recurrent background noise. We computationally explore the embedding capacity limit, and use a mean field analysis to describe the equilibrium state. Simulations confirm the mean field analysis over broad ranges of pool sizes and connectivity levels; the number of pools embedded in the system trades off against the firing rate and the number of waves. An optimal inhibition level balances the conflicting requirements of stable synfire propagation and limited response to background noise. A simplified analysis shows that the present conductance-based synapses achieve higher contrast between the responses to synfire input and background noise compared to current-based synapses, while regulation of wave numbers is traced to the use of variable transmission delays.     

Feedback network controls photoreceptor output at the layer of first visual synapses in Drosophila

At the layer of first visual synapses, information from photoreceptors is processed and transmitted towards the brain. In fly compound eye, output from photoreceptors (R1-R6) that share the same visual field is pooled and transmitted via histaminergic synapses to two classes of interneuron, large monopolar cells (LMCs) and amacrine cells (ACs). The interneurons also feed back to photoreceptor terminals via numerous ligand-gated synapses, yet the significance of these connections has remained a mystery. We investigated the role of feedback synapses by comparing intracellular responses of photoreceptors and LMCs in wild-type Drosophila and in synaptic mutants, to light and current pulses and to naturalistic light stimuli. The recordings were further subjected to rigorous statistical and information-theoretical analysis. We show that the feedback synapses form a negative feedback loop that controls the speed and amplitude of photoreceptor responses and hence the quality of the transmitted signals. These results highlight the benefits of feedback synapses for neural information processing, and suggest that similar coding strategies could be used in other nervous systems.     

Stabilization of Hebbian neural nets by inhibitory learning

In Hebbian neural models synaptic reinforcement occurs when the pre- and post-synaptic neurons are simultaneously active. This causes an instability toward unlimited growth of excitatory synapses. The system can be stabilized by recurrent inhibition via modifiable inhibitory synapses. When this process is included, it is possible to dispense with the non-linear normalization or cut-off conditions which were necessary for stability in previous models. The present formulation is response-linear if synaptic changes are slow. It is self-consistent because the stabilizing effects will tend to keep most neural activity in the middle range, where neural response is approximately linear. The linearized equations are tensor invariant under a class of rotations of the state space. Using this, the response to stimulation may be derived as a set of independent modes of activity distributed over the net, which may be identified with cell assemblies. A continuously infinite set of equivalent solutions exists.     

Mathematical theory on formation of category detecting nerve cells

The nerve cells are believed to have such ability of self-organization that, given a number of input patterns, each cell tunes itself to become responsive to only one of the patterns, or to one subset of patterns having some features in common. The detectors of patterns or pattern subsets are formed in this manner. A simple but plausible mechanism of self-organization is proposed based on the two hypotheses: 1) Synaptic modification process is non-linear, activated when the output of a cell is positive. 2) Not only excitatory but also inhibitory synapses are modifiable. A rigorous mathematical analysis is given to elucidate the characteristics of modifiable synapses to form these detectors. The present model fits well most of the experiments on the developmental plasticity of the visual cortex such as the formation of orientation detecting cells, monocular and alternate monocular deprivation in normal and abnormal environments.