A neural circuit model forming semantic network with exception using spike-timing-dependent plasticity of inhibitory synapses

We propose a neural circuit model forming a semantic network with exceptions using the spike-timing-dependent plasticity (STDP) of inhibitory synapses. To evaluate the proposed model, we conducted nine types of computer simulation by combining the three STDP rules for inhibitory synapses and the three spike pairing rules. The simulation results obtained with the STDP rule for inhibitory synapses by Haas et al. [Haas, J.S., Nowotny, T., Abarbanel, H.D.I., 2006, Spike-timing-dependent plasticity of inhibitory synapses in the entorhinal cortex. J. Neurophysiol. 96, 3305–3313] are successful, whereas, the other results are unsuccessful. The results and examinations suggested that the inhibitory connection from the concept linked with an exceptional feature to the general feature is necessary for forming a semantic network with an exception.     

Interrelated modification of excitatory and inhibitory synapses in three-layer olivary-cerebellar neural network

The model of three-layer olivary-cerebellar neural network with modifiable excitatory and inhibitory connections between diverse elements is suggested. The same Hebbian modification rules are proposed for Purkinje cells, granule (input) cells, and deep cerebellar nuclei (output) cells. The inverse calcium-dependent modification rules for these cells and hippocampal/neocortical neurones or Golgi cells are conceivably the result of the involvement of cGMP and cAMP in postsynaptic processes. The sign of simultaneous modification of excitatory and inhibitory inputs to a cell is opposite and determined by the variations in pre- and/or postsynaptic cell activity. Modification of excitatory transmission between parallel fibers and Purkinje cells, mossy fibers and granule cells, and mossy fibers and deep cerebellar nuclei cells essentially depends on inhibition effected by stellate/basket cells, Golgi cells and Purkinje cells, respectively. The character of interrelated modifications of diverse synapses in all three layers of the network is influenced by olivary cell activity. In the absence (presence) of a signal from inferior olive, the long-term potentiation (depression) in the efficacy of a synapse between input mossy fiber and output cell can be induced. The results of the suggested model are in accordance with known experimental data.     

Multisensory integration in the superior colliculus: a neural network model

Neurons in the superior colliculus (SC) are known to integrate stimuli of different modalities (e.g., visual and auditory) following specific properties. In this work, we present a mathematical model of the integrative response of SC neurons, in order to suggest a possible physiological mechanism underlying multisensory integration in SC. The model includes three distinct neural areas: two unimodal areas (auditory and visual) are devoted to a topological representation of external stimuli, and communicate via synaptic connections with a third downstream area (in the SC) responsible for multisensory integration. The present simulations show that the model, with a single set of parameters, can mimic various responses to different combinations of external stimuli including the inverse effectiveness, both in terms of multisensory enhancement and contrast, the existence of within- and cross-modality suppression between spatially disparate stimuli, a reduction of network settling time in response to cross-modal stimuli compared with individual stimuli. The model suggests that non-linearities in neural responses and synaptic (excitatory and inhibitory) connections can explain several aspects of multisensory integration.     

A spiking neural network model of the midbrain superior colliculus that generates saccadic motor commands

Single-unit recordings suggest that the midbrain superior colliculus (SC) acts as an optimal controller for saccadic gaze shifts. The SC is proposed to be the site within the visuomotor system where the nonlinear spatial-to-temporal transformation is carried out: the population encodes the intended saccade vector by its location in the motor map (spatial), and its trajectory and velocity by the distribution of firing rates (temporal). The neurons’ burst profiles vary systematically with their anatomical positions and intended saccade vectors, to account for the nonlinear main-sequence kinematics of saccades. Yet, the underlying collicular mechanisms that could result in these firing patterns are inaccessible to current neurobiological techniques. Here, we propose a simple spiking neural network model that reproduces the spike trains of saccade-related cells in the intermediate and deep SC layers during saccades. The model assumes that SC neurons have distinct biophysical properties for spike generation that depend on their anatomical position in combination with a center–surround lateral connectivity. Both factors are needed to account for the observed firing patterns. Our model offers a basis for neuronal algorithms for spatiotemporal transformations and bio-inspired optimal controllers.     

Effect of destruction of the inferior colliculus on function of the echolocation system in horseshoe bats

The effect of unilateral and bilateral destruction of the inferior colliculus on the sensitivity of the auditory system, on parameters of the sonor signals, and on Doppler shift compensation in echo signals was studied in experiments on horseshoe bats (Rhinolophus ferrum-equinum). The results show that complete bilateral destruction of the inferior colliculus in bats does not lead to total disturbance of function of the auditory system but it sharply reduces the sensitivity of that system, as shown by a decrease in the maximal obstacle detection range and inability to respond to an insect emitting a feeble sound. It can also be concluded that the inferior colliculus plays a direct part in maintenance of the emission frequency and that different parts of the inferior colliculus play different roles in this process. The Doppler shift compensation effect requires preservation of the integrity of not less than half of the central nucleus of at least one inferior colliculus.A. A. Ukhtomskii Physiological Institute, A. A. Zhdanov State University, Leningrad. Translated from Neirofiziologiya, Vol. 12, No. 4, pp. 375–381, July–August, 1980.     

A neural network model for trace conditioning

We studied the dynamics of a neural network that has both recurrent excitatory and random inhibitory connections. Neurons started to become active when a relatively weak transient excitatory signal was presented and the activity was sustained due to the recurrent excitatory connections. The sustained activity stopped when a strong transient signal was presented or when neurons were disinhibited. The random inhibitory connections modulated the activity patterns of neurons so that the patterns evolved without recurrence with time. Hence, a time passage between the onsets of the two transient signals was represented by the sequence of activity patterns. We then applied this model to represent the trace eye blink conditioning, which is mediated by the hippocampus. We assumed this model as CA3 of the hippocampus and considered an output neuron corresponding to a neuron in CA1. The activity pattern of the output neuron was similar to that of CA1 neurons during trace eye blink conditioning, which was experimentally observed.     

A neural network model for the mechanism of feature-extraction

We propose a new multilayered neural network model which has the ability of rapid self-organization. This model is a modified version of the cognitron (Fukushima, 1975). It has modifiable inhibitory feedback connections, as well as conventional modifiable excitatory feedforward connections, between the cells of adjoining layers. If a feature-extracting cell in the network is excited by a stimulus which is already familiar to the network, the cell immediately feeds back inhibitory signals to its presynaptic cells in the preceding layer, which suppresses their response. On the other hand, the feature-extracting cell does not respond to an unfamiliar feature, and the responses from its presynaptic cells are therefore not suppressed because they do not receive any feedback inhibition. Modifiable synapses in the new network are reinforced in a way similar to those in the cognitron, and synaptic connections from cells yielding a large sustained output are reinforced. Since familiar stimulus features do not elicit a sustained response from the cells of the network, only circuits which detect novel stimulus features develop. The network therefore quickly acquires favorable pattern-selectivity by the mere repetitive presentation of set of learning patterns.     

A neural network model for texture discrimination

A model of texture discrimination in visual cortex was built using a feedforward network with lateral interactions among relatively realistic spiking neural elements. The elements have various membrane currents, equilibrium potentials and time constants, with action potentials and synapses. The model is derived from the modified programs of MacGregor (1987). Gabor-like filters are applied to overlapping regions in the original image; the neural network with lateral excitatory and inhibitory interactions then compares and adjusts the Gabor amplitudes in order to produce the actual texture discrimination. Finally, a combination layer selects and groups various representations in the output of the network to form the final transformed image material. We show that both texture segmentation and detection of texture boundaries can be represented in the firing activity of such a network for a wide variety of synthetic to natural images. Performance details depend most strongly on the global balance of strengths of the excitatory and inhibitory lateral interconnections. The spatial distribution of lateral connective strengths has relatively little effect. Detailed temporal firing activities of single elements in the lateral connected network were examined under various stimulus conditions. Results show (as in area 17 of cortex) that a single element's response to image features local to its receptive field can be altered by changes in the global context.     

Neuronal network with modifiable synapses: decoding of composite sensory messages under unsupervised and permanent learning

Afferent sensory fibers carry usually messages which combine several primitive entities. A model of neural structure under unsupervised and permanent learning is proposed, which can detect the primary independent entities mixed in the afferent message. The model is a network of neurons mutually interconnected by means of inhibitory modifiable synapses with efficacies locally controlled by a specific conjunction law of pre- and post-synaptic activities.     

A neural network model approach to the study of human TAP transporter

We used an artificial neural network (ANN) computer model to study peptide binding to the human transporter associated with antigen processing (TAP). After validation, an ANN model of TAP-peptide binding was used to mine a database of HLA-binding peptides to elucidate patterns of TAP binding. The affinity of HLA-binding peptides for TAP was found to differ according to the HLA supertype concerned: HLA-B27, -A3 or -A24 binding peptides had high, whereas HLA-A2, -B7 or -B8 binding peptides had low affinity for TAP. These results support the idea that TAP and particular HLA molecules may have co-evolved for efficient peptide processing and presentation. The strong similarity between the sets of peptides bound by TAP or HLA-B27 suggests functional co-evolution whereas the lack of a relationship between the sets of peptides bound by TAP or HLA-A2 is against these particular molecules having co-evolved. In support of these conclusions, the affinities of HLA-A2 and HLA-B7 binding peptides for TAP show similar distributions to that of randomly generated peptides. On the basis of these results we propose that HLA alleles constitute two separate classes: those that are TAP-efficient for peptide loading (HLA-B27, -A3 and -A24) and those that are TAP-inefficient (HLA-A2, -B7 and -B8). Computer modelling can be used to complement laboratory experiments and thereby speed up knowledge discovery in biology. In particular, we provide evidence that large-scale experiments can be avoided by combining initial experimental data with limited laboratory experiments sufficient to develop and validate appropriate computer models. These models can then be used to perform large-scale simulated experiments the results of which can then be validated by further small-scale laboratory experiments.     

Energy coding in neural network with inhibitory neurons

This paper aimed at assessing and comparing the effects of the inhibitory neurons in the neural network on the neural energy distribution, and the network activities in the absence of the inhibitory neurons to understand the nature of neural energy distribution and neural energy coding. Stimulus, synchronous oscillation has significant difference between neural networks with and without inhibitory neurons, and this difference can be quantitatively evaluated by the characteristic energy distribution. In addition, the synchronous oscillation difference of the neural activity can be quantitatively described by change of the energy distribution if the network parameters are gradually adjusted. Compared with traditional method of correlation coefficient analysis, the quantitative indicators based on nervous energy distribution characteristics are more effective in reflecting the dynamic features of the neural network activities. Meanwhile, this neural coding method from a global perspective of neural activity effectively avoids the current defects of neural encoding and decoding theory and enormous difficulties encountered. Our studies have shown that neural energy coding is a new coding theory with high efficiency and great potential.     

A neural network model for the emergence of grating cells

A correlation-based learning (CBL) neural network model is proposed, which simulates the emergence of grating cells as well as some of their response characteristics to periodic pattern stimuli. These cells, found in areas V1 and V2 of the visual cortex of monkeys, respond vigorously and exclusively to bar gratings of a preferred orientation and periodicity. Their non-linear behaviour differentiates grating cells from other orientation-selective cells, which show linear spatial frequency filtering. Received: 9 June 1997 / Accepted in revised form: 9 February 1998     

Analysis of stability of neural network with inhibitory neurons

Phase coding in a neural network composed of neural oscillators with inhibitory neurons was studied based on the theory of stochastic phase dynamics. We found that with increasing the coupling coefficients of inhibitory neural oscillators, the firing density in excitatory population transits to a critical state. In this case, when we increase the inhibitory coupling, the firing density will come into dynamic balance again and tend to a fixed value gradually. According to the phenomenon, in the paper we found parameter regions to exhibit those different population states, called dividing zones including flat fading zone, rapid fading zone and critical zone. Based on the dividing zones we can choose the number ratio between inhibitory neurons and excitatory neurons in the neural network, and estimate the coupling action of inhibitory population and excitatory population. Our research also shows that the balance value, enabling the firing density to reach the dynamic balance, does not depend on initial conditions. In addition, the critical value in critical state is only related to the number ratio between inhibitory neurons and excitatory neurons, but is independent of inhibitory coupling and excitatory coupling.     

A neural network model for stroboscopic alternative motion

A neural network which models multistable perception is presented. The network consists of sensor and inner neurons. The dynamics is established by a stochastic neuronal dynamics, a formal Hebb-type coupling dynamics and a resource mechanism that corresponds to saturation effects in perception. From this a system of coupled differential equations is derived and analyzed. Single stimuli are bound to exactly one percept, even in ambiguous situations where multistability occurs. The network exhibits discontinuous as well as continuous phase transitions and models various empirical findings, including the percepts of succession, alternative motion and simultaneity; the percept of oscillation is explained by oscillating percepts at a continuous phase transition. Received: 13 September 1995 / Accepted: 3 June 1996     

A neural network model for limb trajectory formation

This paper deals with the problem of representing and generating unconstrained aiming movements of a limb by means of a neural network architecture. The network produced time trajectories of a limb from a starting posture toward targets specified by sensory stimuli. Thus the network performed a sensory-motor transformation. The experimenters trained the network using a bell-shaped velocity profile on the trajectories. This type of profile is characteristic of most movements performed by biological systems. We investigated the generalization capabilities of the network as well as its internal organization. Experiments performed during learning and on the trained network showed that: (i) the task could be learned by a three-layer sequential network; (ii) the network successfully generalized in trajectory space and adjusted the velocity profiles properly; (iii) the same task could not be learned by a linear network; (iv) after learning, the internal connections became organized into inhibitory and excitatory zones and encoded the main features of the training set; (v) the model was robust to noise on the input signals; (vi) the network exhibited attractor-dynamics properties; (vii) the network was able to solve the motorequivalence problem. A key feature of this work is the fact that the neural network was coupled to a mechanical model of a limb in which muscles are represented as springs. With this representation the model solved the problem of motor redundancy.     

A hierarchical neural network model for associative memory

A hierarchical neural network model with feedback interconnections, which has the function of associative memory and the ability to recognize patterns, is proposed. The model consists of a hierarchical multi-layered network to which efferent connections are added, so as to make positive feedback loops in pairs with afferent connections. The cell-layer at the initial stage of the network is the input layer which receives the stimulus input and at the same time works as an output layer for associative recall. The deepest layer is the output layer for pattern-recognition. Pattern-recognition is performed hierarchically by integrating information by converging afferent paths in the network. For the purpose of associative recall, the integrated information is again distributed to lower-order cells by diverging efferent paths. These two operations progress simultaneously in the network. If a fragment of a training pattern is presented to the network which has completed its self-organization, the entire pattern will gradually be recalled in the initial layer. If a stimulus consisting of a number of training patterns superposed is presented, one pattern gradually becomes predominant in the recalled output after competition between the patterns, and the others disappear. At about the same time when the recalled pattern reaches a steady state in he initial layer, in the deepest layer of the network, a response is elicited from the cell corresponding to the category of the finally-recalled pattern. Once a steady state has been reached, the response of the network is automatically extinguished by inhibitory signals from a steadiness-detecting cell. If the same stimulus is still presented after inhibition, a response for another pattern, formerly suppressed, will now appear, because the cells of the network have adaptation characteristics which makes the same response unlikely to recur. Since inhibition occurs repeatedly, the superposed input patterns are recalled one by one in turn.     

Doubly modifiable synapses: a model of short and long term auto-associative memory

Synapses that can be strengthened in temporary and persistent manners by two separate mechanisms are shown to have powerful advantages in neural networks that perform auto-associative recall and recognition. A multiplicative relation between the two weights allows the same set of connections to be used in a closely interactive way for short-term and long-term memory. Algorithms and simulations are described for the storage, consolidation and recall of patterns that have been presented only once to a network. With double modifiability, the short-term performance is dramatically improved, becoming almost independent of the amount of long-term experience. The high quality of short-term recall allows consolidation to take place, with benefits from the selection and optimization of long term engrams to take account of relations between stored patterns. Long-term capacity is greater than short-term capacity, with little or no deficit compared with that obtained with singly modifiable synapses. Long-term recall requires special, simply implemented, procedures for increasing the temporary weights of the synapses being used to initiate recall. A consolidation algorithm is described for improving long-term recall when there is overlap between patterns. Confusional errors are reduced by strengthening the associations between non-overlapping elements in the patterns, in a two-stage process that has several of the characteristics of sleep.     

A model of neural network for spatiotemporal pattern recognition

A model of neural network to recognize spatiotemporal patterns is presented. The network consists of two kinds of neural cells: P-cells and B-cells. A P-cell generates an impulse responding to more than one impulse and embodies two special functions: short term storage (STS) and heterosynaptic facilitation (HSF). A B-cell generates several impulses with high frequency as soon as it receives an impulse. In recognizing process, an impulse generated by a P-cell represents a recognition of stimulus pattern, and triggers the generation of impulses of a B-cell. Inhibitory impulses with high frequency generated by a B-cell reset the activities of all P-cells in the network.Two examples of spatiotemporal pattern recognition are presented. They are achieved by giving different values to the parameters of the network. In one example, the network recognizes both directional and non-directional patterns. The selectivities to directional and non-directional patterns are realized by only adjusting excitatory synaptic weights of P-cells. In the other example, the network recognizes time series of spatial patterns, where the lengths of the series are not necessarily the same and the transitional speeds of spatial patterns are not always the same. In both examples, the HSF signal controls the total activity of the network, which contributes to exact recognition and error recovery. In the latter example, it plays a role to trigger and execute the recognizing process. Finally, we discuss the correspondence between the model and physiological findings.     

A neural network model of the McCollough effect

A self-organizing, feature-extracting network (von der Malsburg, 1973) is extended to two feature dimensions to encompass line orientation and color. It is applied to McCollough effects, particularly longlasting, contingent-aftereffects. McCollough effects are thought to involve low-level associative memory in the form of synaptic modification. The McCollough-Malsburg Model (MMM) embodies positive synaptic modification with correlated firing of units in an input layer and an excitatory cortical layer. Computer simulation of MMM reproduces orientation-contingent color aftereffects. The model embodies many of the mechanisms thought to be operating in developmental plasticity, suggesting that equivalent mechanisms may be involved in adult long-term adaptation as well.This work was supported in part by NIH Grant No. 5 R01 NS09755-4 COM of the National Institute of Neurological Diseases and Stroke (M.A. Arbib, Principal Investigator)