A neuron model with learning capability and its relation to mechanisms of association

A neuron model with the ability of learning has been examined by means of mathematical and statistical methods. By use of the established anatomical concepts the main features of the model can be described as follows.The synapses are randomly distributed on the dendrites in a way that can be described by poisson processes. The afferent connections to the synapses are also random.The input signals are divided into excitatory, inhibitory and unspecified signals. The latter, whose detailed action is not specified, may involve excitatory as well as inhibitory action on the cell. Signals are described in terms of impulse frequencies.Learning takes place through facilitation of excitatory synapses. The condition for facilitation is the occurrence of simultaneous presynaptic and postsynaptic activity. The synaptical changes occurring during repeated learning are superimposed. Inhibitory synapses are capable of influencing learning by blocking the dendritic transmission.It is shown that, under certain conditions, a collection of model cells is able to work as an associative memory. This means that a pattern of output signals that once occurred through the combined action of the excitatory, the inhibitory, and the unspecified signals may later be recalled by applying just the two former signal patterns. It is shown that excitatory and inhibitory signals are similar in their ability to evoke associations.However there is also a difference between excitation and inhibition due to the fact that the pattern of inhibitory signals is subject to a non-linear transformation. This implies that great similarity is required between the inhibitory pattern once present during learning and the inhibitory pattern that is fed in later in order to obtain an associative recall. This phenomenon is called pattern separation and is supposed to be of importance when discriminating between patterns.     

Analysis of excitatory microcircuitry in the medial entorhinal cortex reveals cell-type-specific differences

Medial entorhinal cortex (MEC) plays an important role in physiological processes underlying navigation, learning, and memory. Excitatory cells in the different MEC layers project in a region-specific manner to the hippocampus. However, the intrinsic microcircuitry of the main excitatory cells in the superficial MEC layers is largely unknown. Using scanning photostimulation, we investigated the functional microcircuitry of two such cell types, stellate and pyramidal cells. We found cell-type-specific intralaminar and ascending interlaminar feedback inputs. The ascending interlaminar inputs display distinct organizational principles depending on the cell-type and its position within the superficial lamina: the spatial spread of inputs for stellate cells is narrower than for pyramidal cells, while inputs to pyramidal cells in layer 3, but not in layer 2, exhibit an asymmetric offset to the medial side of the cell's main axis. Differential laminar sources of excitatory inputs might contribute to the functional diversity of stellate and pyramidal cells.     

A novel network of multipolar bursting interneurons generates theta frequency oscillations in neocortex

GABAergic interneurons can phase the output of principal cells, giving rise to oscillatory activity in different frequency bands. Here we describe a new subtype of GABAergic interneuron, the multipolar bursting (MB) cell in the mouse neocortex. MB cells are parvalbumin positive but differ from fast-spiking multipolar (FS) cells in their morphological, neurochemical, and physiological properties. MB cells are reciprocally connected with layer 2/3 pyramidal cells and are coupled with each other by chemical and electrical synapses. MB cells innervate FS cells but not vice versa. MB to MB cell as well as MB to pyramidal cell synapses exhibit paired-pulse facilitation. Carbachol selectively induced synchronized theta frequency oscillations in MB cells. Synchrony required both gap junction coupling and GABAergic chemical transmission, but not excitatory glutamatergic input. Hence, MB cells form a distinct inhibitory network, which upon cholinergic drive can generate rhythmic and synchronous theta frequency activity, providing temporal coordination of pyramidal cell output.     

Mechanisms of dendritic integration underlying gain control in fly motion-sensitive interneurons

In the compensatory optomotor response of the fly the interesting phenomenon of gain control has been observed by Reichardt and colleagues (Reichardt et al., 1983): The amplitude of the response tends to saturate with increasing stimulus size, but different saturation plateaus are assumed with different velocities at which the stimulus is moving. This characteristic can already be found in the motion-sensitive large field neurons of the fly optic lobes that play a role in mediating this behavioral response (Hausen, 1982; Reichardt et al, 1983; Egelhaaf, 1985; Haag et al., 1992). To account for gain control a model was proposed involving shunting inhibition of these cells by another cell, the so-called pool cell (Reichardt et al., 1983), both cells sharing common input from an array of local motion detectors. This article describes an alternative model which only requires dendritic integration of the output signals of two types of local motion detectors with opposite polarity. The explanation of gain control relies on recent findings that these input elements are not perfectly directionally selective and that their direction selectivity is a function of pattern velocity. As a consequence, the resulting postsynaptic potential in the dendrite of the integrating cell saturates with increasing pattern size at a level between the excitatory and inhibitory reversal potentials. The exact value of saturation is then set by the activation ratio of excitatory and inhibitory input elements which in turn is a function of other stimulus parameters such as pattern velocity. Thus, the apparently complex phenomenon of gain control can be simply explained by the biophysics of dendritic integration in conjunction with the properties of the motion-sensitive input elements.     

Interrelated modification of excitatory and inhibitory synapses in three-layer olivary-cerebellar neural network

The model of three-layer olivary-cerebellar neural network with modifiable excitatory and inhibitory connections between diverse elements is suggested. The same Hebbian modification rules are proposed for Purkinje cells, granule (input) cells, and deep cerebellar nuclei (output) cells. The inverse calcium-dependent modification rules for these cells and hippocampal/neocortical neurones or Golgi cells are conceivably the result of the involvement of cGMP and cAMP in postsynaptic processes. The sign of simultaneous modification of excitatory and inhibitory inputs to a cell is opposite and determined by the variations in pre- and/or postsynaptic cell activity. Modification of excitatory transmission between parallel fibers and Purkinje cells, mossy fibers and granule cells, and mossy fibers and deep cerebellar nuclei cells essentially depends on inhibition effected by stellate/basket cells, Golgi cells and Purkinje cells, respectively. The character of interrelated modifications of diverse synapses in all three layers of the network is influenced by olivary cell activity. In the absence (presence) of a signal from inferior olive, the long-term potentiation (depression) in the efficacy of a synapse between input mossy fiber and output cell can be induced. The results of the suggested model are in accordance with known experimental data.     

Visual modelling

The first purpose of this paper is to present a neural net model of the visual cortex of higher vertebrates based on the electrophysiological properties of the ganglion cells. This model takes Hebb's law [1] as the physiological learning rule for synaptic modification. The model consists of 85 × 85 neurons forming a layer similar to the cortex. The neurones are massively connected via weights that are typically adapted. We simulate several input patterns and show that the model reproduces the pattern recognition, contours pictures and moving perception.     

Probabilistic secretion of quanta in the central nervous system: granule cell synaptic control of pattern separation and activity regulation.

The implications of probabilistic secretion of quanta for the functioning of neural networks in the central nervous system have been explored. A model of stochastic secretion at synapses in simple networks, consisting of large numbers of granule cells and a relatively small number of inhibitory interneurons, has been analysed. Such networks occur in the input to the cerebellum Purkinje cells as well as to hippocampal CA3 pyramidal cells and to pyramidal cells in the visual cortex. In this model the input axons terminate on granule cells as well as on an inhibitory interneuron that projects to the granule cells. Stochastic secretion at these synapses involves both temporal variability in secretion at single synapses in the network as well as spatial variability in the secretion at different synapses. The role of this stochastic variability in controlling the size of the granule cell output to a level independent of the size of the input and in separating overlapping inputs has been determined analytically as well as by simulation. The regulation of granule-cell output activity to a reasonably constant value for different size inputs does not occur in the absence of an inhibitory interneuron when both spatial and temporal stochastic variability occurs at the remaining synapses; it is still very poor in the presence of such an interneuron but in the absence of stochastic variability. However, quite good regulation is achieved when the inhibitory interneuron is present with spatial and temporal stochastic variability of secretion at synapses in the network. Excellent regulation is achieved if, in addition, allowance is made for the nonlinear behaviour of the input-output characteristics of inhibitory interneurons. The capacity of granule-cell networks to separate overlapping patterns of activity on their inputs is adequate, with spatial variability in the secretion at synapses, but is improved if there is also temporal variability in the stochastic secretion at individual synapses, although this is at the expense of reliability in the network. Other factors which improve pattern separation are control of the output to very low activity levels, and a restriction on the cumulative size of the excitatory input terminals of each granule cell. Application of the theory to the input neural networks of the cerebellum and the hippocampus shows the role of stochastic variability in quantal transmission in determining the capacity of these networks for pattern separation and activity regulation.     

Improving recall from an associative memory

The associative net as a model of biological associative memory is investigated. Calculating the output pattern retrieved from a partially connected associative net presented with noisy input cues involves several computations. This is complicated by variations in the dendritic sums of the output units due to errors in the cue and differences in input activity and unit usage. The possible implementation of these computations by biological neural machinery is unclear. We demonstrate that a relatively simple transformation can reduce variation in the dendritic sums. This leads to a winners-take-all type of strategy that produces increased recall performance which is equivalent to the more complicated optimal strategy proposed by others. We describe in detail the possible biological implications of our strategies, the novel feature of which ascribes a role to the NMDA and non-NMDA channels found in hippocampal pyramidal cells. Received: 13 April 1994 / Accepted: 25 October 1994     

A quantitative profile of the synapses on the stellate cell body and axon in the cochlear nucleus of the chinchilla

Summary. One of the most numerous neurons in the cochlear nucleus is the type I stellate cell. Previous attempts to understand the structural basis for its signal coding assumed that integration of synaptic potentials arising from axodendritic synapses should account for the generation of its response properties. However, the present study documents the importance of excitatory and inhibitory types of synapses on the soma and axon. Retrograde transport of cholera toxin B subunit, injected in the inferior colliculus of chinchillas, was used to label exclusively type I stellate cells in the anteroventral cochlear nucleus. The relative distribution of terminal types by vesicle morphology was pleomorphic < large spherical < flattened < smaller spherical. The somatic perimeter covered by endings ranged from almost none to nearly half. More flattened-vesicle terminals contacted somata in the high-frequency than in the low-frequency region. Eight of twenty axons received endings that contained large spherical vesicles and made asymmetric junctions; half of these extensively apposed the initial segment, forming a collar of presumed excitatory input. Thus, type I stellate cells are a heterogeneous group. Inhibitory synapses probably compose the majority of terminals. Some cells receive mostly inhibitory synapses near the presumed site of the spike generator, but others also have a prominent excitatory input. These findings call for a new look at the mechanisms for signal coding in stellate cells in the auditory system in particular and raise issues concerning the stochastic nature of information processing in sensory systems in general.     

High-frequency oscillations and sequence generation in two-population models of hippocampal region CA1

Hippocampal sharp wave/ripple oscillations are a prominent pattern of collective activity, which consists of a strong overall increase of activity with superimposed (140 − 200 Hz) ripple oscillations. Despite its prominence and its experimentally demonstrated importance for memory consolidation, the mechanisms underlying its generation are to date not understood. Several models assume that recurrent networks of inhibitory cells alone can explain the generation and main characteristics of the ripple oscillations. Recent experiments, however, indicate that in addition to inhibitory basket cells, the pattern requires in vivo the activity of the local population of excitatory pyramidal cells. Here, we study a model for networks in the hippocampal region CA1 incorporating such a local excitatory population of pyramidal neurons. We start by investigating its ability to generate ripple oscillations using extensive simulations. Using biologically plausible parameters, we find that short pulses of external excitation triggering excitatory cell spiking are required for sharp/wave ripple generation with oscillation patterns similar to in vivo observations. Our model has plausible values for single neuron, synapse and connectivity parameters, random connectivity and no strong feedforward drive to the inhibitory population. Specifically, whereas temporally broad excitation can lead to high-frequency oscillations in the ripple range, sparse pyramidal cell activity is only obtained with pulse-like external CA3 excitation. Further simulations indicate that such short pulses could originate from dendritic spikes in the apical or basal dendrites of CA1 pyramidal cells, which are triggered by coincident spike arrivals from hippocampal region CA3. Finally we show that replay of sequences by pyramidal neurons and ripple oscillations can arise intrinsically in CA1 due to structured connectivity that gives rise to alternating excitatory pulse and inhibitory gap coding; the latter denotes phases of silence in specific basket cell groups, which induce selective disinhibition of groups of pyramidal neurons. This general mechanism for sequence generation leads to sparse pyramidal cell and dense basket cell spiking, does not rely on synfire chain-like feedforward excitation and may be relevant for other brain regions as well.     

Optimal learning with excitatory and inhibitory synapses

Characterizing the relation between weight structure and input/output statistics is fundamental for understanding the computational capabilities of neural circuits. In this work, I study the problem of storing associations between analog signals in the presence of correlations, using methods from statistical mechanics. I characterize the typical learning performance in terms of the power spectrum of random input and output processes. I show that optimal synaptic weight configurations reach a capacity of 0.5 for any fraction of excitatory to inhibitory weights and have a peculiar synaptic distribution with a finite fraction of silent synapses. I further provide a link between typical learning performance and principal components analysis in single cases. These results may shed light on the synaptic profile of brain circuits, such as cerebellar structures, that are thought to engage in processing time-dependent signals and performing on-line prediction.     

A neural network model for trace conditioning

We studied the dynamics of a neural network that has both recurrent excitatory and random inhibitory connections. Neurons started to become active when a relatively weak transient excitatory signal was presented and the activity was sustained due to the recurrent excitatory connections. The sustained activity stopped when a strong transient signal was presented or when neurons were disinhibited. The random inhibitory connections modulated the activity patterns of neurons so that the patterns evolved without recurrence with time. Hence, a time passage between the onsets of the two transient signals was represented by the sequence of activity patterns. We then applied this model to represent the trace eye blink conditioning, which is mediated by the hippocampus. We assumed this model as CA3 of the hippocampus and considered an output neuron corresponding to a neuron in CA1. The activity pattern of the output neuron was similar to that of CA1 neurons during trace eye blink conditioning, which was experimentally observed.     

Dynamic model of neuronal response in the rat hippocampus

A linear lumped model was proposed for the hippocampal CA 1 region of anesthetized rats using differential equations of time-independent coefficients, the afferent and efferent fibers of the alveus as inputs and the averaged evoked potentials (AEPs) and poststimulus time histograms as outputs. The alvear tract, a major efferent path, was proposed to activate interneurons monosynaptically while the anterior alveus activated orthodromically pyramidal cells which then excited the interneurons. The interneurons then inhibited pyramidal cells. The observable field outputs were the excitatory postsynaptic potentials (EPSPs) of interneurons and the inhibitory postsynaptic potentials (IPSPs) of pyramidal cells. Positive neurophysiological feedbacks were proposed among interneurons and among pyramidal cells in order to account for the prolonged time courses of the interneuronal EPSPs and the pyramidal cell IPSPs. The parameters of the model were optimized by a nonlinear regression program which minimized the sum of squared deviations between the model-generated and actual AEPs. The parameters included the temporal dispersion of the input tract (about 3 ms) and the membrane time constant of interneuronal and pyramidal cell populations (4.8 ms). In anesthetized rats, positive feedback gain coefficients were 0.07 among interneurons and 0.85 among pyramidal cells. After a compound spike (I), two postsynaptic AEP components (II and III) of different time courses were detectable at all depths within CA 1 except at the turnover for each component. The hypothesis that the AEP component II was generated by interneurons was tested and confirmed. The quantitative model constitutes a concise construct of the functional organization of the hippocampal CA 1 region, which suggests further theoretical extensions and experimentation.     

Firing statistics of inhibitory neuron with delayed feedback. I. Output ISI probability density

Activity of inhibitory neuron with delayed feedback is considered in the framework of point stochastic processes. The neuron receives excitatory input impulses from a Poisson stream, and inhibitory impulses from the feedback line with a delay. We investigate here, how does the presence of inhibitory feedback affect the output firing statistics. Using binding neuron (BN) as a model, we derive analytically the exact expressions for the output interspike intervals (ISI) probability density, mean output ISI and coefficient of variation as functions of model's parameters for the case of threshold 2. Using the leaky integrate-and-fire (LIF) model, as well as the BN model with higher thresholds, these statistical quantities are found numerically. In contrast to the previously studied situation of no feedback, the ISI probability densities found here both for BN and LIF neuron become bimodal and have discontinuity of jump type. Nevertheless, the presence of inhibitory delayed feedback was not found to affect substantially the output ISI coefficient of variation. The ISI coefficient of variation found ranges between 0.5 and 1. It is concluded that introduction of delayed inhibitory feedback can radically change neuronal output firing statistics. This statistics is as well distinct from what was found previously (Vidybida and Kravchuk, 2009) by a similar method for excitatory neuron with delayed feedback.     

The influences of somatic and dendritic inhibition on bursting patterns in a neuronal circuit model

The balance between inhibition and excitation plays a crucial role in the generation of synchronous bursting activity in neuronal circuits. In human and animal models of epilepsy, changes in both excitatory and inhibitory synaptic inputs are known to occur. Locations and distribution of these excitatory and inhibitory synaptic inputs on pyramidal cells play a role in the integrative properties of neuronal activity, e.g., epileptiform activity. Thus the location and distribution of the inputs onto pyramidal cells are important parameters that influence neuronal activity in epilepsy. However, the location and distribution of inhibitory synapses converging onto pyramidal cells have not been fully studied. The objectives of this study are to investigate the roles of the relative location of inhibitory synapses on the dendritic tree and soma in the generation of bursting activity. We investigate influences of somatic and dendritic inhibition on bursting activity patterns in several paradigms of potential connections using a simplified multicompartmental model. We also investigate the effects of distribution of fast and slow components of GABAergic inhibition in pyramidal cells. Interspike interval (ISI) analysis is used for examination of bursting patterns. Simulations show that the inhibitory interneuron regulates neuronal bursting activity. Bursting behavior patterns depend on the synaptic weight and delay of the inhibitory connection as well as the location of the synapse. When the inhibitory interneuron synapses on the pyramidal neuron, inhibitory action is stronger if the inhibitory synapse is close to the soma. Alterations of synaptic weight of the interneuron can be compensatory for changes in the location of synaptic input. The relative changes in these parameters exert a considerable influence on whether synchronous bursting activity is facilitated or reduced. Additional simulations show that the slow GABAergic inhibitory component is more effective than the fast component in distal dendrites. Taken together, these findings illustrate the potential for GABAergic inhibition in the soma and dendritic tree to play an important modulatory role in bursting activity patterns.     

Computational simulation of the input-output relationship in hippocampal pyramidal cells

The precise mapping of how complex patterns of synaptic inputs are integrated into specific patterns of spiking output is an essential step in the characterization of the cellular basis of network dynamics and function. Relative to other principal neurons of the hippocampus, the electrophysiology of CA1 pyramidal cells has been extensively investigated. Yet, the precise input-output relationship is to date unknown even for this neuronal class. CA1 pyramidal neurons receive laminated excitatory inputs from three distinct pathways: recurrent CA1 collaterals on basal dendrites, CA3 Schaffer collaterals, mostly on oblique and proximal apical dendrites, and entorhinal perforant pathway on distal apical dendrites. We implemented detailed computer simulations of pyramidal cell electrophysiology based on three-dimensional anatomical reconstructions and compartmental models of available biophysical properties from the experimental literature. To investigate the effect of synaptic input on axosomatic firing, we stochastically distributed a realistic number of excitatory synapses in each of the three dendritic layers. We then recorded the spiking response to different stimulation patterns. For all dendritic layers, synchronous stimuli resulted in trains of spiking output and a linear relationship between input and output firing frequencies. In contrast, asynchronous stimuli evoked non-bursting spike patterns and the corresponding firing frequency input-output function was logarithmic. The regular/irregular nature of the input synaptic intervals was only reflected in the regularity of output inter-burst intervals in response to synchronous stimulation, and never affected firing frequency. Synaptic stimulations in the basal and proximal apical trees across individual neuronal morphologies yielded remarkably similar input-output relationships. Results were also robust with respect to the detailed distributions of dendritic and synaptic conductances within a plausible range constrained by experimental evidence. In contrast, the input-output relationship in response to distal apical stimuli showed dramatic differences from the other dendritic locations as well as among neurons, and was more sensible to the exact channel densities. Action Editor: Alain Destexhe     

Dynamical features of higher-order correlation events: impact on cortical cells

Cortical neurons receive signals from thousands of other neurons. The statistical properties of the input spike trains substantially shape the output response properties of each neuron. Experimental and theoretical investigations have mostly focused on the second order statistical features of the input spike trains (mean firing rates and pairwise correlations). Little is known of how higher order correlations affect the integration and firing behavior of a cell independently of the second order statistics. To address this issue, we simulated the dynamics of a population of 5000 neurons, controlling both their second order and higher-order correlation properties to reflect physiological data. We then used these ensemble dynamics as the input stage to morphologically reconstructed cortical cells (layer 5 pyramidal, layer 4 spiny stellate cell), and to an integrate and fire neuron. Our results show that changes done solely to the higher-order correlation properties of the network’s dynamics significantly affect the response properties of a target neuron, both in terms of output rate and spike timing. Moreover, the neuronal morphology and voltage dependent mechanisms of the target neuron considerably modulate the quantitative aspects of these effects. Finally, we show how these results affect sparseness of neuronal representations, tuning properties, and feature selectivity of cortical cells. An erratum to this article can be found at     

A hierarchical neural network model for associative memory

A hierarchical neural network model with feedback interconnections, which has the function of associative memory and the ability to recognize patterns, is proposed. The model consists of a hierarchical multi-layered network to which efferent connections are added, so as to make positive feedback loops in pairs with afferent connections. The cell-layer at the initial stage of the network is the input layer which receives the stimulus input and at the same time works as an output layer for associative recall. The deepest layer is the output layer for pattern-recognition. Pattern-recognition is performed hierarchically by integrating information by converging afferent paths in the network. For the purpose of associative recall, the integrated information is again distributed to lower-order cells by diverging efferent paths. These two operations progress simultaneously in the network. If a fragment of a training pattern is presented to the network which has completed its self-organization, the entire pattern will gradually be recalled in the initial layer. If a stimulus consisting of a number of training patterns superposed is presented, one pattern gradually becomes predominant in the recalled output after competition between the patterns, and the others disappear. At about the same time when the recalled pattern reaches a steady state in he initial layer, in the deepest layer of the network, a response is elicited from the cell corresponding to the category of the finally-recalled pattern. Once a steady state has been reached, the response of the network is automatically extinguished by inhibitory signals from a steadiness-detecting cell. If the same stimulus is still presented after inhibition, a response for another pattern, formerly suppressed, will now appear, because the cells of the network have adaptation characteristics which makes the same response unlikely to recur. Since inhibition occurs repeatedly, the superposed input patterns are recalled one by one in turn.     

Long-latency responses to conditioned stimulus in the cat motor cortex can be generated through the enhancement of efficiency excitatory collaterals of pyramidal neuron

The long-latency excitatory components are the characteristic feature of neuronal responses to conditional stimuli in the motor cortex of the cat. The data presented suggest that the neuronal machine that generates these reactions is that, generating long-latency epileptiform discharges in epileptogenic cortex. The long-latency component generation is based on NMDA-receptor activation in the recurrent excitatory collaterals of the cortical pyramidal neurons. The response delay is dependent on initial activation of inhibitory GABA(A) receptors. The emergence of the late components in the course of motor learning take place as a result of efficiency enhancement of recurrent collaterals synaptic linkage with pyramidal neurons.     

A computer model of generation of the motor cortex neuron processes seen in the course of execution of instrumental movement

A computer model of neuronal processes in the motor cortex column is presented. The model is consisted of two pyramidal cell layers with two groups of inhibitory interneurons, selectively controlling pyramidal cell soma and dendrite, in each. Active Na, Ca and K conductances are included in the model of a single neuron. Horizontal excitatory connections between pyramidal cells in the upper layer are largely of NMDA-receptor type, that in the lower layer--of non-NMDA-type. All inhibitory synapses are of GABA(A)-type. The model reproduces the main phenomenon observed in the motor cortex during the execution of conditioned movements. Consequent to an early excitation the upper layer pyramidal cells generate a late NMDA-dependent reflexive response to afferent conditional stimulation, which as in a real case is diminished by GABA(A)-type synaptic inhibition and afferent stimulus strength increase. The characteristic inverse relation between the late response manifestation and the stimulus strength observed in the real cortex can be reproduced in the model only if NMDA-glutamate receptors were preferentially localized in the terminals of pyramidal cell backward collaterals, not in the terminals of the afferent fibers on pyramidal neurons. The intended component of motor cortex neuronal activity is generated in NMDA-independent manner by the pyramidal cells of lower layer. The slow time coarse of intended component as compared with short duration of AMPA epsp's is due to a consecutive relay-race--like activation of pyramidal neurons with different dendrit-to-soma ratio.