A template matching model for pattern recognition: Self-organization of templates and template matching by a disinhibitory neural network

A template matching model for pattern recognition is proposed. By following a previouslyproposed algorithm for synaptic modification (Hirai, 1980), the template of a stimulus pattern is selforganized as a spatial distribution pattern of matured synapses on the cells receiving modifiable synapses. Template matching is performed by the disinhibitory neural network cascaded beyond the neural layer composed of the cells receiving the modifiable synapses. The performance of the model has been simulated on a digital computer. After repetitive presentations of a stimulus pattern, a cell receiving the modifiable synapses comes to have the template of that pattern. And the cell in the latter layer of the disinhibitory bitory neural network that receives the disinhibitory input from that cell becomes electively sensitive to that pattern. Learning patterns are not restricted by previously learned ones. They can be subset or superset patterns of the ones previously learned. If an unknown pattern is presented to the model, no cell beyond the disinhibitory neural network will respond. However, if previously learned patterns are embedded in that pattern, the cells which have the templates of those patterns respond and are assumed to transmit the information to higher center. The computer simulation also shows that the model can organize a clean template under a noisy environment.     

Intrinsic versus extrinsic influences in the development of neuronal maps

Accumulating evidence suggests that the plasticity of extrinsic thalamocortical inputs in cortical layer IV may be guided or instructed by earlier plasticity events in the intrinsic, horizontal connections within the extragranular cortical layers. We analyse a rate-based model of the plasticity of a set of extrinsic afferents in the presence of a pre-existing (and fixed) plexus of intrinsic, overall excitatory horizontal connections between a set of target neurons. We determine conditions under which afferent synaptic pattern formation respects this pre-existing lateral structure. We find three broad regimes under which extrinsic afferent plasticity may violate this structure: the initial pattern of extrinsic afferent innervation of the target cells is far from balanced; the gain of the extrinsic afferents greatly exceeds the overall scale of the strength of lateral excitation; the target cell horizontal coupling matrix is sparse. If none of these conditions is satisfied, then extrinsic afferent plasticity respects the pre-existing lateral connectivity, so that afferent synaptic pattern formation conforms to the pattern of lateral excitation.     

A role of the basal ganglia in the occurrence of visual hallucinations (a hypothetical mechanism)

A hypothetical mechanism of the basal ganglia involvement in visual hallucinations is proposed. According to this mechanism, hallucination is the result of modulation of the efficacy of corticostriatal synaptic inputs and changes in spiny cell activity due to the rise of striatal dopamine concentration (or due to other reasons). These changes cause an inhibition of neurons in the substantia nigra pars reticulata and subsequent disinhibition of neurons in the superior colliculus and pedunculopontine nucleus (including its cholinergic cells). In the absence of afferentation from the retina this disinhibition leads to activation of neurons in the lateral geniculate nucleus, pulvinar and other thalamic nuclei projecting to the primary and highest visual cortical areas, prefrontal cortex, and also back to the striatum. Hallucinations as conscious visual patterns are the result of selection of signals circulating in several interconnected loops each of which includes one of above mentioned neocortical areas, one of thalamic nuclei, limbic and one of visual areas of the basal ganglia, superior colliculus and/or pedunculopontine nucleus. According to our model, cannabinoids, opioids and ketamine may lead to hallucinations due to their promotional role in the LTD of cortical inputs to GABAergic spiny cells of striatal striosomes projecting to dopaminergic neurons, disinhibition of the lasts, and increase in striatal dopamine concentration.     

Experience-Driven Axon Retraction in the Pharmacologically Inactivated Visual Cortex Does Not Require Synaptic Transmission

Background

Experience during early postnatal development plays an important role in the refinement of specific neural connections in the brain. In the mammalian visual system, altered visual experiences induce plastic adaptation of visual cortical responses and guide rearrangements of afferent axons from the lateral geniculate nucleus. Previous studies using visual deprivation demonstrated that the afferents serving an open eye significantly retract when cortical neurons are pharmacologically inhibited by applying a γ-aminobutyric acid type A receptor agonist, muscimol, whereas those serving a deprived eye are rescued from retraction, suggesting that presynaptic activity can lead to the retraction of geniculocortical axons in the absence of postsynaptic activity. Because muscimol application suppresses the spike activity of cortical neurons leaving transmitter release intact at geniculocortical synapses, local synaptic interaction may underlie the retraction of active axons in the inhibited cortex.

Method and Findings

New studies reported here determined whether experience-driven axon retraction can occur in the visual cortex inactivated by blocking synaptic inputs. We inactivated the primary visual cortex of kittens by suppressing synaptic transmission with cortical injections of botulinum neurotoxin type E, which cleaves a synaptic protein, SNAP-25, and blocks transmitter release, and examined the geniculocortical axon morphology in the animals with normal vision and those deprived of vision binocularly. We found that afferent axons in the animals with normal vision showed a significant retraction in the inactivated cortex, as similarly observed in the muscimol-treated cortex, whereas the axons in the binocularly deprived animals were preserved.

Conclusions

Therefore, the experience-driven axon retraction in the inactivated cortex can proceed in the absence of synaptic transmission. These results suggest that presynaptic mechanisms play an important role in the experience-driven refinement of geniculocortical axons.     

The Possible Role of Spike Patterns in Cortical Information Processing

When the same visual stimulus is presented across many trials, neurons in the visual cortex receive stimulus-related synaptic inputs that are reproducible across trials (S) and inputs that are not (N). The variability of spike trains recorded in the visual cortex and their apparent lack of spike-to-spike correlations beyond that implied by firing rate fluctuations, has been taken as evidence for a low S/N ratio. A recent re-analysis of in vivo cortical data revealed evidence for spike-to-spike correlations in the form of spike patterns. We examine neural dynamics at a higher S/N in order to determine what possible role spike patterns could play in cortical information processing. In vivo-like spike patterns were obtained in model simulations. Superpositions of multiple sinusoidal driving currents were especially effective in producing stable long-lasting patterns. By applying current pulses that were either short and strong or long and weak, neurons could be made to switch from one pattern to another. Cortical neurons with similar stimulus preferences are located near each other, have similar biophysical properties and receive a large number of common synaptic inputs. Hence, recordings of a single neuron across multiple trials are usually interpreted as the response of an ensemble of these neurons during one trial. In the presence of distinct spike patterns across trials there is ambiguity in what would be the corresponding ensemble, it could consist of the same spike pattern for each neuron or a set of patterns across neurons. We found that the spiking response of a neuron receiving these ensemble inputs was determined by the spike-pattern composition, which, in turn, could be modulated dynamically as a means for cortical information processing.     

Opponent inhibition: a developmental model of layer 4 of the neocortical circuit.

We model the development of the functional circuit of layer 4 (the input-recipient layer) of cat primary visual cortex. The observed thalamocortical and intracortical circuitry codevelop under Hebb-like synaptic plasticity. Hebbian development yields opponent inhibition: inhibition evoked by stimuli anticorrelated with those that excite a cell. Strong opponent inhibition enables recognition of stimulus orientation in a manner invariant to stimulus contrast. These principles may apply to cortex more generally: Hebb-like plasticity can guide layer 4 of any piece of cortex to create opposition between anticorrelated stimulus pairs, and this enables recognition of specific stimulus patterns in a manner invariant to stimulus magnitude. Properties that are invariant across a cortical column are predicted to be those shared by opponent stimulus pairs; this contrasts with the common idea that a column represents cells with similar response properties.     

A neural network model for the self-organization of cortical grating cells

A neural network model with incremental Hebbian learning of afferent and lateral synaptic couplings is proposed,which simulates the activity-dependent self-organization of grating cells in upper layers of striate cortex. These cells, found in areas V1 and V2 of the visual cortex of monkeys, respond vigorously and exclusively to bar gratings of a preferred orientation and periodicity. Response behavior to varying contrast and to an increasing number of bars in the grating show threshold and saturation effects. Their location with respect to the underlying orientation map and their nonlinear response behavior are investigated. The number of emerging grating cells is controlled in the model by the range and strength of the lateral coupling structure.     

An incremental Hebbian learning model of the primary visual cortex with lateral plasticity and real input patterns

We present a simplified binocular neural network model of the primary visual cortex with separate ON/OFF-pathways and modifiable afferent as well as intracortical synaptic couplings. Random as well as natural image stimuli drive the weight adaptation which follows Hebbian learning rules stabilized with constant norm and constant sum constraints. The simulations consider the development of orientation and ocular dominance maps under different conditions concerning stimulus patterns and lateral couplings. With random input patterns realistic orientation maps with +/- 1/2-vortices mostly develop and plastic lateral couplings self-organize into mexican hat type structures on average. Using natural greyscale images as input patterns, realistic orientation maps develop as well and the lateral coupling profiles of the cortical neurons represent the two point correlations of the input image used.     

The ultrastructural differentiation of synaptic sites in the inner plexiform layer of a teleostean retina.

The differentiation of axon terminals in the retina inner plexiform layer was studied by electron microscopy, with special reference to synaptic junctions, number and size of synaptic vesicles, dense core vesicles, biogenic amines and ATPase. Five types of synaptic junctions were found, including a ribbon type. They appear on different days of embryonic life and show different patterns of increase. The ultrastructural differentiation of the most frequent type is described in detail. The numbers of synaptic vesicles indicate the existence of three types of axon terminals which appear on different days. These and other data lead to the following model: The contacts between amacrine cells are the first to appear, followed by amacrine--bipolar cell contacts and amacrine--ganglion cell contacts. The latter are the most frequent ones and increase immediately after having appeared, while amacrine--bipolar cell contacts increase only some days later, which is also the case for bipolar--amacrine cell contacts. Biogenic amines and cytochemically detectable ATPase appear along with the formation of synaptic sites.     

Patterns of correlated spontaneous bursting activity in the developing mammalian retina

The adult visual system is highly organized in its patterns of connectivity. Connections between the retina and its central target, the dorsal lateral geniculate nucleus (dLGN), are remodeled during development as inappropriate synaptic inputs are eliminated by a process that requires retinal activity. Multineuronal recordings of the neonatal ferret retina reveal that during the refinement period, retinal ganglion cells spontaneously display rhythmic bursting activity in which the bursts of neighboring cells are correlated by propagating excitatory waves. These spontaneous retinal waves have temporal and spatial properties that appear instructive for the refinement of the early patterns of retinogeniculate connections prior to visual stimulation.     

Golgi studies of the first optic ganglion of the ant,Cataglyphis bicolor

The neurons of the first optic ganglion (the lamina) in the desert ant, Cataglyphis bicolor, have been studied with the light microscope after Golgi silver impregnation. The different types of retinal and laminal fibres and their configuration are compared with the results obtained in the bee. The first synaptic region in the visual system of the ant lies proximally to the fenestrated layer below the basement membrane and the layer containing the monopolar cell bodies. The synaptic region can be separated into three morphologically different zones: (1) The most distal layer where the short visual fibres end at two different levels. The short visual fibres and some laminal fibres (monopolar cell fibres) also show lateral elements in this region. (2) The second layer appears almost free of branches of retinal and laminal fibres. (3) The most proximal layer, which has a characteristically dense horizontal structure resulting from the lateral elements of long visual, centrifugal, monopolar and tangential fibres. Nine cell axons arising from each ommatidium leave the retina. Six of these are short visual fibres and end at two different levels in the lamina. Three different types of short visual fibres can be distinguished by their different terminal depths and lateral branching pattern. The remaining three fibres, the long visual fibres, terminate in the medulla. They can be distinguished from each other by their lateral elements in the lamina neuropile. The five morphologically different laminal fibre types (axons of the monopolar cells in the lamina) have different shapes and different arborizations at different levels. Tangential, centrifugal and incerta sedis-fibres, which originate either from cell bodies in the cell body layer at the periphery of the outer chiasma or more centrally, terminate in the synaptic region of the lamina. Consideration is given to the clearly demarkated arrangement and length of the branching pattern of retinal and laminal fibres at different levels of the synaptic region of the lamina. In addition, a hypothetical connectivity pattern is discussed.     

Connectivity and dynamics in the olfactory bulb

Dendrodendritic interactions between excitatory mitral cells and inhibitory granule cells in the olfactory bulb create a dense interaction network, reorganizing sensory representations of odors and, consequently, perception. Large-scale computational models are needed for revealing how the collective behavior of this network emerges from its global architecture. We propose an approach where we summarize anatomical information through dendritic geometry and density distributions which we use to calculate the connection probability between mitral and granule cells, while capturing activity patterns of each cell type in the neural dynamical systems theory of Izhikevich. In this way, we generate an efficient, anatomically and physiologically realistic large-scale model of the olfactory bulb network. Our model reproduces known connectivity between sister vs. non-sister mitral cells; measured patterns of lateral inhibition; and theta, beta, and gamma oscillations. The model in turn predicts testable relationships between network structure and several functional properties, including lateral inhibition, odor pattern decorrelation, and LFP oscillation frequency. We use the model to explore the influence of cortex on the olfactory bulb, demonstrating possible mechanisms by which cortical feedback to mitral cells or granule cells can influence bulbar activity, as well as how neurogenesis can improve bulbar decorrelation without requiring cell death. Our methodology provides a tractable tool for other researchers.     

The distribution of orientation of optimal stimuli for cells of striate cortex

The mechanism mediating the adaptation of cortical Area 17 oriented line detector cells is modeled. A novel form of environment and convergence property (which places a requirement on distribution of optimal stimuli) is utilized to show that classical concepts of synaptic efficiency loss due only to disuse are inadequate under the convergence property. A reasonable alternative is presented: a form of synaptic control which reduces synaptic efficiency at non-active synapses as a function of cell firing rate. We show that adequate solutions to our convergence property exist for members of this class. Simulations of this adequate mechanism indicate that under the very disjoint environment defined, small perturbations of the environment's distribution of stimuli may lead to large perturbations of the distribution of stimuli to which Area 17 cells are optimally responsive. A simulation indicates that this effect may be accentuated by a lateral interaction which causeslike cell optimal stimuli to form near each other in cortex; and the effect may be reduced by a lateral interaction which causes like cell optimal stimuli to not form near each other. A form of neurophysiological experiment is suggested for verification.     

The anatomy of geniculocortical connections in monocularly deprived cats

In monocularly deprived (MD) cats, many cells in the lateral geniculate nucleus (LGN) but few cells in the visual cortex respond to input from the deprived eye, suggesting that the connections to visual cortex from the deprived geniculate laminae may have been disrupted. It has been known for some time that the afferents representing the deprived eye terminate over a smaller percentage of layer IV than do those representing the experienced eye, but it is becoming increasingly clear that this alone cannot explain the inability of the deprived pathway to activate cortical cells. 2-Deoxyglucose studies of ocular dominance columns in MD cats have shown that the columns are often (a) restricted to layer IV, suggesting that intracortical connections may be disrupted, and (b) very faint, suggesting that MD alters the efficacy of the deprived pathway in addition to restricting its territory. Electron microscopy has shown that both deprived and experienced afferents end in terminals that contain mitochondria and round synaptic vesicles and that make asymmetric contacts with dendritic profiles. However, the terminals of deprived afferents differ from those of experienced afferents: they are 25% smaller, contain 33% fewer mitochondria, are more likely to make synapses that are presynaptically convex (and thus, perhaps, immature), make fewer perforated synapses, and synapse onto smaller spines. Further, the geniculocortical axons from deprived laminae appear to end in fewer synaptic terminals, than do those from the experienced laminae. The finding that the synaptic terminals of deprived afferents are both abnormal morphologically and fewer in number can help to explain the reduced effectiveness of the deprived eye in driving cortical cells but does not rule out additional effects such as suppression and loss of intracortical connectivity.     

Corticofugal feedback can reduce the visual latency of responses to antagonistic stimuli

 A biophysically realistical model of the primary visual pathway is designed, including feedback connections from the visual cortex to the lateral geniculate nucleus (LGN) – the so-called corticofugal pathway. The model comprises up to 10 000 retina and LGN cells divided into the ON and the OFF pathway according to their contrast response characteristics. An additional 6000 cortical simple cells are modeled. Apart from the direct excitatory afferent pathway we include strong mutual inhibition between the ON and the OFF subsystems. In addition, we propose a novel type of paradoxical corticofugal connection pattern which links ON dominated cortical simple cells to OFF-center LGN cells and vice versa. In accordance with physiological findings these connections are weakly excitatory and do not interfere with the steady-state responses to constant illumination, because during the steady-state inhibition arising from the active pathway effectively silences the nonstimulated pathway. At the moment of a contrast reversal the effect of the paradoxical connection pattern comes into play and the depolarization of the previously silent channel is accelerated, leading to a latency reduction of up to 4 ms using moderate synaptic weights. With increased weights reductions of more than 10 ms can be achieved. We introduce different synaptic characteristics for the feedback (AMPA, NMDA, AMPA+NMDA) and show that the strongest latency reduction is obtained for a combination of the membrane channels (i.e., AMPA+NMDA). The effect of the proposed paradoxical connection pattern is self-regulating; because the levels of inhibition and paradoxical excitation are always driven by the same inputs (strong inhibition is counterbalanced by a stronger paradoxical excitation and vice versa). In addition, the latency reduction for a contrast inversion which ends at a small absolute contrast level (small contrast step) is stronger than the reduction for an inversion with large final contrast (large contrast step). This leads to a more pronounced reduction in the reaction times for weak stimuli. Thus, reaction time differences for different contrast steps are smoothed out. Received: 22 January 1996/Accepted in revised form: 20 May 1996     

Role of the basal ganglia in the occurrence of paradoxical sleep dreams (hypothetical mechanism)

A hypothetical mechanism of the basal ganglia involvement in the occurrence of paradoxical sleep dreams and rapid eye movements is proposed. According to this mechanism, paradoxical sleep is provided by facilitation of activation of cholinergic neurons in the pedunculopontine nucleus as a result of suppression of their inhibition from the output basal ganglia nuclei. This disinhibition is promoted by activation of dopaminergic cells by pedunculopontine neurons, subsequent rise in dopamine concentration in the input basal ganglia structure. striatum, and modulation of the efficacy of cortico-striatal inputs. In the absence of signals from retina, a disinhibition of neurons in the pedunculopontine nucleus and superior colliculus allows them to excite neurons in the lateral geniculate body and other thalamic nuclei projecting to the primary and higher visual cortical areas, prefrontal cortex and back into the striatum. Dreams as visual images and "motor hallucinations" are the result of an increase in activity of definitely selected groups of thalamic and neocortical neurons. This selection is caused by modifiable action of dopamine on long-term changes in the efficacy of synaptic transmission during circulation of signals in closed interconnected loops, each of which includes one of the visual cortical areas (motor cortex), one of the thalamic nuclei, limbic and one of the visual areas (motor area) of the basal ganglia. pedunculopontine nucleus, and superior colliculus. Simultaneous modification and modulation of synapses in diverse units of neuronal loops is provided by PGO waves. Disinhibition of superioir colliculus neurons and their excitation by pedunculopontine nucleus lead to an appearance of rapid eye movements during paradoxical sleep.     

Glial cell development and function in the Drosophila visual system

In the developing nervous system, building a functional neuronal network relies on coordinating the formation, specification and survival to diverse neuronal and glial cell subtypes. The establishment of neuronal connections further depends on sequential neuron-neuron and neuron-glia interactions that regulate cell-migration patterns and axon guidance. The visual system of Drosophila has a highly regular, retinotopic organization into reiterated interconnected synaptic circuits. It is therefore an excellent invertebrate model to investigate basic cellular strategies and molecular determinants regulating the different developmental processes that lead to network formation. Studies in the visual system have provided important insights into the mechanisms by which photoreceptor axons connect with their synaptic partners within the optic lobe. In this review, we highlight that this system is also well suited for uncovering general principles that underlie glial cell biology. We describe the glial cell subtypes in the visual system and discuss recent findings about their development and migration. Finally, we outline the pivotal roles of glial cells in mediating neural circuit assembly, boundary formation, neural proliferation and survival, as well as synaptic function.     

Neurons and their synaptic organization in the visual cortex of the rat

Summary Cells in the visual cortex (area 17) of adult rats were impregnated by the rapid Golgi method and characterized by light microscopy. Selected cells were then sectioned for electron microscopy and their cytological characteristics and the pattern of synapses on their cell bodies and dendrites were studied Twelve classical pyramidal cells from layers II–VI, two pyramid-like cells from layer VI, two inverted pyramidal cells from layers V and VI, ten spine-free non-pyramidal cells from layers II–VI and two spinous non-pyramidal cells from layer IV were examined.The cytoplasmic features of the identified cells, where these could be discerned, corresponded to those previously reported for the different cell types in conventionally prepared tissue. Pyramidal Cells received exclusively type 2 synaptic contacts on their cell bodies, type 1 contacts on their dendritic spines and a mixture of synaptic types (type II predominating) on their shafts, where synaptic density was relatively low. This pattern of synaptic contacts was consistent for all portions of the dendritic tree; inverted pyramidal cells and pyramid-like cells showed the same synaptic organization as classical pyramids. The axon collaterals of pyramidal cells established type I contacts with dendritic spines (or, rarely, shafts) of unknown origin. Non-Pyramidal Cells received both type 1 and type 2 contacts (the former predominating) on their cell bodies and dendrites. The spinous variety also received type I contacts on their dendritic spines. Axon terminal of spine-free non-pyramidal cells established type II synaptic contacts with dendritic shafts of unknown origin. The similarity in synaptic organization between the spine-free and spinous non-pyramidal cells examined in this study suggest that the latter correspond to the sparsely spinous stellate cells rather than to the spinous stellate cells of cat and monkey visual cortex.We thank the Medical Research Council for financial support     

A theory for the acquisition and loss of neuron specificity in visual cortex

We assume that between lateral geniculate and visual cortical cells there exist labile synapses that modify themselves in a new fashion called threshold passive modification and in addition, non-labile synapses that contain permanent information. In the theory which results there is an increase in the specificity of response of a cortical cell when it is exposed to stimuli due to normal patterned visual experience. Non-patterned input, such as might be expected when an animal is dark-reared or raised with eyelids sutured, results in a loss of specificity, with details depending on whether noise to labile and non-labile junctions is correlated. Specificity can sometimes be regained, however, with a return of input due to patterned vision. We propose that this provides a possible explanation of experimental results obtained by Imbert and Buisseret (1975); Blakemore and Van Sluyters (1975); Buisseret and Imbert (1976); and Frégnac and Imbert (1977, 1978).This work was supported in part by a grant from the Ittleson Foundation, Inc.     

Dynamics of pruning in simulated large-scale spiking neural networks

Massive synaptic pruning following over-growth is a general feature of mammalian brain maturation. This article studies the synaptic pruning that occurs in large networks of simulated spiking neurons in the absence of specific input patterns of activity. The evolution of connections between neurons were governed by an original bioinspired spike-timing-dependent synaptic plasticity (STDP) modification rule which included a slow decay term. The network reached a steady state with a bimodal distribution of the synaptic weights that were either incremented to the maximum value or decremented to the lowest value. After 1x10(6) time steps the final number of synapses that remained active was below 10% of the number of initially active synapses independently of network size. The synaptic modification rule did not introduce spurious biases in the geometrical distribution of the remaining active projections. The results show that, under certain conditions, the model is capable of generating spontaneously emergent cell assemblies.