Modelling studies on the computational function of fast temporal structure in cortical circuit activity

The interplay between modelling and experimental studies can support the exploration of the function of neuronal circuits in the cortex. We exemplify such an approach with a study on the role of spike timing and gamma-oscillations in associative memory in strongly connected circuits of cortical neurones. It is demonstrated how associative memory studies on different levels of abstraction can specify the functionality to be expected in real cortical neuronal circuits. In our model overlapping random configurations of sparse cell populations correspond to memory items that are stored by simple Hebbian coincidence learning. This associative memory task will be implemented with biophysically well tested compartmental neurones developed by Pinsky and Rinzel . We ran simulation experiments to study memory recall in two network architectures: one interconnected pool of cells, and two reciprocally connected pools. When recalling a memory by stimulating a spatially overlapping set of cells, the completed pattern is coded by an event of synchronized single spikes occurring after 25-60 ms. These fast associations are performed even at a memory load corresponding to the memory capacity of optimally tuned formal associative networks (>0.1 bit/synapse). With tonic stimulation or feedback loops in the network the neurones fire periodically in the gamma-frequency range (20-80 Hz). With fast changing inputs memory recall can be switched between items within a single gamma cycle. Thus, oscillation is not a primary coding feature necessary for associative memory. However, it accompanies reverberatory feedback providing an improved iterative memory recall completed after a few gamma cycles (60-260 ms). In the bidirectional architecture reverberations do not express in a rigid phase locking between the pools. For small stimulation sets bursting occurred in these cells acting as a supportive mechanism for associative memory.     

Low-frequency stimulation induces stable transitions in stereotypical activity in cortical networks

Reverberating spontaneous synchronized brain activity is believed to play an important role in neural information processing. Whether and how external stimuli can influence this spontaneous activity is poorly understood. Because periodic synchronized network activity is also prominent in in vitro neuronal cultures, we used cortical cultures grown on multielectrode arrays to examine how spontaneous activity is affected by external stimuli. Spontaneous network activity before and after low-frequency electrical stimulation was quantified in several ways. Our results show that the initially stable pattern of stereotypical spontaneous activity was transformed into another activity pattern that remained stable for at least 1 h. The transformations consisted of changes in single site and culture-wide network activity as well as in the spatiotemporal dynamics of network bursting. We show for the first time that low-frequency electrical stimulation can induce long-lasting alterations in spontaneous activity of cortical neuronal networks. We discuss whether the observed transformations in network activity could represent a switch in attractor state.     

Associative sequential recall in a synaptic matrix

An adaptive neuronal network (synaptic matrix) was simulated. After learning a number of randomly generated scenes, the network was tested for associative sequential recall in a stimulus-bound mode and in an image-bound mode. It is demonstrated that neuronal mechanisms of this kind can exhibit orderliness or looseness of associative response suggestive of human recall behavior.     

Neural representation of visual objects: encoding and top-down activation

Knowledge or experiences are voluntarily recalled from memory by reactivation of their neural representations in the association cortex. Mnemonic representations of visual objects, located in the ventral processing stream of visual perception, provide the best indication of how neuronal codes are created, organized and reactivated. Associative codes are created by neurons that have the ability to link the representations of temporally associated stimuli. Recent experiments suggest that not only bottom-up signals from the retina but also top-down signals from the prefrontal cortex can trigger the retrieval of associative codes, which may serve as a neural basis for conscious recall.     

Network Evolution Induced by Asynchronous Stimuli through Spike-Timing-Dependent Plasticity

In sensory neural system, external asynchronous stimuli play an important role in perceptual learning, associative memory and map development. However, the organization of structure and dynamics of neural networks induced by external asynchronous stimuli are not well understood. Spike-timing-dependent plasticity (STDP) is a typical synaptic plasticity that has been extensively found in the sensory systems and that has received much theoretical attention. This synaptic plasticity is highly sensitive to correlations between pre- and postsynaptic firings. Thus, STDP is expected to play an important role in response to external asynchronous stimuli, which can induce segregative pre- and postsynaptic firings. In this paper, we study the impact of external asynchronous stimuli on the organization of structure and dynamics of neural networks through STDP. We construct a two-dimensional spatial neural network model with local connectivity and sparseness, and use external currents to stimulate alternately on different spatial layers. The adopted external currents imposed alternately on spatial layers can be here regarded as external asynchronous stimuli. Through extensive numerical simulations, we focus on the effects of stimulus number and inter-stimulus timing on synaptic connecting weights and the property of propagation dynamics in the resulting network structure. Interestingly, the resulting feedforward structure induced by stimulus-dependent asynchronous firings and its propagation dynamics reflect both the underlying property of STDP. The results imply a possible important role of STDP in generating feedforward structure and collective propagation activity required for experience-dependent map plasticity in developing in vivo sensory pathways and cortices. The relevance of the results to cue-triggered recall of learned temporal sequences, an important cognitive function, is briefly discussed as well. Furthermore, this finding suggests a potential application for examining STDP by measuring neural population activity in a cultured neural network.     

Associative recall of images

Orthogonal projection operations in a linear vector space are shown to have a close relation to the processes by which optimal associative recall of patterned information can be implemented. Two association schemes, the autoassociative mapping and the associative encoding, are introduced. The latter has a bearing on pattern recognition, especially in the recognition of an optical image from a small fragment of it. Analytical expressions for the quality of the recollections are derived, and computerized demonstrations of associative recall with 3024-element optical images are presented. Some preprocessing principles of the images are studied, whereby the two-dimensional Laplacian is found very effective. This finding may have some connection to lateral inhibition effects occurring in biological visual systems.     

NMDA receptors are not required for pattern completion during associative memory recall

Pattern completion, the ability to retrieve complete memories initiated by subsets of external cues, has been a major focus of many computation models. A previously study reports that such pattern completion requires NMDA receptors in the hippocampus. However, such a claim was derived from a non-inducible gene knockout experiment in which the NMDA receptors were absent throughout all stages of memory processes as well as animal's adult life. This raises the critical question regarding whether the previously described results were truly resulting from the requirement of the NMDA receptors in retrieval. Here, we have examined the role of the NMDA receptors in pattern completion via inducible knockout of NMDA receptors limited to the memory retrieval stage. By using two independent mouse lines, we found that inducible knockout mice, lacking NMDA receptor in either forebrain or hippocampus CA1 region at the time of memory retrieval, exhibited normal recall of associative spatial reference memory regardless of whether retrievals took place under full-cue or partial-cue conditions. Moreover, systemic antagonism of NMDA receptor during retention tests also had no effect on full-cue or partial-cue recall of spatial water maze memories. Thus, both genetic and pharmacological experiments collectively demonstrate that pattern completion during spatial associative memory recall does not require the NMDA receptor in the hippocampus or forebrain.     

On the perception of probable things: neural substrates of associative memory, imagery, and perception

Perception is influenced both by the immediate pattern of sensory inputs and by memories acquired through prior experiences with the world. Throughout much of its illustrious history, however, study of the cellular basis of perception has focused on neuronal structures and events that underlie the detection and discrimination of sensory stimuli. Relatively little attention has been paid to the means by which memories interact with incoming sensory signals. Building upon recent neurophysiological/behavioral studies of the cortical substrates of visual associative memory, I propose a specific functional process by which stored information about the world supplements sensory inputs to yield neuronal signals that can account for visual perceptual experience. This perspective represents a significant shift in the way we think about the cellular bases of perception.     

Embedding Responses in Spontaneous Neural Activity Shaped through Sequential Learning

Recent experimental measurements have demonstrated that spontaneous neural activity in the absence of explicit external stimuli has remarkable spatiotemporal structure. This spontaneous activity has also been shown to play a key role in the response to external stimuli. To better understand this role, we proposed a viewpoint, “memories-as-bifurcations,” that differs from the traditional “memories-as-attractors” viewpoint. Memory recall from the memories-as-bifurcations viewpoint occurs when the spontaneous neural activity is changed to an appropriate output activity upon application of an input, known as a bifurcation in dynamical systems theory, wherein the input modifies the flow structure of the neural dynamics. Learning, then, is a process that helps create neural dynamical systems such that a target output pattern is generated as an attractor upon a given input. Based on this novel viewpoint, we introduce in this paper an associative memory model with a sequential learning process. Using a simple Hebbian-type learning, the model is able to memorize a large number of input/output mappings. The neural dynamics shaped through the learning exhibit different bifurcations to make the requested targets stable upon an increase in the input, and the neural activity in the absence of input shows chaotic dynamics with occasional approaches to the memorized target patterns. These results suggest that these dynamics facilitate the bifurcations to each target attractor upon application of the corresponding input, which thus increases the capacity for learning. This theoretical finding about the behavior of the spontaneous neural activity is consistent with recent experimental observations in which the neural activity without stimuli wanders among patterns evoked by previously applied signals. In addition, the neural networks shaped by learning properly reflect the correlations of input and target-output patterns in a similar manner to those designed in our previous study.     

Dynamic Spatial Organization of Receptive Fields of Neurons in the 21a Cortical Area

We studied changes in the spatial parameters of receptive fields (RFs) of visually sensitive neurons in the associative area 21a of the cat cortex under conditions of presentation of moving visual stimuli. The results of experiments demonstrated that these parameters are dynamic and depend, from many aspects, on the pattern of the stimulus used for their estimation. Angular lengths of the horizontal and vertical axes of the RFs measured in the case of movement of the visual stimuli exceeded many times those determined by presentation of stationary blinking stimuli. As is supposed, a visual stimulus, when moving along the field of vision, activates a certain number of the neurons synaptically connected with the examined cell and possessing RFs localized along the movement trajectory. As a result, such integrated activity of the neuronal group can change the excitation threshold and discharge frequency of the studied neuron. It seems probable that correlated directed activation of the neuronal groups represents a significant neurophysiological mechanism providing dynamic modifications of the RF parameters of visually sensitive neurons in the course of processes of visual perception and identification of moving objects within the field of vision.     

Low-Dimensional Dynamics in Sensory Biology 1: Thermally Sensitive Electroreceptors of the Catfish

We report the results of a search for evidence of periodic unstableorbits in the electroreceptors of the catfish. The function of thesereceptor organs is to sense weak external electric fields. Inaddition, they respond to the ambient temperature and to the ioniccomposition of the water. These quantities are encoded by receptorsthat make use of an internal oscillator operating at the level of themembrane potential. If such oscillators have three or more degreesof freedom, and at least one of which also exhibits a nonlinearity,they are potentially capable of chaotic dynamics. By detecting theexistence of stable and unstable periodic orbits, we demonstratebifurcations between noisy stable and chaotic behavior using theambient temperature as a parameter. We suggest that the techniquedeveloped herein be regarded as an additional tool for the analysisof data in sensory biology and thus can be potentially useful instudies of functional responses to external stimuli. We speculatethat the appearance of unstable orbits may be indicative of a stateof heightened sensory awareness by the animal.     

Associative Memory and Segmentation in an Oscillatory Neural Model of the Olfactory Bulb

We discuss the first few stages of olfactory processing in the framework of a layered neural network. Its central component is an oscillatory associative memory, describing the external plexiform layer, that consists of inhibitory and excitatory neurons with dendrodendritic interactions. We explore the computational properties of this neural network and point out its possible functional role in the olfactory bulb. When receiving a complex input that is composed of several odors, the network segments it into its components. This is done in two stages. First, multiple odor input is preprocessed in the glomerular layer via a decorrelation mechanism that relies on temporal independence of odor sources. Second, as the recall process of a pattern consists of associative convergence to an oscillatory attractor, multiple inputs are identified by alternate dominance of memory patterns during different sniff cycles. This could explain how quick analysis of mixed odors is subserved by the rapid sniffing behavior of highly olfactory animals. When one of the odors is much stronger than the rest, the network converges onto it, thus displaying odor masking.     

Associative memory design using overlapping decomposition and generalized brain-state-in-a-box neural networks

This paper is concerned with large scale associative memory design. A serious problem with neural associative memories is the quadratic growth of the number of interconnections with the problem size. An overlapping decomposition algorithm is proposed to attack this problem. Specifically, a pattern to be processed is decomposed into overlapping sub-patterns. Then, neural sub-networks are constructed that process the sub-patterns. An error correction algorithm operates on the outputs of each sub-network in order to correct the mismatches between sub-patterns that are obtained from the independent recall processes of individual sub-networks. The performance of the proposed large scale associative memory is illustrated using two-dimensional images. It is shown that the proposed method reduces the computing cost of the design of the associative memories compared with non-interconnected associative memories.     

Cluster associative memory formation in a three-layer network of phase oscillators

A three-layer network model of oscillatory associative memory is proposed. The network is capable of storing binary images, which can be retrieved upon presenting an appropriate stimulus. Binary images are encoded in the form of the spatial distribution of oscillatory phase clusters in-phase and anti-phase relative to a reference periodic signal. The information is loaded into the network using a set of interlayer connection weights. A condition for error-free pattern retrieval is formulated, delimiting the maximal number of patterns to be stored in the memory (storage capacity). It is shown that the capacity can be significantly increased by generating an optimal alphabet (basis pattern set). The number of stored patterns can reach values of the network size (the number of oscillators in each layer), which is significantly higher than the capacity of conventional oscillatory memory models. The dynamical and information characteristics of the retrieval process based on the optimal alphabet, including the size of “attraction basins“ and the input pattern distortion admissible for error-free retrieval, are investigated.     

A hierarchical neural network model for associative memory

A hierarchical neural network model with feedback interconnections, which has the function of associative memory and the ability to recognize patterns, is proposed. The model consists of a hierarchical multi-layered network to which efferent connections are added, so as to make positive feedback loops in pairs with afferent connections. The cell-layer at the initial stage of the network is the input layer which receives the stimulus input and at the same time works as an output layer for associative recall. The deepest layer is the output layer for pattern-recognition. Pattern-recognition is performed hierarchically by integrating information by converging afferent paths in the network. For the purpose of associative recall, the integrated information is again distributed to lower-order cells by diverging efferent paths. These two operations progress simultaneously in the network. If a fragment of a training pattern is presented to the network which has completed its self-organization, the entire pattern will gradually be recalled in the initial layer. If a stimulus consisting of a number of training patterns superposed is presented, one pattern gradually becomes predominant in the recalled output after competition between the patterns, and the others disappear. At about the same time when the recalled pattern reaches a steady state in he initial layer, in the deepest layer of the network, a response is elicited from the cell corresponding to the category of the finally-recalled pattern. Once a steady state has been reached, the response of the network is automatically extinguished by inhibitory signals from a steadiness-detecting cell. If the same stimulus is still presented after inhibition, a response for another pattern, formerly suppressed, will now appear, because the cells of the network have adaptation characteristics which makes the same response unlikely to recur. Since inhibition occurs repeatedly, the superposed input patterns are recalled one by one in turn.     

Vector representation of associative learning

I. P. Pavlov [12] has shown that conditioned reflexes are selective both with respect to conditioned stimuli and to conditioned reflexes elicited by those conditioned stimuli. At the neuronal level selective aspects of conditioned stimuli are based on detectors selectively tuned to respective stimuli. The selective aspects of conditioned reflexes are due to command neurons representing specific unconditioned reflexes. It can be assumed that conditioned reflexes result from association between selective detectors and specific command neurons. The detectors activated by a conditioned stimulus constitute a combination of excitations--a detector excitation vector. The detector excitation vector acts on a command neuron via a set of plastic synapses--a synaptic weight vector. Plastic synapses are modified in the process of learning making command neuron selectively tuned to a specific conditioned stimulus. The selective tuning of a particular command neuron to a specific excitation vector referred to a conditioned stimulus is a basis of associative learning. The probabilities of conditioned reflexes elicited by conditioned and differential stimuli implicitly contain information concerning excitation vectors that encode respective stimuli. Contribution of the vector code to associative learning was explored combining differential color conditioning with intracellular recording from color-coding neurons. It was shown that colors in carps and monkeys are represented on a hypersphere in the four-dimensional space similar to human color space. The basis of the color space is constituted by red-green, blue-yellow, brightness and darkness neurons.     

Individual/Typological Peculiarities of EEG Rearrangements in Humans Related to the Analysis of Olfactory Information: Role of Extroversion/Introversion

We studied changes in human EEGs related to presentation of olfactory stimuli (smells of essential oils) and dependence of such rearrangements on the level of extroversion/introversion typical of the tested subject. It was shown that this feature of personality noticeably influences the pattern of odorant-induced changes in EEG. Persons with a predominance of introversion were characterized by higher levels of nonspecific activation of the brain related to perception of olfactory stimuli, which was manifested in a decrease in the power of low-frequency EEG components in the parietal, occipital, and temporal cerebral cortices. The pattern of the rise in coherence level of high-frequency spectrum range oscillations directed toward caudal leads is considered a manifestation of intensification of the intrinsic associative processes. In individuals with a predominance of extroversion, we observed, on the whole, smaller levels of nonspecific cerebral activation upon the action of olfactory stimulation. The fronto-parietal pattern of spatial EEG synchronization is indicative of the development of sensory-analytical processes related to perception of external information. In general, our data agree with the interpretation of such a psychological parameter as extroversion/introversion.     

Contextual taste cues modulate olfactory learning in C. elegans by an occasion-setting mechanism

Manipulations of context can affect learning and memory performance across species in many associative learning paradigms. Using taste cues to create distinct contexts for olfactory adaptation assays in the nematode Caenorhabditis elegans, we now show that performance in this associative learning paradigm is sensitive to context manipulations, and we investigate the mechanism(s) used for the integration of context cues in learning. One possibility is that the taste and olfactory stimuli are perceived as a combined, blended cue that the animals then associate with the unconditioned stimulus (US) in the same manner as with any other unitary conditioned stimuli (CS). Alternatively, an occasion-setting model suggests that the taste cues only define the appropriate context for olfactory memory retrieval without directly entering into the primary association. Analysis of genetic mutants demonstrated that the olfactory and context cues are sensed by distinct primary sensory neurons and that the animals' ability to use taste cues to modulate olfactory learning is independent from their ability to utilize these same taste cues for adaptation. We interpret these results as evidence for the occasion-setting mechanism in which context cues modulate primary Pavlovian association by functioning in a hierarchical manner to define the appropriate setting for memory recall.     

Selectivity of stimulus induced responses in cultured hippocampal networks on microelectrode arrays

Sensory information can be encoded using the average firing rate and spike occurrence times in neuronal network responses to external stimuli. Decoding or retrieving stimulus characteristics from the response pattern generally implies that the corresponding neural network has a selective response to various input signals. The role of various spiking activity characteristics (e.g., spike rate and precise spike timing) for basic information processing was widely investigated on the level of neural populations but gave inconsistent evidence for particular mechanisms. Multisite electrophysiology of cultured neural networks grown on microelectrode arrays is a recently developed tool and currently an active research area. In this study, we analyzed the stimulus responses represented by network-wide bursts evoked from various spatial locations (electrodes). We found that the response characteristics, such as the burst initiation time and the spike rate, can be used to retrieve information about the stimulus location. The best selectivity in the response spiking pattern could be found for a small subpopulation of neurones (electrodes) at relatively short post-stimulus intervals. Such intervals were unique for each culture due to the non-uniform organization of the functional connectivity in the network during spontaneous development.     

Influence of external periodic stimuli on heart rate variability in healthy subjects and in coronary heart disease patients

Frequency estimates of the heart rate variability (HRV) spectrum influenced by external periodic stimuli were studied in healthy subjects and patients with coronary heart disease (CHD). Sensory stimulation by periodic eye opening at a rate of 15, 10, 8, 6, or 5 times per minute, as well as spontaneous and controlled breathing at a rate of 15, 10, 8, 6, or 5 times per minute, was used. It was found that the spectral response to external periodic oscillations was determined by a frequency-dependent phenomenon, the maximal amplitude of heart rate variations being observed in the case of external stimuli at a frequency of 0.1 Hz. A resonance frequency in the 0.1-Hz range may be suggested to exist in the cardiovascular controls. Significant differences in the HRV frequency characteristics between CHD patients and healthy subjects were shown. CHD patients had a characteristic decline in HRV responses to external oscillations; the power of these responses did not depend on the frequency of external stimuli.