Distributed Fading Memory for Stimulus Properties in the Primary Visual Cortex

It is currently not known how distributed neuronal responses in early visual areas carry stimulus-related information. We made multielectrode recordings from cat primary visual cortex and applied methods from machine learning in order to analyze the temporal evolution of stimulus-related information in the spiking activity of large ensembles of around 100 neurons. We used sequences of up to three different visual stimuli (letters of the alphabet) presented for 100 ms and with intervals of 100 ms or larger. Most of the information about visual stimuli extractable by sophisticated methods of machine learning, i.e., support vector machines with nonlinear kernel functions, was also extractable by simple linear classification such as can be achieved by individual neurons. New stimuli did not erase information about previous stimuli. The responses to the most recent stimulus contained about equal amounts of information about both this and the preceding stimulus. This information was encoded both in the discharge rates (response amplitudes) of the ensemble of neurons and, when using short time constants for integration (e.g., 20 ms), in the precise timing of individual spikes (≤∼20 ms), and persisted for several 100 ms beyond the offset of stimuli. The results indicate that the network from which we recorded is endowed with fading memory and is capable of performing online computations utilizing information about temporally sequential stimuli. This result challenges models assuming frame-by-frame analyses of sequential inputs.     

综述: 初级视皮层的亮度信息加工

亮度(luminance)是最基本的视觉信息.与其他视觉特征相比,由于视神经元对亮度刺激的反应较弱,并且许多神经元对均匀亮度无反应,对亮度信息编码的神经机制知之甚少.初级视皮层部分神经元对亮度的反应要慢于对比度反应,被认为是由边界对比度诱导的亮度知觉(brightness)的神经基础.我们的研究表明,初级视皮层许多神经元的亮度反应要快于对比度反应,并且这些神经元偏好低的空间频率、高的时间频率和高的运动速度,提示皮层下具有低空间频率和高运动速度通路的信息输入对产生初级视皮层神经元的亮度反应有贡献.已经知道初级视皮层神经元对空间频率反应的时间过程是从低空间频率到高空间频率,我们发现的早期亮度反应是对极低空间频率的反应,与这一时间过程是一致的,是这一从粗到细的视觉信息加工过程的第一步,揭示了处理最早的粗的视觉信息的神经基础.另外,初级视皮层含有偏好亮度下降和高运动速度的神经元,这群神经元的活动有助于在光照差的环境中检测高速运动的低亮度物体.     

幼年大鼠皮层听-视多感觉神经元和听-视信息整合

应用常规电生理学细胞外记录技术,研究了生后3周龄幼年大鼠皮层听-视双模态神经元及听-视信息整合特性,并与成年动物进行对照。在听皮层的背侧,听皮层和视皮层的交界处,即颞-顶-枕联合皮层区,共记录到了324个神经元,其中45个为听-视双模态神经元,占13.9%,远低于成年动物双模态神经元所占比例(42.8%)。这些双模态神经元可分为A-V型,v-A型和a-V型3种类型。根据它们对听-视信息的整合效应,可分为增强型、抑制型和调制型。整合效应与给予的声和光组合刺激的时间间隔有关,以获得整合效应的时间间隔范围为整合时间窗,幼年动物的平均整合时间窗为11.9 ms,远小于成年动物的整合时间窗(平均为23.2 ms)。结果提示,与单模态感觉神经元对模态特异性反应特性一样,皮层听-视双模态神经元生后有一个发育、成熟的过程。研究结果为深入研究中枢神经元多感觉整合机制提供了重要实验资料。     

Responses of neurons in primary and inferior temporal visual cortices to natural scenes.

The primary visual cortex (V1) is the first cortical area to receive visual input, and inferior temporal (IT) areas are among the last along the ventral visual pathway. We recorded, in area V1 of anaesthetized cats and area IT of awake macaque monkeys, responses of neurons to videos of natural scenes. Responses were analysed to test various hypotheses concerning the nature of neural coding in these two regions. A variety of spike-train statistics were measured including spike-count distributions, interspike interval distributions, coefficients of variation, power spectra, Fano factors and different sparseness measures. All statistics showed non-Poisson characteristics and several revealed self-similarity of the spike trains. Spike-count distributions were approximately exponential in both visual areas for eight different videos and for counting windows ranging from 50 ms to 5 seconds. The results suggest that the neurons maximize their information carrying capacity while maintaining a fixed long-term-average firing rate, or equivalently, minimize their average firing rate for a fixed information carrying capacity.     

Neural mechanism of dynamic responses of neurons in inferior temporal cortex in face perception

Understanding the neural mechanisms of object and face recognition is one of the fundamental challenges of visual neuroscience. The neurons in inferior temporal (IT) cortex have been reported to exhibit dynamic responses to face stimuli. However, little is known about how the dynamic properties of IT neurons emerge in the face information processing. To address this issue, we made a model of IT cortex, which performs face perception via an interaction between different IT networks. The model was based on the face information processed by three resolution maps in early visual areas. The network model of IT cortex consists of four kinds of networks, in which the information about a whole face is combined with the information about its face parts and their arrangements. We show here that the learning of face stimuli makes the functional connections between these IT networks, causing a high spike correlation of IT neuron pairs. A dynamic property of subthreshold membrane potential of IT neuron, produced by Hodgkin–Huxley model, enables the coordination of temporal information without changing the firing rate, providing the basis of the mechanism underlying face perception. We show also that the hierarchical processing of face information allows IT cortex to perform a “coarse-to-fine” processing of face information. The results presented here seem to be compatible with experimental data about dynamic properties of IT neurons.     

Representation of visual stimuli in inferior temporal cortex.

In primates, inferior temporal (IT) cortex is crucial for the processing and storage of visual information about form and colour. This article reviews the properties of IT neurons and considers how these properties may underlie the perceptual and mnemonic functions of IT cortex. The available evidence suggests that the processing of the facial image by IT cortex is similar to its processing of other visual patterns. Faces and other complex visual stimuli appear to be represented by the pattern of responses over a population of IT neurons rather than by the responses of specific 'feature detectors' or 'grandmother' cells. IT neurons with adult-like stimulus properties are present in monkeys as young as six weeks old.     

Timing Precision in Population Coding of Natural Scenes in the Early Visual System

The timing of spiking activity across neurons is a fundamental aspect of the neural population code. Individual neurons in the retina, thalamus, and cortex can have very precise and repeatable responses but exhibit degraded temporal precision in response to suboptimal stimuli. To investigate the functional implications for neural populations in natural conditions, we recorded in vivo the simultaneous responses, to movies of natural scenes, of multiple thalamic neurons likely converging to a common neuronal target in primary visual cortex. We show that the response of individual neurons is less precise at lower contrast, but that spike timing precision across neurons is relatively insensitive to global changes in visual contrast. Overall, spike timing precision within and across cells is on the order of 10 ms. Since closely timed spikes are more efficient in inducing a spike in downstream cortical neurons, and since fine temporal precision is necessary to represent the more slowly varying natural environment, we argue that preserving relative spike timing at a ~10-ms resolution is a crucial property of the neural code entering cortex.     

A multi-stage model for fundamental functional properties in primary visual cortex

Many neurons in mammalian primary visual cortex have properties such as sharp tuning for contour orientation, strong selectivity for motion direction, and insensitivity to stimulus polarity, that are not shared with their sub-cortical counterparts. Successful models have been developed for a number of these properties but in one case, direction selectivity, there is no consensus about underlying mechanisms. We here define a model that accounts for many of the empirical observations concerning direction selectivity. The model describes a single column of cat primary visual cortex and comprises a series of processing stages. Each neuron in the first cortical stage receives input from a small number of on-centre and off-centre relay cells in the lateral geniculate nucleus. Consistent with recent physiological evidence, the off-centre inputs to cortex precede the on-centre inputs by a small (～4 ms) interval, and it is this difference that confers direction selectivity on model neurons. We show that the resulting model successfully matches the following empirical data: the proportion of cells that are direction selective; tilted spatiotemporal receptive fields; phase advance in the response to a stationary contrast-reversing grating stepped across the receptive field. The model also accounts for several other fundamental properties. Receptive fields have elongated subregions, orientation selectivity is strong, and the distribution of orientation tuning bandwidth across neurons is similar to that seen in the laboratory. Finally, neurons in the first stage have properties corresponding to simple cells, and more complex-like cells emerge in later stages. The results therefore show that a simple feed-forward model can account for a number of the fundamental properties of primary visual cortex.     

Neuronal responses related to long-term recognition memory processes in prefrontal cortex

Much evidence indicates that prefrontal cortex plays an important role in long-term recognition memory processes. Here, we report primate prefrontal neuronal responses carrying information necessary for long-term visual recognition memory. The responses of many neurons signaled stimulus familiarity even when the period over which stimuli had to be remembered extended to 24 hr. Such responses occurred frequently in ventromedial, orbitofrontal, and anterior cingulate but not dorsolateral prefrontal cortex. Prefrontal information processing, as indicated by the response latencies, started after that in inferior temporal cortex and might be related to retrieval processes, as responses were typically larger for familiar than for novel stimuli.     

Neurophysiological mechanisms underlying face processing within and beyond the temporal cortical visual areas.

The ways in which information about faces is represented and stored in the temporal lobe visual areas of primates, as shown by recordings from single neurons in macaques, are considered. Some neurons that respond primarily to faces are found in the cortex in the anterior part of the superior temporal sulcus (in which neurons are especially likely to be tuned to facial expression and to face movement involved in gesture), and in the TE areas more ventrally forming the inferior temporal gyrus (in which neurons are more likely to have responses related to the identity of faces). Quantitative studies of the responses of the neurons that respond differently to the faces of different individuals show that information about the identity of the individual is represented by the responses of a population of neurons, that is, ensemble encoding rather than 'grandmother cell' encoding is used. It is argued that this type of tuning is a delicate compromise between very fine tuning, which has the advantage of low interference in neuronal network operations but the disadvantage of losing the useful properties (such as generalization, completion and graceful degradation) of storage in neuronal networks, and broad tuning, which has the advantage of allowing these properties of neuronal networks to be realized but the disadvantage of leading to interference between the different memories stored in an associative network. There is evidence that the responses of some of these neurons are altered by experience so that new stimuli become incorporated in the network. It is shown that the representation that is built in temporal cortical areas shows considerable invariance for size, contrast, spatial frequency and translation. Thus the representation is in a form which is particularly useful for storage and as an output from the visual system. It is also shown that one of the representations that is built is object based, which is suitable for recognition and as an input to associative memory, and that another is viewer centred, which is appropriate for conveying information about gesture. Ways are considered in which such cortical representations might be built by competitive self-organization aided by back projections in the multi-stage cortical processing hierarchy which has convergence from stage to stage.     

Prefrontal cortex and neural mechanisms of executive function

Executive function is a product of the coordinated operation of multiple neural systems and an essential prerequisite for a variety of cognitive functions. The prefrontal cortex is known to be a key structure for the performance of executive functions. To accomplish the coordinated operations of multiple neural systems, the prefrontal cortex must monitor the activities in other cortical and subcortical structures and control and supervise their operations by sending command signals, which is called top-down signaling. Although neurophysiological and neuroimaging studies have provided evidence that the prefrontal cortex sends top-down signals to the posterior cortices to control information processing, the neural correlate of these top-down signals is not yet known. Through use of the paired association task, it has been demonstrated that top-down signals are used to retrieve specific information stored in long-term memory. Therefore, we used a paired association task to examine the neural correlates of top-down signals in the prefrontal cortex. The preliminary results indicate that 32% of visual neurons exhibit pair-selectivity, which is similar to the characteristics of pair-coding activities in temporal neurons. The latency of visual responses in prefrontal neurons was longer than bottom-up signals but faster than top-down signals in inferior temporal neurons. These results suggest that pair-selective visual responses may be top-down signals that the prefrontal cortex provides to the temporal cortex, although further studies are needed to elucidate the neural correlates of top-down signals and their characteristics to understand the neural mechanism of executive control by the prefrontal cortex.     

Information in the Neuronal Representation of Individual Stimuli in the Primate Temporal Visual Cortex

To analyze the information provided about individual visual stimuliin the responses of single neurons in the primate temporal lobevisual cortex, neuronal responses to a set of 65 visual stimuli wererecorded in macaques performing a visual fixation task and analyzedusing information theoretical measures. The population of neuronsanalyzed responded primarily to faces. The stimuli included 23 facesand 42 nonface images of real-world scenes, so that the function ofthis brain region could be analyzed when it was processing relativelynatural scenes.It was found that for the majority of the neurons significantamounts of information were reflected about which of several of the23 faces had been seen. Thus the representation was not local, forin a local representation almost all the information available canbe obtained when the single stimulus to which the neuron respondsbest is shown. It is shown that the information available about anyone stimulus depended on how different (for example, how manystandard deviations) the response to that stimulus was from theaverage response to all stimuli. This was the case for responsesbelow the average response as well as above.It is shown that the fraction of information carried by the lowfiring rates of a cell was large—much larger than that carried bythe high firing rates. Part of the reason for this is that theprobability distribution of different firing rates is biased towardlow values (though with fewer very low values than would bepredicted by an exponential distribution). Another factor is thatthe variability of the response is large at intermediate and highfiring rates.Another finding is that at short sampling intervals (such as 20 ms)the neurons code information efficiently, by effectively acting asbinary variables and behaving less noisily than would be expectedof a Poisson process.     

Active maintenance of associative mnemonic signal in monkey inferior temporal cortex

We investigated the contribution of the inferior temporal (IT) cortical neurons to the active maintenance of internal representations. The activity of single neurons in the IT cortex was recorded while the monkeys performed a sequential-type associative memory task in which distractor stimuli interrupted the delay epoch between the cue and target (paired-associate) stimuli. For each neuron, information about each stimulus conveyed by the delay activity was estimated as a coefficient of multiple regression analysis. We found that target information derived from long-term memory (LTM) persisted despite the distractors. By contrast, cue information derived from the visual system was attenuated and frequently replaced by distractor information. These results suggest that LTM-derived information required for upcoming behavior is actively maintained in the IT neurons, whereas visually derived information tends to be updated irrespective of behavioral relevance.     

Directing the mind's eye: prefrontal, inferior and medial temporal mechanisms for visual working memory

Human and nonhuman primates have a remarkable ability to recall, maintain and manipulate visual images in the absence of external sensory stimulation. Evidence from lesion, single-unit neurophysiological and neuroimaging studies shows that these visual working memory processes are consistently associated with sustained activity in object-selective inferior temporal neurons. Furthermore, results from these studies suggest that mnemonic activity in the inferior temporal cortex is, in turn, supported by top-down inputs from multimodal regions in prefrontal and medial temporal cortex, and under some circumstances, from the hippocampus.     

Relating Information,Encoding and Adaptation: Decoding the Population Firing Rate in Visual Areas 17/18 in Response to a Stimulus Transition

Neurons in the primary visual cortex typically reach their highest firing rate after an abrupt image transition. Since the mutual information between the firing rate and the currently presented image is largest during this early firing period it is tempting to conclude this early firing encodes the current image. This view is, however, made more complicated by the fact that the response to the current image is dependent on the preceding image. Therefore we hypothesize that neurons encode a combination of current and previous images, and that the strength of the current image relative to the previous image changes over time. The temporal encoding is interesting, first, because neurons are, at different time points, sensitive to different features such as luminance, edges and textures; second, because the temporal evolution provides temporal constraints for deciphering the instantaneous population activity. To study the temporal evolution of the encoding we presented a sequence of 250 ms stimulus patterns during multiunit recordings in areas 17 and 18 of the anaesthetized ferret. Using a novel method we decoded the pattern given the instantaneous population-firing rate. Following a stimulus transition from stimulus A to B the decoded stimulus during the first 90ms was more correlated with the difference between A and B (B-A) than with B alone. After 90ms the decoded stimulus was more correlated with stimulus B than with B-A. Finally we related our results to information measures of previous (B) and current stimulus (A). Despite that the initial transient conveys the majority of the stimulus-related information; we show that it actually encodes a difference image which can be independent of the stimulus. Only later on, spikes gradually encode the stimulus more exclusively.     

Coarse-to-fine changes of receptive fields in lateral geniculate nucleus have a transient and a sustained component that depend on distinct mechanisms

Visual processing in the brain seems to provide fast but coarse information before information about fine details. Such dynamics occur also in single neurons at several levels of the visual system. In the dorsal lateral geniculate nucleus (LGN), neurons have a receptive field (RF) with antagonistic center-surround organization, and temporal changes in center-surround organization are generally assumed to be due to a time-lag of the surround activity relative to center activity. Spatial resolution may be measured as the inverse of center size, and in LGN neurons RF-center width changes during static stimulation with durations in the range of normal fixation periods (250-500 ms) between saccadic eye-movements. The RF-center is initially large, but rapidly shrinks during the first ~100 ms to a rather sustained size. We studied such dynamics in anesthetized cats during presentation (250 ms) of static spots centered on the RF with main focus on the transition from the first transient and highly dynamic component to the second more sustained component. The results suggest that the two components depend on different neuronal mechanisms that operate in parallel and with partial temporal overlap rather than on a continuously changing center-surround balance. Results from mathematical modeling further supported this conclusion. We found that existing models for the spatiotemporal RF of LGN neurons failed to account for our experimental results. The modeling demonstrated that a new model, in which the response is given by a sum of an early transient component and a partially overlapping sustained component, adequately accounts for our experimental data.     

Neural substrates of processing syntax and semantics in music

Growing evidence indicates that syntax and semantics are basic aspects of music. After the onset of a chord, initial music-syntactic processing can be observed at about 150-400 ms and processing of musical semantics at about 300-500 ms. Processing of musical syntax activates inferior frontolateral cortex, ventrolateral premotor cortex and presumably the anterior part of the superior temporal gyrus. These brain structures have been implicated in sequencing of complex auditory information, identification of structural relationships, and serial prediction. Processing of musical semantics appears to activate posterior temporal regions. The processes and brain structures involved in the perception of syntax and semantics in music have considerable overlap with those involved in language perception, underlining intimate links between music and language in the human brain.     

Delayed striate cortical activation during spatial attention

Recordings of event-related potentials (ERPs) and event-related magnetic fields (ERMFs) were combined with functional magnetic resonance imaging (fMRI) to study visual cortical activity in humans during spatial attention. While subjects attended selectively to stimulus arrays in one visual field, fMRI revealed stimulus-related activations in the contralateral primary visual cortex and in multiple extrastriate areas. ERP and ERMF recordings showed that attention did not affect the initial evoked response at 60-90 ms poststimulus that was localized to primary cortex, but a similarly localized late response at 140-250 ms was enhanced to attended stimuli. These findings provide evidence that the primary visual cortex participates in the selective processing of attended stimuli by means of delayed feedback from higher visual-cortical areas.     

Knowing with which eye we see: utrocular discrimination and eye-specific signals in human visual cortex

Neurophysiological and behavioral reports converge to suggest that monocular neurons in the primary visual cortex are biased toward low spatial frequencies, while binocular neurons favor high spatial frequencies. Here we tested this hypothesis with functional magnetic resonance imaging (fMRI). Human participants viewed flickering gratings at one of two spatial frequencies presented to either the left or the right eye, and judged which of the two eyes was being stimulated (utrocular discrimination). Using multivoxel pattern analysis we found that local spatial patterns of signals in primary visual cortex (V1) allowed successful decoding of the eye-of-origin. Decoding was above chance for low but not high spatial frequencies, confirming the presence of a bias reported by animal studies in human visual cortex. Behaviorally, we found that reliable judgment of the eye-of-origin did not depend on spatial frequency. We further analyzed the mean response in visual cortex to our stimuli and revealed a weak difference between left and right eye stimulation. Our results are thus consistent with the interpretation that participants use overall levels of neural activity in visual cortex, perhaps arising due to local luminance differences, to judge the eye-of-origin. Taken together, we show that it is possible to decode eye-specific voxel pattern information in visual cortex but, at least in healthy participants with normal binocular vision, these patterns are unrelated to awareness of which eye is being stimulated.     

Modulation of V1 Spike Response by Temporal Interval of Spatiotemporal Stimulus Sequence

The spike activity of single neurons of the primary visual cortex (V1) becomes more selective and reliable in response to wide-field natural scenes compared to smaller stimuli confined to the classical receptive field (RF). However, it is largely unknown what aspects of natural scenes increase the selectivity of V1 neurons. One hypothesis is that modulation by surround interaction is highly sensitive to small changes in spatiotemporal aspects of RF surround. Such a fine-tuned modulation would enable single neurons to hold information about spatiotemporal sequences of oriented stimuli, which extends the role of V1 neurons as a simple spatiotemporal filter confined to the RF. In the current study, we examined the hypothesis in the V1 of awake behaving monkeys, by testing whether the spike response of single V1 neurons is modulated by temporal interval of spatiotemporal stimulus sequence encompassing inside and outside the RF. We used two identical Gabor stimuli that were sequentially presented with a variable stimulus onset asynchrony (SOA): the preceding one (S1) outside the RF and the following one (S2) in the RF. This stimulus configuration enabled us to examine the spatiotemporal selectivity of response modulation from a focal surround region. Although S1 alone did not evoke spike responses, visual response to S2 was modulated for SOA in the range of tens of milliseconds. These results suggest that V1 neurons participate in processing spatiotemporal sequences of oriented stimuli extending outside the RF.