Development of spatial coarse-to-fine processing in the visual pathway

The sequential analysis of information in a coarse-to-fine manner is a fundamental mode of processing in the visual pathway. Spatial frequency (SF) tuning, arguably the most fundamental feature of spatial vision, provides particular intuition within the coarse-to-fine framework: low spatial frequencies convey global information about an image (e.g., general orientation), while high spatial frequencies carry more detailed information (e.g., edges). In this paper, we study the development of cortical spatial frequency tuning. As feedforward input from the lateral geniculate nucleus (LGN) has been shown to have significant influence on cortical coarse-to-fine processing, we present a firing-rate based thalamocortical model which includes both feedforward and feedback components. We analyze the relationship between various model parameters (including cortical feedback strength) and responses. We confirm the importance of the antagonistic relationship between the center and surround responses in thalamic relay cell receptive fields (RFs), and further characterize how specific structural LGN RF parameters affect cortical coarse-to-fine processing. Our results also indicate that the effect of cortical feedback on spatial frequency tuning is age-dependent: in particular, cortical feedback more strongly affects coarse-to-fine processing in kittens than in adults. We use our results to propose an experimentally testable hypothesis for the function of the extensive feedback in the corticothalamic circuit.     

Corticothalamic interactions in the transfer of visual information

Thalamic function does not stand apart, as a discrete processing step, from the cortical circuitry. The thalamus receives extensive feedback from the cortex and this influences the firing pattern, synchronization and sensory response mode of relay cells. A crucial question concerns the extent to which the feedback simply controls the state and transmission mode of relay cells and the extent to which the feedback participates in the specific processing of sensory information. Using examples from experiments examining the influence of feedback from the visual cortex to the lateral geniculate nucleus (LGN), we argue that thalamic mechanisms are selectively focused by visually driven feedback to optimize the thalamic contribution to segmentation and global integration. This involves effects on both the temporal and spatial parameters characterizing the responses of LGN cells and includes, for example, motion-driven feedback effects from MT (middle temporal visual area) relayed via layer 6 of V1 (primary visual cortex).     

Periodicity coding in the primary auditory cortex of the Mongolian gerbil (Merionesunguiculatus ): two different coding strategies for pitch and rhythm?

Periodic envelope or amplitude modulations (AM) with periodicities up to several thousand Hertz are characteristic for many natural sounds. Throughout the auditory pathway, signal periodicity is evident in neuronal discharges phase-locked to the envelope. In contrast to lower levels of the auditory pathway, cortical neurons do not phase-lock to periodicities above about 100 Hz. Therefore, we investigated alternative coding strategies for high envelope periodicities at the cortical level. Neuronal responses in the primary auditory cortex (AI) of gerbils to tones and AM were analysed. Two groups of stimuli were tested: (1) AM with a carrier frequency set to the unit's best frequency evoked phase-locked responses which were confined to low modulation frequencies (fms) up to about 100 Hz, and (2) AM with a spectrum completely outside the unit's frequency-response range evoked completely different responses that never showed phase-locking but a rate-tuning to high fms (50 to about 3000 Hz). In contrast to the phase-locked responses, the best fms determined from these latter responses appeared to be topographically distributed, reflecting a periodotopic organization in the AI. Implications of these results for the cortical representation of the perceptual qualities rhythm, roughness and pitch are discussed. Accepted: 25 July 1997     

Synaptic Plasticity Controls Sensory Responses through Frequency-Dependent Gamma Oscillation Resonance

Synchronized gamma frequency oscillations in neural networks are thought to be important to sensory information processing, and their effects have been intensively studied. Here we describe a mechanism by which the nervous system can readily control gamma oscillation effects, depending selectively on visual stimuli. Using a model neural network simulation, we found that sensory response in the primary visual cortex is significantly modulated by the resonance between “spontaneous” and “stimulus-driven” oscillations. This gamma resonance can be precisely controlled by the synaptic plasticity of thalamocortical connections, and cortical response is regulated differentially according to the resonance condition. The mechanism produces a selective synchronization between the afferent and downstream neural population. Our simulation results explain experimental observations such as stimulus-dependent synchronization between the thalamus and the cortex at different oscillation frequencies. The model generally shows how sensory information can be selectively routed depending on its frequency components.     

Visual threshold is set by linear and nonlinear mechanisms in the retina that mitigate noise: how neural circuits in the retina improve the signal-to-noise ratio of the single-photon response

In sensory biology, a major outstanding question is how sensory receptor cells minimize noise while maximizing signal to set the detection threshold. This optimization could be problematic because the origin of both the signals and the limiting noise in most sensory systems is believed to lie in stimulus transduction. Signal processing in receptor cells can improve the signal-to-noise ratio. However, neural circuits can further optimize the detection threshold by pooling signals from sensory receptor cells and processing them using a combination of linear and nonlinear filtering mechanisms. In the visual system, noise limiting light detection has been assumed to arise from stimulus transduction in rod photoreceptors. In this context, the evolutionary optimization of the signal-to-noise ratio in the retina has proven critical in allowing visual sensitivity to approach the limits set by the quantal nature of light. Here, we discuss how noise in the mammalian retina is mitigated to allow for highly sensitive night vision.     

Retinal and cortical nonlinearities combine to produce masking in V1 responses to plaids

The visual response of a cell in the primary visual cortex (V1) to a drifting grating stimulus at the cell’s preferred orientation decreases when a second, perpendicular, grating is superimposed. This effect is called masking. To understand the nonlinear masking effect, we model the response of Macaque V1 simple cells in layer 4Cα to input from magnocellular Lateral Geniculate Nucleus (LGN) cells. The cortical model network is a coarse-grained reduction of an integrate-and-fire network with excitation from LGN input and inhibition from other cortical neurons. The input is modeled as a sum of LGN cell responses. Each LGN cell is modeled as the convolution of a spatio-temporal filter with the visual stimulus, normalized by a retinal contrast gain control, and followed by rectification representing the LGN spike threshold. In our model, the experimentally observed masking arises at the level of LGN input to the cortex. The cortical network effectively induces a dynamic threshold that forces the test grating to have high contrast before it can overcome the masking provided by the perpendicular grating. The subcortical nonlinearities and the cortical network together account for the masking effect. Melinda Koelling is formerly from Center for Neural Science and Courant Institute, New York University.     

Integrate-and-fire vs Poisson models of LGN input to V1 cortex: noisier inputs reduce orientation selectivity

One of the reasons the visual cortex has attracted the interest of computational neuroscience is that it has well-defined inputs. The lateral geniculate nucleus (LGN) of the thalamus is the source of visual signals to the primary visual cortex (V1). Most large-scale cortical network models approximate the spike trains of LGN neurons as simple Poisson point processes. However, many studies have shown that neurons in the early visual pathway are capable of spiking with high temporal precision and their discharges are not Poisson-like. To gain an understanding of how response variability in the LGN influences the behavior of V1, we study response properties of model V1 neurons that receive purely feedforward inputs from LGN cells modeled either as noisy leaky integrate-and-fire (NLIF) neurons or as inhomogeneous Poisson processes. We first demonstrate that the NLIF model is capable of reproducing many experimentally observed statistical properties of LGN neurons. Then we show that a V1 model in which the LGN input to a V1 neuron is modeled as a group of NLIF neurons produces higher orientation selectivity than the one with Poisson LGN input. The second result implies that statistical characteristics of LGN spike trains are important for V1’s function. We conclude that physiologically motivated models of V1 need to include more realistic LGN spike trains that are less noisy than inhomogeneous Poisson processes.     

Visual Receptive Field Properties of Neurons in the Mouse Lateral Geniculate Nucleus

The lateral geniculate nucleus (LGN) is increasingly regarded as a “smart-gating” operator for processing visual information. Therefore, characterizing the response properties of LGN neurons will enable us to better understand how neurons encode and transfer visual signals. Efforts have been devoted to study its anatomical and functional features, and recent advances have highlighted the existence in rodents of complex features such as direction/orientation selectivity. However, unlike well-researched higher-order mammals such as primates, the full array of response characteristics vis-à-vis its morphological features have remained relatively unexplored in the mouse LGN. To address the issue, we recorded from mouse LGN neurons using multisite-electrode-arrays (MEAs) and analysed their discharge patterns in relation to their location under a series of visual stimulation paradigms. Several response properties paralleled results from earlier studies in the field and these include centre-surround organization, size of receptive field, spontaneous firing rate and linearity of spatial summation. However, our results also revealed “high-pass” and “low-pass” features in the temporal frequency tuning of some cells, and greater average contrast gain than reported by earlier studies. In addition, a small proportion of cells had direction/orientation selectivity. Both “high-pass” and “low-pass” cells, as well as direction and orientation selective cells, were found only in small numbers, supporting the notion that these properties emerge in the cortex. ON- and OFF-cells showed distinct contrast sensitivity and temporal frequency tuning properties, suggesting parallel projections from the retina. Incorporating a novel histological technique, we created a 3-D LGN volume model explicitly capturing the morphological features of mouse LGN and localising individual cells into anterior/middle/posterior LGN. Based on this categorization, we show that the ON/OFF, DS/OS and linear response properties are not regionally restricted. Our study confirms earlier findings of spatial pattern selectivity in the LGN, and builds on it to demonstrate that relatively elaborate features are computed early in the visual pathway.     

A cooperation and competition based simple cell receptive field model and study of feed-forward linear and nonlinear contributions to orientation selectivity

We present a model for development of orientation selectivity in layer IV simple cells. Receptive field (RF) development in the model, is determined by diffusive cooperation and resource limited competition guided axonal growth and retraction in geniculocortical pathway. The simulated cortical RFs resemble experimental RFs. The receptive field model is incorporated in a three-layer visual pathway model consisting of retina, LGN and cortex. We have studied the effect of activity dependent synaptic scaling on orientation tuning of cortical cells. The mean value of hwhh (half width at half the height of maximum response) in simulated cortical cells is 58° when we consider only the linear excitatory contribution from LGN. We observe a mean improvement of 22.8° in tuning response due to the non-linear spiking mechanisms that include effects of threshold voltage and synaptic scaling factor.     

Profound contrast adaptation early in the visual pathway

Prior exposure to a moving grating of high contrast led to a substantial and persistent reduction in the contrast sensitivity of neurons in the lateral geniculate nucleus (LGN) of macaque. This slow contrast adaptation was potent in all magnocellular (M) cells but essentially absent in parvocellular (P) cells and neurons that received input from S cones. Simultaneous recordings of M cells and the potentials of ganglion cells driving them showed that adaptation originated in ganglion cells. As expected from the spatiotemporal tuning of M cells, adaptation was broadly tuned for spatial frequency and lacked orientation selectivity. Adaptation could be induced by high temporal frequencies to which cortical neurons do not respond, but not by low temporal frequencies that can strongly adapt cortical neurons. Our observations confirm that contrast adaptation occurs at multiple levels in the visual system, and they provide a new way to reveal the function and perceptual significance of the M pathway.     

A model of striate response properties based on geniculate anisotropies

The orientation biases seen in the responses of cells in the retina and dLGN are dependent on the spatial frequency of the stimulus, being appreciable only at higher spatial frequencies. An inhibitory mechanism that suppresses the responses to low spatial frequencies would leave a striate cell receiving a biased geniculate input with an orientation sensitivity at the higher spatial frequencies. Such an inhibition could in fact come from one or a small group of LGN cells (through cortical interneurones), since their response extends to spatial frequencies much lower than for cortical cells at the same eccentricity. According to this scheme, a number of other striate response characteristics, e.g., their length and spatial frequency response functions, can also be explained.     

S cone contributions to the magnocellular visual pathway in macaque monkey

The magnocellular visual pathway is believed to receive input from long (L) and middle (M), but not short (S), wavelength-sensitive cones. Recording from neurons in magnocellular layers of lateral geniculate nucleus (LGN) in macaque monkeys, we found that magnocellular neurons were unequivocally responsive to S cone-isolating stimuli. A quantitative analysis suggests that S cones provided about 10% of the input to these cells, on average, while L:M ratios were far more variable. S cone signals influenced responses with the same sign as L and M cone inputs (i.e., no color opponency). Magnocellular afferent recordings following inactivation of primary visual cortex demonstrated that S cone signals were feedforward in nature and did not arise from cortical feedback to LGN     

Natural scene statistics and nonlinear neural interactions between frequency-selective mechanisms

Linear filtering is a basic concept in neural models of early sensory information processing. In particular the visual system has been described to perform a wavelet-like multi-channel decomposition by a set of independent spatial-frequency selective filter mechanisms. Here we suggest that this principle of linear filtering deserves a critical re-evaluation. We propose that an optimal adaptation to natural scene statistics would require AND-like nonlinear interactions between the frequency-selective filter channels. We describe how this hypothesis can be tested by predicted violations of the principle of linearity that should be observable if cortical neurons would actually implement the proposed nonlinearities. We further explain why these effects might have been easily overlooked in earlier tests of the linearity of neurons in primary visual cortex.     

A spherical model for orientation and spatial-frequency tuning in a cortical hypercolumn

A theory is presented of the way in which the hypercolumns in primary visual cortex (V1) are organized to detect important features of visual images, namely local orientation and spatial-frequency. Given the existence in V1 of dual maps for these features, both organized around orientation pinwheels, we constructed a model of a hypercolumn in which orientation and spatial-frequency preferences are represented by the two angular coordinates of a sphere. The two poles of this sphere are taken to correspond, respectively, to high and low spatial-frequency preferences. In Part I of the paper, we use mean-field methods to derive exact solutions for localized activity states on the sphere. We show how cortical amplification through recurrent interactions generates a sharply tuned, contrast-invariant population response to both local orientation and local spatial frequency, even in the case of a weakly biased input from the lateral geniculate nucleus (LGN). A major prediction of our model is that this response is non-separable with respect to the local orientation and spatial frequency of a stimulus. That is, orientation tuning is weaker around the pinwheels, and there is a shift in spatial-frequency tuning towards that of the closest pinwheel at non-optimal orientations. In Part II of the paper, we demonstrate that a simple feed-forward model of spatial-frequency preference, unlike that for orientation preference, does not generate a faithful representation when amplified by recurrent interactions in V1. We then introduce the idea that cortico-geniculate feedback modulates LGN activity to generate a faithful representation, thus providing a new functional interpretation of the role of this feedback pathway. Using linear filter theory, we show that if the feedback from a cortical cell is taken to be approximately equal to the reciprocal of the corresponding feed-forward receptive field (in the two-dimensional Fourier domain), then the mismatch between the feed-forward and cortical frequency representations is eliminated. We therefore predict that cortico-geniculate feedback connections innervate the LGN in a pattern determined by the orientation and spatial-frequency biases of feed-forward receptive fields. Finally, we show how recurrent cortical interactions can generate cross-orientation suppression.     

Cortical Contributions to Saccadic Suppression

The stability of visual perception is partly maintained by saccadic suppression: the selective reduction of visual sensitivity that accompanies rapid eye movements. The neural mechanisms responsible for this reduced perisaccadic visibility remain unknown, but the Lateral Geniculate Nucleus (LGN) has been proposed as a likely site. Our data show, however, that the saccadic suppression of a target flashed in the right visual hemifield increased with an increase in background luminance in the left visual hemifield. Because each LGN only receives retinal input from a single hemifield, this hemifield interaction cannot be explained solely on the basis of neural mechanisms operating in the LGN. Instead, this suggests that saccadic suppression must involve processing in higher level cortical areas that have access to a considerable part of the ipsilateral hemifield.     

Receptive field mechanisms of cat X and Y retinal ganglion cells

We investigated receptive field properties of cat retinal ganglion cells with visual stimuli which were sinusoidal spatial gratings amplitude modulated in time by a sum of sinusoids. Neural responses were analyzed into the Fourier components at the input frequencies and the components at sum and difference frequencies. The first-order frequency response of X cells had a marked spatial phase and spatial frequency dependence which could be explained in terms of linear interactions between center and surround mechanisms in the receptive field. The second-order frequency response of X cells was much smaller than the first-order frequency response at all spatial frequencies. The spatial phase and spatial frequency dependence of the first-order frequency response in Y cells in some ways resembled that of X cells. However, the Y first-order response declined to zero at a much lower spatial frequency than in X cells. Furthermore, the second-order frequency response was larger in Y cells; the second-order frequency components became the dominant part of the response for patterns of high spatial frequency. This implies that the receptive field center and surround mechanisms are physiologically quite different in Y cells from those in X cells, and that the Y cells also receive excitatory drive from an additional nonlinear receptive field mechanism.     

Neural mechanisms of human texture processing: texture boundary detection and visual search

Texture of various appearances, geometric distortions, spatial frequency content and densities is utilized by the human visual system to segregate items from background and to enable recognition of complex geometric forms. For automatic, or pre-attentive, segmentation of a visual scene, sophisticated analysis and comparison of surface properties over wide areas of the visual field are required. We investigated the neural substrate underlying human texture processing, particularly the computational mechanisms of texture boundary detection. We present a neural network model which uses as building blocks model cortical areas that are bi-directionally linked to implement cycles of feedforward and feedback interaction for signal detection, hypothesis generation and testing within the infero-temporal pathway of form processing. In the spirit of Jake Beck's early investigations our model particularly builds upon two key hypotheses, namely that (i) texture segregation is based on boundary detection, rather than clustering homogeneous items, and (ii) texture boundaries are detected mainly on the basis of larger scenic contexts mediated by higher cortical areas, such as area V4. The latter constraint provides a basis for element grouping in accordance to the Gestalt laws of similarity and good continuation. It is shown through simulations that the model integrates a variety of psychophysical findings on texture processing and provides a link to the underlying physiology. The functional role of feedback processing is demonstrated by context dependent modulation of V1 cell activation, leading to sharply localized detection of texture boundaries. It furthermore explains why pre-attentive processing in visual search tasks can be directly linked to texture boundary processing as revealed by recent EEG studies on visual search.     

Initial processing of visual information within the retina and the LGN

The initial stage of information processing by the visual system reduces the information contained in the continuous image on the retina into a discrete set of responses which are carried from the lateral geniculate nucleus (LGN) to the visual cortex.-1. The optimal sampling of the light intensity distribution in the visual environment is achieved only if each channel in the visual pathways carries undistorted information corresponding to an image element. The visual system approaches as closely as possible the scheme of optimal spatial sampling, retaining the full information on the low spatial frequency content of the object light intensity. The ideal receptive field of a sustained LGN cell is then of the form J ₁ (Kr)/Kr.-2. The experimentally determined receptive fields of sustained LGN cells (and to some extent retinal ganglion cells as well) in cat closely resemble the functional form J ₁ (Kr)/Kr. The centre-surround organization of the receptive fields is therefore understood as a scheme which leads to a maximal information flow through the visual pathways.-3. The optimal sampling scheme cannot be realized by the retina alone, because of restrictions on the size of neural networks. It is therefore constructed in two stages, ending at the LGN level. A recombination of ganglion cell signals into optimal receptive fields is a major role of the LGN.     

End-to-end neural system identification with neural information flow

Neural information flow (NIF) provides a novel approach for system identification in neuroscience. It models the neural computations in multiple brain regions and can be trained end-to-end via stochastic gradient descent from noninvasive data. NIF models represent neural information processing via a network of coupled tensors, each encoding the representation of the sensory input contained in a brain region. The elements of these tensors can be interpreted as cortical columns whose activity encodes the presence of a specific feature in a spatiotemporal location. Each tensor is coupled to the measured data specific to a brain region via low-rank observation models that can be decomposed into the spatial, temporal and feature receptive fields of a localized neuronal population. Both these observation models and the convolutional weights defining the information processing within regions are learned end-to-end by predicting the neural signal during sensory stimulation. We trained a NIF model on the activity of early visual areas using a large-scale fMRI dataset recorded in a single participant. We show that we can recover plausible visual representations and population receptive fields that are consistent with empirical findings.     

Saccades differentially modulate human LGN and V1 responses in the presence and absence of visual stimulation

Saccades occur several times each second in normal human vision. The visual image moves across the retina at high velocity during a saccade, yet no blurring of the visual scene is perceived . Active suppression of visual input may account for this perceptual continuity, but the neural mechanisms underlying such saccadic suppression remain unclear. We used functional MRI to specifically examine responses in the lateral geniculate nucleus (LGN) and primary visual cortex (V1) during saccades. Activity in both V1 and LGN was strongly modulated by saccades. Furthermore, this modulation depended on whether visual stimulation was present or absent. In complete darkness, saccades led to reliable signal increases in V1 and LGN, whereas in the presence of visual stimulation, saccades led to suppression of visually evoked responses. These findings represent unequivocal evidence for saccadic suppression in human LGN and retinotopically defined V1 and are consistent with the earliest site of saccadic suppression lying at or before V1.