Contour saliency in primary visual cortex

Contour integration is an important intermediate stage of object recognition, in which line segments belonging to an object boundary are perceptually linked and segmented from complex backgrounds. Contextual influences observed in primary visual cortex (V1) suggest the involvement of V1 in contour integration. Here, we provide direct evidence that, in monkeys performing a contour detection task, there was a close correlation between the responses of V1 neurons and the perceptual saliency of contours. Receiver operating characteristic analysis showed that single neuronal responses encode the presence or absence of a contour as reliably as the animal's behavioral responses. We also show that the same visual contours elicited significantly weaker neuronal responses when they were not detected in the detection task, or when they were unattended. Our results demonstrate that contextual interactions in V1 play a pivotal role in contour integration and saliency.     

Learning to link visual contours

In complex visual scenes, linking related contour elements is important for object recognition. This process, thought to be stimulus driven and hard wired, has substrates in primary visual cortex (V1). Here, however, we find contour integration in V1 to depend strongly on perceptual learning and top-down influences that are specific to contour detection. In naive monkeys, the information about contours embedded in complex backgrounds is absent in V1 neuronal responses and is independent of the locus of spatial attention. Training animals to find embedded contours induces strong contour-related responses specific to the trained retinotopic region. These responses are most robust when animals perform the contour detection task but disappear under anesthesia. Our findings suggest that top-down influences dynamically adapt neural circuits according to specific perceptual tasks. This may serve as a general neuronal mechanism of perceptual learning and reflect top-down mediated changes in cortical states.     

Attention-dependent representation of a size illusion in human V1

One of the most fundamental properties of human primary visual cortex (V1) is its retinotopic organization, which makes it an ideal candidate for encoding spatial properties, such as size, of objects. However, three-dimensional (3D) contextual information can lead to size illusions that are reflected in the spatial pattern of activity in V1 [1]. A critical question is how complex 3D contextual information can influence spatial activity patterns in V1. Here, we assessed whether changes in the spatial distribution of activity in V1 depend on the focus of attention, which would be suggestive of feedback of 3D contextual information from higher visual areas. We presented two 3D rings at close and far apparent depths in a 3D scene. When subjects fixated its center, the far ring appeared to be larger and occupy a more eccentric portion of the visual field, relative to the close ring. Using functional magnetic resonance imaging, we found that the spatial distribution of V1 activity induced by the far ring was also shifted toward a more eccentric representation of the visual field, whereas that induced by the close ring was shifted toward the foveal representation, consistent with their perceptual appearances. This effect was significantly reduced when the focus of spatial attention was narrowed with a demanding central fixation task. We reason that focusing attention on the fixation task resulted in reduced activity in--and therefore reduced feedback from--higher visual areas that process the 3D depth cues.     

Attention to stimulus features shifts spectral tuning of V4 neurons during natural vision

Previous neurophysiological studies suggest that attention can alter the baseline or gain of neurons in extrastriate visual areas but that it cannot change tuning. This suggests that neurons in visual cortex function as labeled lines whose meaning does not depend on task demands. To test this common assumption, we used a system identification approach to measure spatial frequency and orientation tuning in area V4 during two attentionally demanding visual search tasks, one that required fixation and one that allowed free viewing during search. We found that spatial attention modulates response baseline and gain but does not alter tuning, consistent with previous reports. In contrast, feature-based attention often shifts neuronal tuning. These tuning shifts are inconsistent with the labeled-line model and tend to enhance responses to stimulus features that distinguish the search target. Our data suggest that V4 neurons behave as matched filters that are dynamically tuned to optimize visual search.     

V1 mechanisms and some figure-ground and border effects.

V1 neurons have been observed to respond more strongly to figure than background regions. Within a figure region, the responses are usually stronger near figure boundaries (the border effect), than further inside the boundaries. Sometimes the medial axes of the figures (e.g., the vertical midline of a vertical figure strip) induce secondary, intermediate, response peaks (the medial axis effect). Related is the physiologically elusive "cross-orientation facilitation", the observation that a cell's response to a grating patch can be facilitated by an orthogonally oriented grating in the surround. Higher center feedbacks have been suggested to cause these figure-ground effects. It has been shown, using a V1 model, that the causes could be intra-cortical interactions within V1 that serve pre-attentive visual segmentation, particularly, object boundary detection. Furthermore, whereas the border effect is robust, the figure-ground effects in the interior of a figure, in particular, the medial axis effect, are by-products of the border effect and are predicted to diminish to zero for larger figures. This model prediction (of the figure size dependence) was subsequently confirmed physiologically, and supported by findings that the response modulations by texture surround do not depend on feedbacks from V2. In addition, the model explains the "cross-orientation facilitation" as caused by a dis-inhibition, to the cell responding to the center of the central grating, by the background grating. Furthermore, the elusiveness of this phenomena was accounted for by the insight that it depends critically on the size of the figure grating. The model is applied to understand some figure-ground effects and segmentation in psychophysics: in particular, that contrast discrimination threshold is lower within and at the center of a closed contour than that in the background, and that a very briefly presented vernier target can perceptually shine through a subsequently presented large grating centered at the same location.     

Separate modulations of human V1 associated with spatial attention and task structure

Functional magnetic resonance imaging (fMRI) was used while normal human volunteers engaged in simple detection and discrimination tasks, revealing separable modulations of early visual cortex associated with spatial attention and task structure. Both modulations occur even when there is no change in sensory stimulation. The modulation due to spatial attention is present throughout the early visual areas V1, V2, V3, and VP, and varies with the attended location. The task structure activations are strongest in V1 and are greater in regions that represent more peripheral parts of the visual field. Control experiments demonstrate that the task structure activations cannot be attributed to visual, auditory, or somatosensory processing, the motor response for the detection/discrimination judgment, or oculomotor responses such as blinks or saccades. These findings demonstrate that early visual areas are modulated by at least two types of endogenous signals, each with distinct cortical distributions.     

Empirical mode decomposition of field potentials from macaque V4 in visual spatial attention

Empirical mode decomposition (EMD) has recently been introduced as a local and fully data-driven technique for the analysis of non-stationary time-series. It allows the frequency and amplitude of a time-series to be evaluated with excellent time resolution. In this article we consider the application of EMD to the analysis of neuronal activity in visual cortical area V4 of a macaque monkey performing a visual spatial attention task. We show that, by virtue of EMD, field potentials can be resolved into a sum of intrinsic components with different degrees of oscillatory content. Low-frequency components in single-trial recordings contribute to the average visual evoked potential (AVEP), whereas high-frequency components do not, but are identified as gamma-band (30–90 Hz) oscillations. The magnitude of time-varying gamma activity is shown to be enhanced when the monkey attends to a visual stimulus as compared to when it is not attending to the same stimulus. Comparison with Fourier analysis shows that EMD may offer better temporal and frequency resolution. These results support the idea that the magnitude of gamma activity reflects the modulation of V4 neurons by visual spatial attention. EMD, coupled with instantaneous frequency analysis, is demonstrated to be a useful technique for the analysis of neurobiological time-series.     

Modulation of activity in human visual area V1 during memory masking

Neurons in the primary visual cortex, V1, are specialized for the processing of elemental features of the visual stimulus, such as orientation and spatial frequency. Recent fMRI evidence suggest that V1 neurons are also recruited in visual perceptual memory; a number of studies using multi-voxel pattern analysis have successfully decoded stimulus-specific information from V1 activity patterns during the delay phase in memory tasks. However, consistent fMRI signal modulations reflecting the memory process have not yet been demonstrated. Here, we report evidence, from three subjects, that the low V1 BOLD activity during retention of low-level visual features is caused by competing interactions between neural populations coding for different values along the spectrum of the dimension remembered. We applied a memory masking paradigm in which the memory representation of a masker stimulus interferes with a delayed spatial frequency discrimination task when its frequency differs from the discriminanda with ±1 octave and found that impaired behavioral performance due to masking is reflected in weaker V1 BOLD signals. This cross-channel inhibition in V1 only occurs with retinotopic overlap between the masker and the sample stimulus of the discrimination task. The results suggest that memory for spatial frequency is a local process in the retinotopically organized visual cortex.     

The Effects of Context and Attention on Spiking Activity in Human Early Visual Cortex

Here we report the first quantitative analysis of spiking activity in human early visual cortex. We recorded multi-unit activity from two electrodes in area V2/V3 of a human patient implanted with depth electrodes as part of her treatment for epilepsy. We observed well-localized multi-unit receptive fields with tunings for contrast, orientation, spatial frequency, and size, similar to those reported in the macaque. We also observed pronounced gamma oscillations in the local-field potential that could be used to estimate the underlying spiking response properties. Spiking responses were modulated by visual context and attention. We observed orientation-tuned surround suppression: responses were suppressed by image regions with a uniform orientation and enhanced by orientation contrast. Additionally, responses were enhanced on regions that perceptually segregated from the background, indicating that neurons in the human visual cortex are sensitive to figure-ground structure. Spiking responses were also modulated by object-based attention. When the patient mentally traced a curve through the neurons’ receptive fields, the accompanying shift of attention enhanced neuronal activity. These results demonstrate that the tuning properties of cells in the human early visual cortex are similar to those in the macaque and that responses can be modulated by both contextual factors and behavioral relevance. Our results, therefore, imply that the macaque visual system is an excellent model for the human visual cortex.     

V1 mechanisms and some figure–ground and border effects

V1 neurons have been observed to respond more strongly to figure than background regions. Within a figure region, the responses are usually stronger near figure boundaries (the border effect), than further inside the boundaries. Sometimes the medial axes of the figures (e.g., the vertical midline of a vertical figure strip) induce secondary, intermediate, response peaks (the medial axis effect). Related is the physiologically elusive “cross-orientation facilitation”, the observation that a cell's response to a grating patch can be facilitated by an orthogonally oriented grating in the surround. Higher center feedbacks have been suggested to cause these figure–ground effects. It has been shown, using a V1 model, that the causes could be intra-cortical interactions within V1 that serve pre-attentive visual segmentation, particularly, object boundary detection. Furthermore, whereas the border effect is robust, the figure–ground effects in the interior of a figure, in particular, the medial axis effect, are by-products of the border effect and are predicted to diminish to zero for larger figures. This model prediction (of the figure size dependence) was subsequently confirmed physiologically, and supported by findings that the response modulations by texture surround do not depend on feedbacks from V2. In addition, the model explains the “cross-orientation facilitation” as caused by a dis-inhibition, to the cell responding to the center of the central grating, by the background grating. Furthermore, the elusiveness of this phenomena was accounted for by the insight that it depends critically on the size of the figure grating. The model is applied to understand some figure–ground effects and segmentation in psychophysics: in particular, that contrast discrimination threshold is lower within and at the center of a closed contour than that in the background, and that a very briefly presented vernier target can perceptually shine through a subsequently presented large grating centered at the same location.     

Cell-type-specific synchronization of neural activity in FEF with V4 during attention

Shifts of gaze and shifts of attention are closely linked and it is debated whether they result from the same neural mechanisms. Both processes involve the frontal eye fields (FEF), an area which is also a source of top-down feedback to area V4 during covert attention. To test the relative contributions of oculomotor and attention-related FEF signals to such feedback, we recorded simultaneously from both areas in a covert attention task and in a saccade task. In the attention task, only visual and visuomovement FEF neurons showed enhanced responses, whereas movement cells were unchanged. Importantly, visual, but not movement or visuomovement cells, showed enhanced gamma frequency synchronization with activity in V4 during attention. Within FEF, beta synchronization was increased for movement cells during attention but was suppressed in the saccade task. These findings support the idea that the attentional modulation of visual processing is not mediated by movement neurons.     

Different processing phases for features, figures, and selective attention in the primary visual cortex

Our visual system imposes structure onto images that usually contain a diversity of surfaces, contours, and colors. Psychological theories propose that there are multiple steps in this process that occur in hierarchically organized regions of the cortex: early visual areas register basic features, higher areas bind them into objects, and yet higher areas select the objects that are relevant for behavior. Here we test these theories by recording from the primary visual cortex (area V1) of monkeys. We demonstrate that the V1 neurons first register the features (at a latency of 48 ms), then segregate figures from the background (after 57 ms), and finally select relevant figures over irrelevant ones (after 137 ms). We conclude that the psychological processing stages map onto distinct time episodes that unfold in the visual cortex after the presentation of a new stimulus, so that area V1 may contribute to all these processing steps.     

A human extrastriate area functionally homologous to macaque V4

Extrastriate area V4 is crucial for intermediate form vision and visual attention in nonhuman primates. Human neuroimaging suggests that an area in the lingual sulcus/fusiform gyrus may correspond to ventral V4 (V4v). We studied a human neurological patient, AR, with a putative V4v lesion. The lesion does not affect early visual processing (luminance, orientation, and motion perception). However, it does impair hue perception, intermediate form vision, and visual attention in the upper contralateral visual field. Form deficits occur during discrimination of illusory borders, Glass patterns, curvature, and non-Cartesian patterns. Attention deficits occur during discrimination of the relative positions of object parts, detection of low-salience targets, and orientation discrimination in the presence of distractors. This pattern of deficits is consistent with the known properties of area V4 in nonhuman primates, indicating that AR's lesion affects a cortical region functionally homologous to macaque V4.     

Behavioral detection of electrical microstimulation in different cortical visual areas

The extent to which areas in the visual cerebral cortex differ in their ability to support perceptions has been the subject of considerable speculation. Experiments examining the activity of individual neurons have suggested that activity in later stages of the visual cortex is more closely linked to perception than that in earlier stages [1-9]. In contrast, results from functional imaging, transcranial magnetic stimulation, and lesion studies have been interpreted as showing that earlier stages are more closely coupled to perception [10-15]. We examined whether neuronal activity in early and later stages differs in its ability to support detectable signals by measuring behavioral thresholds for detecting electrical microstimulation in different cortical areas in two monkeys. By training the animals to perform a two-alternative temporal forced-choice task, we obtained criterion-free thresholds from five visual areas--V1, V2, V3A, MT, and the inferotemporal cortex. Every site tested yielded a reliable threshold. Thresholds varied little within and between visual areas, rising gradually from early to later stages. We similarly found no systematic differences in the slopes of the psychometric detection functions from different areas. These results suggest that neuronal signals of similar magnitude evoked in any part of visual cortex can generate percepts.     

Attention Narrows Position Tuning of Population Responses in V1

When attention is directed to a region of space, visual resolution at that location flexibly adapts, becoming sharper to resolve fine-scale details or coarser to reflect large-scale texture and surface properties [1]. By what mechanism does attention improve spatial resolution? An improved signal-to-noise ratio (SNR) at the attended location contributes [2], because of retinotopically specific signal gain [3], [4], [5], [6], [7], [8], [9] and [10]. Additionally, attention could sharpen position tuning at the neural population level, so that adjacent objects activate more distinct regions of the visual cortex. A dual mechanism involving both signal gain and sharpened position tuning would be highly efficient at improving visual resolution, but there is no direct evidence that attention can narrow the position tuning of population responses. Here, we compared the spatial spread of the fMRI BOLD response for attended versus ignored stimuli. The activity produced by adjacent stimuli overlapped less when subjects were attending at their locations versus attending elsewhere, despite a stronger peak response with attention. Our results show that even as early as primary visual cortex (V1), spatially directed attention narrows the tuning of population-coded position representations.     

Differential attention-dependent response modulation across cell classes in macaque visual area V4

The cortex contains multiple cell types, but studies of attention have not distinguished between them, limiting understanding of the local circuits that transform attentional feedback into improved visual processing. Parvalbumin-expressing inhibitory interneurons can be distinguished from pyramidal neurons based on their briefer action potential durations. We recorded neurons in area V4 as monkeys performed an attention-demanding task. We find that the distribution of action potential durations is strongly bimodal. Neurons with narrow action potentials have higher firing rates and larger attention-dependent increases in absolute firing rate than neurons with broad action potentials. The percentage increase in response is similar across the two classes. We also find evidence that attention increases the reliability of the neuronal response. This modulation is more than two-fold stronger among putative interneurons. These findings lead to the surprising conclusion that the strongest attentional modulation occurs among local interneurons that do not transmit signals between areas.     

Perceptual learning via modification of cortical top-down signals

The primary visual cortex (V1) is pre-wired to facilitate the extraction of behaviorally important visual features. Collinear edge detectors in V1, for instance, mutually enhance each other to improve the perception of lines against a noisy background. The same pre-wiring that facilitates line extraction, however, is detrimental when subjects have to discriminate the brightness of different line segments. How is it possible to improve in one task by unsupervised practicing, without getting worse in the other task? The classical view of perceptual learning is that practicing modulates the feedforward input stream through synaptic modifications onto or within V1. However, any rewiring of V1 would deteriorate other perceptual abilities different from the trained one. We propose a general neuronal model showing that perceptual learning can modulate top-down input to V1 in a task-specific way while feedforward and lateral pathways remain intact. Consistent with biological data, the model explains how context-dependent brightness discrimination is improved by a top-down recruitment of recurrent inhibition and a top-down induced increase of the neuronal gain within V1. Both the top-down modulation of inhibition and of neuronal gain are suggested to be universal features of cortical microcircuits which enable perceptual learning.     

The role of attention in visual processing

Attention to a visual stimulus typically increases the responses of cortical neurons to that stimulus. Because many studies have shown a close relationship between the performance of individual neurons and behavioural performance of animal subjects, it is important to consider how attention affects this relationship. Measurements of behavioural and neuronal performance taken from rhesus monkeys while they performed a motion detection task with two attentional states show that attention alters the relationship between behaviour and neuronal response. Notably, attention affects the relationship differently in different cortical visual areas. This indicates that a close relationship between neuronal and behavioural performance on a given task persists over changes in attentional state only within limited regions of visual cortex.     

Defining the V5/MT neuronal pool for perceptual decisions in a visual stereo-motion task

In the primate visual cortex, neurons signal differences in the appearance of objects with high precision. However, not all activated neurons contribute directly to perception. We defined the perceptual pool in extrastriate visual area V5/MT for a stereo-motion task, based on trial-by-trial co-variation between perceptual decisions and neuronal firing (choice probability (CP)). Macaque monkeys were trained to discriminate the direction of rotation of a cylinder, using the binocular depth between the moving dots that form its front and rear surfaces. We manipulated the activity of single neurons trial-to-trial by introducing task-irrelevant stimulus changes: dot motion in cylinders was aligned with neuronal preference on only half the trials, so that neurons were strongly activated with high firing rates on some trials and considerably less activated on others. We show that single neurons maintain high neurometric sensitivity for binocular depth in the face of substantial changes in firing rate. CP was correlated with neurometric sensitivity, not level of activation. In contrast, for individual neurons, the correlation between perceptual choice and neuronal activity may be fundamentally different when responding to different stimulus versions. Therefore, neuronal pools supporting sensory discrimination must be structured flexibly and independently for each stimulus configuration to be discriminated.This article is part of the themed issue ‘Vision in our three-dimensional world''.     

Properties of V1 neurons tuned to conjunctions of visual features: application of the V1 saliency hypothesis to visual search behavior

From a computational theory of V1, we formulate an optimization problem to investigate neural properties in the primary visual cortex (V1) from human reaction times (RTs) in visual search. The theory is the V1 saliency hypothesis that the bottom-up saliency of any visual location is represented by the highest V1 response to it relative to the background responses. The neural properties probed are those associated with the less known V1 neurons tuned simultaneously or conjunctively in two feature dimensions. The visual search is to find a target bar unique in color (C), orientation (O), motion direction (M), or redundantly in combinations of these features (e.g., CO, MO, or CM) among uniform background bars. A feature singleton target is salient because its evoked V1 response largely escapes the iso-feature suppression on responses to the background bars. The responses of the conjunctively tuned cells are manifested in the shortening of the RT for a redundant feature target (e.g., a CO target) from that predicted by a race between the RTs for the two corresponding single feature targets (e.g., C and O targets). Our investigation enables the following testable predictions. Contextual suppression on the response of a CO-tuned or MO-tuned conjunctive cell is weaker when the contextual inputs differ from the direct inputs in both feature dimensions, rather than just one. Additionally, CO-tuned cells and MO-tuned cells are often more active than the single feature tuned cells in response to the redundant feature targets, and this occurs more frequently for the MO-tuned cells such that the MO-tuned cells are no less likely than either the M-tuned or O-tuned neurons to be the most responsive neuron to dictate saliency for an MO target.