Toward a unified theory of visual area V4

Visual area V4 is a midtier cortical area in the ventral visual pathway. It is crucial for visual object recognition and has been a focus of many studies on visual attention. However, there is no unifying view of V4's role in visual processing. Neither is there an understanding of how its role in feature processing interfaces with its role in visual attention. This review captures our current knowledge of V4, largely derived from electrophysiological and imaging studies in the macaque monkey. Based on recent discovery of functionally specific domains in V4, we propose that the unifying function of V4 circuitry is to enable selective extraction of specific functional domain-based networks, whether it be by bottom-up specification of object features or by top-down attentionally driven selection.     

Shape perception: complex contour representation in visual area V4

A recent study has put forward a physiologically plausible population model that implements a parts-based shape-coding scheme for macaque visual area V4.     

Feature-based attention in the frontal eye field and area V4 during visual search

When we search for a target in a crowded visual scene, we often use the distinguishing features of the target, such as color or shape, to guide our attention and eye movements. To investigate the neural mechanisms of feature-based attention, we simultaneously recorded neural responses in the frontal eye field (FEF) and area V4 while monkeys performed a visual search task. The responses of cells in both areas were modulated by feature attention, independent of spatial attention, and the magnitude of response enhancement was inversely correlated with the number of saccades needed to find the target. However, an analysis of the latency of sensory and attentional influences on responses suggested that V4 provides bottom-up sensory information about stimulus features, whereas the FEF provides a top-down attentional bias toward target features that modulates sensory processing in V4 and that could be used to guide the eyes to a searched-for target.     

Differential attention-dependent response modulation across cell classes in macaque visual area V4

The cortex contains multiple cell types, but studies of attention have not distinguished between them, limiting understanding of the local circuits that transform attentional feedback into improved visual processing. Parvalbumin-expressing inhibitory interneurons can be distinguished from pyramidal neurons based on their briefer action potential durations. We recorded neurons in area V4 as monkeys performed an attention-demanding task. We find that the distribution of action potential durations is strongly bimodal. Neurons with narrow action potentials have higher firing rates and larger attention-dependent increases in absolute firing rate than neurons with broad action potentials. The percentage increase in response is similar across the two classes. We also find evidence that attention increases the reliability of the neuronal response. This modulation is more than two-fold stronger among putative interneurons. These findings lead to the surprising conclusion that the strongest attentional modulation occurs among local interneurons that do not transmit signals between areas.     

Neurons in area V4 of the macaque translate attended visual features into behaviorally relevant categories

Neural processing at most stages of the primate visual system is modulated by selective attention, such that behaviorally relevant information is emphasized at the expenses of irrelevant, potentially distracting information. The form of attention best understood at the cellular level is when stimuli at a given location in the visual field must be selected (space-based attention). In contrast, fewer single-unit recording studies have so far explored the cellular mechanisms of attention operating on individual stimulus features, specifically when one feature (e.g., color) of an object must guide behavioral responses while a second feature (e.g., shape) of the same object is potentially interfering and therefore must be ignored. Here we show that activity of neurons in macaque area V4 can underlie the selection of elemental object features and their "translation" into a categorical format that can directly contribute to the control of the animal's behavior.     

Interacting roles of attention and visual salience in V4

Attention increases the contrast gain of V4 neurons, causing them to respond to an attended stimulus as though its contrast had increased. When multiple stimuli appear within a neuron's receptive field (RF), the neuron responds primarily to the attended stimulus. This suggests that cortical cells may be "hard wired" to respond preferentially to the highest-contrast stimulus in their RF, and neural systems for attention capitalize on this mechanism by dynamically increasing the effective contrast of the stimulus that is task relevant. To test this, we varied the relative contrast of two stimuli within the recorded neurons' RFs, while the monkeys attended away to another location. Increasing the physical contrast of one stimulus caused V4 neurons to respond preferentially to that stimulus and reduced their responses to competing stimuli. When attention was directed to the lower-contrast stimulus, it partially overcame the influence of a competing, higher-contrast stimulus.     

A direct projection from area V1 to area V3A of rhesus monkey visual cortex

Small cortical lesions were made in regions of the primary visual cortex (V1) representing different retinal eccentricities. It was found that, whereas all parts of V1 project to visual areas V2, V3 and the motion area of the superior temporal sulcus, only parts of V1 representing peripheral eccentricities (in excess of 30 degrees) project directly to visual area V3A.     

Latency of chromatic information in area V4

In the primate visual system, information about color is known to be carried in separate divisions of the retino-geniculo-cortical pathway. From the retina, responses of photoreceptors to short (S), medium (M), and long (L) wavelengths of light are processed in two different opponent pathways. Signals in the S-opponent pathway, or blue/yellow channel, have been found to lag behind signals in the L/M-opponent pathway, or red/green channel in primary visual area V1, and psychophysical studies have suggested similar perceptual delays. However, more recent psychophysical studies have found that perceptual differences are negligible with the proper controls, suggesting that information between the two channels is integrated at some stage of processing beyond V1. To study the timing of color signals further downstream in visual cortex, we examined the responses of neurons in area V4 to colored stimuli varying along the two cardinal axes of the equiluminant opponent color space. We used information theory to measure the mutual information between the stimuli presented and the neural responses in short time windows in order to estimate the latency of color information in area V4. We found that on average, despite the latency difference in V1, information about S-opponent signals arrives in V4 at the same time as information about L/M-opponent signals. This work indicates a convergence of signal timing among chromatic channels within extrastriate cortex.     

Specialized circuits from primary visual cortex to V2 and area MT

Primary visual cortex recombines inputs from magnocellular (M) and parvocellular (P) streams to create functionally specialized outputs. Understanding these input-output relationships is complicated by the fact that layer 4B, which provides outputs to dorsal visual areas, contains multiple cell types. Using a modified rabies virus that expresses green fluorescent protein, we show that layer 4B neurons projecting to MT are a majority spiny stellate, whereas those projecting to V2 are overwhelmingly pyramidal. Regardless of cell type, MT-projecting neurons have larger cell bodies, more dendritic length, and are deeper within layer 4B. Furthermore, MT-projecting pyramidal neurons are located preferentially underneath cytochrome oxidase blobs, indicating that MT-projecting neurons of both types restrict their dendrites to M-recipient zones. We conclude that MT-projecting layer 4B neurons are specialized for the fast transmission of information from the M pathway, while V2-projecting neurons are likely to mediate slower computations involving mixed M and P signals.     

Modulation of activity in human visual area V1 during memory masking

Neurons in the primary visual cortex, V1, are specialized for the processing of elemental features of the visual stimulus, such as orientation and spatial frequency. Recent fMRI evidence suggest that V1 neurons are also recruited in visual perceptual memory; a number of studies using multi-voxel pattern analysis have successfully decoded stimulus-specific information from V1 activity patterns during the delay phase in memory tasks. However, consistent fMRI signal modulations reflecting the memory process have not yet been demonstrated. Here, we report evidence, from three subjects, that the low V1 BOLD activity during retention of low-level visual features is caused by competing interactions between neural populations coding for different values along the spectrum of the dimension remembered. We applied a memory masking paradigm in which the memory representation of a masker stimulus interferes with a delayed spatial frequency discrimination task when its frequency differs from the discriminanda with ±1 octave and found that impaired behavioral performance due to masking is reflected in weaker V1 BOLD signals. This cross-channel inhibition in V1 only occurs with retinotopic overlap between the masker and the sample stimulus of the discrimination task. The results suggest that memory for spatial frequency is a local process in the retinotopically organized visual cortex.     

A sparse object coding scheme in area V4

Sparse coding has long been recognized as a primary goal of image transformation in the visual system. Sparse coding in early visual cortex is achieved by abstracting local oriented spatial frequencies and by excitatory/inhibitory surround modulation. Object responses are thought to be sparse at subsequent processing stages, but neural mechanisms for higher-level sparsification are not known. Here, convergent results from macaque area V4 neural recording and simulated V4 populations trained on natural object contours suggest that sparse coding is achieved in midlevel visual cortex by emphasizing representation of acute convex and concave curvature. We studied 165 V4 neurons with a random, adaptive stimulus strategy to minimize bias and explore an unlimited range of contour shapes. V4 responses were strongly weighted toward contours containing acute convex or concave curvature. In contrast, the tuning distribution in nonsparse simulated V4 populations was strongly weighted toward low curvature. But as sparseness constraints increased, the simulated tuning distribution shifted progressively toward more acute convex and concave curvature, matching the neural recording results. These findings indicate a sparse object coding scheme in midlevel visual cortex based on uncommon but diagnostic regions of acute contour curvature.     

A motion-dependent distortion of retinotopy in area V4

When one element in an apparent motion sequence differs in color from the others, it is perceived as shifted along the motion trajectory. We examined whether V4 neurons encode the physical or perceived location of this "flashed" element by recording neuronal responses while monkeys viewed these stimuli. The retinotopic locus of V4 activity evoked by the flashed element shifted along the motion trajectory. The magnitude of the shift is consistent with the perceptual shift in humans viewing identical stimuli. This retinotopic distortion depended on the presence of a flashed element but was observed for both color-selective and non-color-selective neurons. The distortion was undiminished when the flashed element terminated the sequence, a condition that reduced the perceptual shift in humans. These findings are consistent with a Bayesian model of localization in which perceived location is derived from position signals optimally integrated across visual areas.     

The role of attention in figure-ground segregation in areas V1 and V4 of the visual cortex

Our visual system segments images into objects and background. Figure-ground segregation relies on the detection of feature discontinuities that signal boundaries between the figures and the background and on a complementary region-filling process that groups together image regions with similar features. The neuronal mechanisms for these processes are not well understood and it is unknown how they depend on visual attention. We measured neuronal activity in V1 and V4 in a task where monkeys either made an eye movement to texture-defined figures or ignored them. V1 activity predicted the timing and the direction of the saccade if the figures were task relevant. We found that boundary detection is an early process that depends little on attention, whereas region filling occurs later and is facilitated by visual attention, which acts in an object-based manner. Our findings are explained by a model with local, bottom-up computations for boundary detection and feedback processing for region filling.     

Knowledge-based vision and simple visual machines.

The vast majority of work in machine vision emphasizes the representation of perceived objects and events: it is these internal representations that incorporate the ''knowledge'' in knowledge-based vision or form the ''models'' in model-based vision. In this paper, we discuss simple machine vision systems developed by artificial evolution rather than traditional engineering design techniques, and note that the task of identifying internal representations within such systems is made difficult by the lack of an operational definition of representation at the causal mechanistic level. Consequently, we question the nature and indeed the existence of representations posited to be used within natural vision systems (i.e. animals). We conclude that representations argued for on a priori grounds by external observers of a particular vision system may well be illusory, and are at best place-holders for yet-to-be-identified causal mechanistic interactions. That is, applying the knowledge-based vision approach in the understanding of evolved systems (machines or animals) may well lead to theories and models that are internally consistent, computationally plausible, and entirely wrong.     

Task-specific impairments and enhancements induced by magnetic stimulation of human visual area V5.

Transcranial magnetic stimulation (TMS) can be used to simulate the effects of highly circumscribed brain damage permanently present in some neuropsychological patients, by reversibly disrupting the normal functioning of the cortical area to which it is applied. By using TMS we attempted to recreate deficits similar to those reported in a motion-blind patient and to assess the specificity of deficits when TMS is applied over human area V5. We used six visual search tasks and showed that subjects were impaired in a motion but not a form ''pop-out'' task when TMS was applied over V5. When motion was present, but irrelevant, or when attention to colour and form were required, TMS applied to V5 enhanced performance. When attention to motion was required in a motion-form conjunction search task, irrespective of whether the target was moving or stationary, TMS disrupted performance. These data suggest that attention to different visual attributes involves mutual inhibition between different extrastriate visual areas.     

A human extrastriate area functionally homologous to macaque V4

Extrastriate area V4 is crucial for intermediate form vision and visual attention in nonhuman primates. Human neuroimaging suggests that an area in the lingual sulcus/fusiform gyrus may correspond to ventral V4 (V4v). We studied a human neurological patient, AR, with a putative V4v lesion. The lesion does not affect early visual processing (luminance, orientation, and motion perception). However, it does impair hue perception, intermediate form vision, and visual attention in the upper contralateral visual field. Form deficits occur during discrimination of illusory borders, Glass patterns, curvature, and non-Cartesian patterns. Attention deficits occur during discrimination of the relative positions of object parts, detection of low-salience targets, and orientation discrimination in the presence of distractors. This pattern of deficits is consistent with the known properties of area V4 in nonhuman primates, indicating that AR's lesion affects a cortical region functionally homologous to macaque V4.     

Transcranial magnetic stimulation reveals attentional feedback to area V1 during serial visual search

Visual search tasks have been used to understand how, where and when attention influences visual processing. Current theories suggest the involvement of a high-level "saliency map" that selects a candidate location to focus attentional resources. For a parallel (or "pop-out") task, the first chosen location is systematically the target, but for a serial (or "difficult") task, the system may cycle on a few distractors before finally focusing on the target. This implies that attentional effects upon early visual areas, involving feedback from higher areas, should be visible at longer latencies during serial search. A previous study from Juan & Walsh (2003) had used Transcranial Magnetic Stimulation (TMS) to support this conclusion; however, only a few post-stimulus delays were compared, and no control TMS location was used. Here we applied TMS double-pulses (sub-threshold) to induce a transient inhibition of area V1 at every post-stimulus delay between 100 ms and 500 ms (50 ms steps). The search array was presented either at the location affected by the TMS pulses (previously identified by applying several pulses at supra-threshold intensity to induce phosphene perception), or in the opposite hemifield, which served as a retinotopically-defined control location. Two search tasks were used: a parallel (+ among Ls) and a serial one (T among Ls). TMS specifically impaired the serial, but not the parallel search. We highlight an involvement of V1 in serial search 300 ms after the onset; conversely, V1 did not contribute to parallel search at delays beyond 100 ms. This study supports the idea that serial search differs from parallel search by the presence of additional cycles of a select-and-focus iterative loop between V1 and higher-level areas.     

The architecture of visual cortex and inferential processes in vision

This paper is organised approximately into two halves. In the first half, I review evidence about the structure of the visual system, and I use that evidence to frame what I think are widely held but often implicit ideas about how it works, namely that vision is principally analysis of retinal input. These ideas have been strongly influenced by engineering approaches; form a default view of the visual system that suffuses all the language used to describe it (at least in visual neuroscience); and are to some extent supported by the structural evidence. In the second half, I explore some inconvenient facts from neuroanatomy and neurophysiology which are quite uncomfortable for the traditional view. I then set out a contrary view of how structure and function are linked in the visual system, which is a neurobiological variety of the quite developed view in psychophysics that vision is better understood as knowledge-rich inference. Finally, I explore some of the ramifications of this view for neurophysiological understanding of how the visual system might operate during normal vision.