Toward a unified theory of visual area V4

Visual area V4 is a midtier cortical area in the ventral visual pathway. It is crucial for visual object recognition and has been a focus of many studies on visual attention. However, there is no unifying view of V4's role in visual processing. Neither is there an understanding of how its role in feature processing interfaces with its role in visual attention. This review captures our current knowledge of V4, largely derived from electrophysiological and imaging studies in the macaque monkey. Based on recent discovery of functionally specific domains in V4, we propose that the unifying function of V4 circuitry is to enable selective extraction of specific functional domain-based networks, whether it be by bottom-up specification of object features or by top-down attentionally driven selection.     

Feedback contributions to visual awareness in human occipital cortex

It has traditionally been assumed that processing within the visual system proceeds in a bottom-up, feedforward manner from retina to higher cortical areas. In addition to feedforward processing, it is now clear that there are also important contributions to sensory encoding that rely upon top-down, feedback (reentrant) projections from higher visual areas to lower ones. By utilizing transcranial magnetic stimulation (TMS) in a metacontrast masking paradigm, we addressed whether feedback processes in early visual cortex play a role in visual awareness. We show that TMS of visual cortex, when timed to produce visual suppression of an annulus serving as a metacontrast mask, induces recovery of an otherwise imperceptible disk. In addition to producing disk recovery, TMS suppression of an annulus was greater when a disk preceded it than when an annulus was presented alone. This latter result suggests that there are effects of the disk on the perceptibility of the subsequent mask that are additive and are revealed with TMS of the visual cortex. These results demonstrate spatial and temporal interactions of conscious vision in visual cortex and suggest that a prior visual stimulus can influence subsequent perception at early stages of visual encoding via feedback projections.     

Recurrent V1–V2 interaction in early visual boundary processing

A majority of cortical areas are connected via feedforward and feedback fiber projections. In feedforward pathways we mainly observe stages of feature detection and integration. The computational role of the descending pathways at different stages of processing remains mainly unknown. Based on empirical findings we suggest that the top-down feedback pathways subserve a context-dependent gain control mechanism. We propose a new computational model for recurrent contour processing in which normalized activities of orientation selective contrast cells are fed forward to the next processing stage. There, the arrangement of input activation is matched against local patterns of contour shape. The resulting activities are subsequently fed back to the previous stage to locally enhance those initial measurements that are consistent with the top-down generated responses. In all, we suggest a computational theory for recurrent processing in the visual cortex in which the significance of local measurements is evaluated on the basis of a broader visual context that is represented in terms of contour code patterns. The model serves as a framework to link physiological with perceptual data gathered in psychophysical experiments. It handles a variety of perceptual phenomena, such as the local grouping of fragmented shape outline, texture surround and density effects, and the interpolation of illusory contours. Received: 28 October 1998 / Accepted in revised form: 19 March 1999     

Disrupted development and imbalanced function in the global neuronal workspace: a positive-feedback mechanism for the emergence of ASD in early infancy

Autism spectrum disorder (ASD) is increasingly being conceptualized as a spectrum disorder of connectome development. We review evidence suggesting that ASD is characterized by a positive feedback loop that amplifies small functional variations in early-developing sensory-processing pathways into structural and functional imbalances in the global neuronal workspace. Using vision as an example, we discuss how early functional variants in visual processing may be feedback-amplified to produce variant object categories and disrupted top-down expectations, atypically large expectation-to-perception mismatches, problems re-identifying individual people and objects, socially inappropriate, generally aversive emotional responses and disrupted sensory-motor coordination. Viewing ASD in terms of feedback amplification of small functional variants allows a number of recent models of ASD to be integrated with neuroanatomical, neurofunctional and genetic data.     

Remembering visual motion: neural correlates of associative plasticity and motion recall in cortical area MT

The pictorial content of visual memories recalled by association is embodied by neuronal activity at the highest processing stages of primate visual cortex. This activity is elicited by top-down signals from the frontal lobe and recapitulates the bottom-up pattern normally obtained by the recalled stimulus. To explore the generality and mechanisms of this phenomenon, we recorded motion-sensitive neurons at an early stage of cortical processing. After monkeys learned to associate directions of motion with static shapes, these neurons exhibited unprecedented selectivity for the shapes. This emergent shape selectivity reflects activation of neurons representing the motion stimuli recalled by association, and it suggests that recall-related activity may be a general feature of neurons in visual cortex.     

Top-Down but Not Bottom-Up Visual Scanning is Affected in Hereditary Pure Cerebellar Ataxia

The aim of this study was to clarify the nature of visual processing deficits caused by cerebellar disorders. We studied the performance of two types of visual search (top-down visual scanning and bottom-up visual scanning) in 18 patients with pure cerebellar types of spinocerebellar degeneration (SCA6: 11; SCA31: 7). The gaze fixation position was recorded with an eye-tracking device while the subjects performed two visual search tasks in which they looked for a target Landolt figure among distractors. In the serial search task, the target was similar to the distractors and the subject had to search for the target by processing each item with top-down visual scanning. In the pop-out search task, the target and distractor were clearly discernible and the visual salience of the target allowed the subjects to detect it by bottom-up visual scanning. The saliency maps clearly showed that the serial search task required top-down visual attention and the pop-out search task required bottom-up visual attention. In the serial search task, the search time to detect the target was significantly longer in SCA patients than in normal subjects, whereas the search time in the pop-out search task was comparable between the two groups. These findings suggested that SCA patients cannot efficiently scan a target using a top-down attentional process, whereas scanning with a bottom-up attentional process is not affected. In the serial search task, the amplitude of saccades was significantly smaller in SCA patients than in normal subjects. The variability of saccade amplitude (saccadic dysmetria), number of re-fixations, and unstable fixation (nystagmus) were larger in SCA patients than in normal subjects, accounting for a substantial proportion of scattered fixations around the items. Saccadic dysmetria, re-fixation, and nystagmus may play important roles in the impaired top-down visual scanning in SCA, hampering precise visual processing of individual items.     

Category differences in brain activation studies: where do they come from?

Differences in the neural processing of six categories of pictorial stimuli (maps, body parts, objects, animals, famous faces and colours) were investigated using positron emission tomography. Stimuli were presented either with or without the written name of the picture, thereby creating a naming condition and a reading condition. As predicted, naming increased the demands on lexical processes. This was demonstrated by activation of the left temporal lobe in a posterior region associated with name retrieval in several previous studies. This lexical effect was common to all meaningful stimuli and no category-specific effects were observed for naming relative to reading. Nevertheless, category differences were found when naming and reading were considered together. Stimuli with greater visual complexity (animals, faces and maps) enhanced activation in the left extrastriate cortex. Furthermore, map recognition, which requires greater spatio-topographical processing, also activated the right occipito-parietal and parahippocampal cortices. These effects in the visuo-spatial regions emphasize inevitable differences in the perceptual properties of pictorial stimuli. In the semantic temporal regions, famous faces and objects enhanced activation in the left antero-lateral and postero-lateral cortices, respectively. In addition, we showed that the same posterior left temporal region is also activated by body parts. We conclude that category-specific brain activations depend more on differential processing at the perceptual and semantic levels rather than at the lexical retrieval level.     

Semantic Wavelet-Induced Frequency-Tagging (SWIFT) Periodically Activates Category Selective Areas While Steadily Activating Early Visual Areas

Primate visual systems process natural images in a hierarchical manner: at the early stage, neurons are tuned to local image features, while neurons in high-level areas are tuned to abstract object categories. Standard models of visual processing assume that the transition of tuning from image features to object categories emerges gradually along the visual hierarchy. Direct tests of such models remain difficult due to confounding alteration in low-level image properties when contrasting distinct object categories. When such contrast is performed in a classic functional localizer method, the desired activation in high-level visual areas is typically accompanied with activation in early visual areas. Here we used a novel image-modulation method called SWIFT (semantic wavelet-induced frequency-tagging), a variant of frequency-tagging techniques. Natural images modulated by SWIFT reveal object semantics periodically while keeping low-level properties constant. Using functional magnetic resonance imaging (fMRI), we indeed found that faces and scenes modulated with SWIFT periodically activated the prototypical category-selective areas while they elicited sustained and constant responses in early visual areas. SWIFT and the localizer were selective and specific to a similar extent in activating category-selective areas. Only SWIFT progressively activated the visual pathway from low- to high-level areas, consistent with predictions from standard hierarchical models. We confirmed these results with criterion-free methods, generalizing the validity of our approach and show that it is possible to dissociate neural activation in early and category-selective areas. Our results provide direct evidence for the hierarchical nature of the representation of visual objects along the visual stream and open up future applications of frequency-tagging methods in fMRI.     

Perceptual load-dependent neural correlates of distractor interference inhibition

Background

The load theory of selective attention hypothesizes that distractor interference is suppressed after perceptual processing (i.e., in the later stage of central processing) at low perceptual load of the central task, but in the early stage of perceptual processing at high perceptual load. Consistently, studies on the neural correlates of attention have found a smaller distractor-related activation in the sensory cortex at high relative to low perceptual load. However, it is not clear whether the distractor-related activation in brain regions linked to later stages of central processing (e.g., in the frontostriatal circuits) is also smaller at high rather than low perceptual load, as might be predicted based on the load theory.

Methodology/Principal Findings

We studied 24 healthy participants using functional magnetic resonance imaging (fMRI) during a visual target identification task with two perceptual loads (low vs. high). Participants showed distractor-related increases in activation in the midbrain, striatum, occipital and medial and lateral prefrontal cortices at low load, but distractor-related decreases in activation in the midbrain ventral tegmental area and substantia nigra (VTA/SN), striatum, thalamus, and extensive sensory cortices at high load.

Conclusions

Multiple levels of central processing involving midbrain and frontostriatal circuits participate in suppressing distractor interference at either low or high perceptual load. For suppressing distractor interference, the processing of sensory inputs in both early and late stages of central processing are enhanced at low load but inhibited at high load.     

Humor as a Reward Mechanism: Event-Related Potentials in the Healthy and Diseased Brain

Humor processing involves distinct processing stages including incongruity detection, emotional response, and engagement of mesolimbic reward regions. Dysfunctional reward processing and clinical symptoms in response to humor have been previously described in both hypocretin deficient narcolepsy-cataplexy (NC) and in idiopathic Parkinson disease (PD). For NC patients, humor is the strongest trigger for cataplexy, a transient loss of muscle tone, whereas dopamine-deficient PD-patients show blunted emotional responses to humor. To better understand the role of reward system and the various contributions of hypocretinergic and dopaminergic mechanisms to different stages of humor processing we examined the electrophysiological response to humorous and neutral pictures when given as reward feedback in PD, NC and healthy controls. Humor compared to neutral feedback demonstrated modulation of early ERP amplitudes likely corresponding to visual processing stages, with no group differences. At 270 ms post-feedback, conditions showed topographical and amplitudinal differences for frontal and left posterior electrodes, in that humor feedback was absent in PD patients but increased in NC patients. We suggest that this effect relates to a relatively early affective response, reminiscent of increased amygdala response reported in NC patients. Later ERP differences, corresponding to the late positive potential, revealed a lack of sustained activation in PD, likely due to altered dopamine regulation in reward structures in these patients. This research provides new insights into the temporal dynamics and underlying mechanisms of humor detection and appreciation in health and disease.     

Occipital Alpha Activity during Stimulus Processing Gates the Information Flow to Object-Selective Cortex

Given the limited processing capabilities of the sensory system, it is essential that attended information is gated to downstream areas, whereas unattended information is blocked. While it has been proposed that alpha band (8–13 Hz) activity serves to route information to downstream regions by inhibiting neuronal processing in task-irrelevant regions, this hypothesis remains untested. Here we investigate how neuronal oscillations detected by electroencephalography in visual areas during working memory encoding serve to gate information reflected in the simultaneously recorded blood-oxygenation-level-dependent (BOLD) signals recorded by functional magnetic resonance imaging in downstream ventral regions. We used a paradigm in which 16 participants were presented with faces and landscapes in the right and left hemifields; one hemifield was attended and the other unattended. We observed that decreased alpha power contralateral to the attended object predicted the BOLD signal representing the attended object in ventral object-selective regions. Furthermore, increased alpha power ipsilateral to the attended object predicted a decrease in the BOLD signal representing the unattended object. We also found that the BOLD signal in the dorsal attention network inversely correlated with visual alpha power. This is the first demonstration, to our knowledge, that oscillations in the alpha band are implicated in the gating of information from the visual cortex to the ventral stream, as reflected in the representationally specific BOLD signal. This link of sensory alpha to downstream activity provides a neurophysiological substrate for the mechanism of selective attention during stimulus processing, which not only boosts the attended information but also suppresses distraction. Although previous studies have shown a relation between the BOLD signal from the dorsal attention network and the alpha band at rest, we demonstrate such a relation during a visuospatial task, indicating that the dorsal attention network exercises top-down control of visual alpha activity.     

A model of the ventral visual system based on temporal stability and local memory

The cerebral cortex is a remarkably homogeneous structure suggesting a rather generic computational machinery. Indeed, under a variety of conditions, functions attributed to specialized areas can be supported by other regions. However, a host of studies have laid out an ever more detailed map of functional cortical areas. This leaves us with the puzzle of whether different cortical areas are intrinsically specialized, or whether they differ mostly by their position in the processing hierarchy and their inputs but apply the same computational principles. Here we show that the computational principle of optimal stability of sensory representations combined with local memory gives rise to a hierarchy of processing stages resembling the ventral visual pathway when it is exposed to continuous natural stimuli. Early processing stages show receptive fields similar to those observed in the primary visual cortex. Subsequent stages are selective for increasingly complex configurations of local features, as observed in higher visual areas. The last stage of the model displays place fields as observed in entorhinal cortex and hippocampus. The results suggest that functionally heterogeneous cortical areas can be generated by only a few computational principles and highlight the importance of the variability of the input signals in forming functional specialization.     

Cerebral Asymmetry of fMRI-BOLD Responses to Visual Stimulation

Hemispheric asymmetry of a wide range of functions is a hallmark of the human brain. The visual system has traditionally been thought of as symmetrically distributed in the brain, but a growing body of evidence has challenged this view. Some highly specific visual tasks have been shown to depend on hemispheric specialization. However, the possible lateralization of cerebral responses to a simple checkerboard visual stimulation has not been a focus of previous studies. To investigate this, we performed two sessions of blood-oxygenation level dependent (BOLD) functional magnetic resonance imaging (fMRI) in 54 healthy subjects during stimulation with a black and white checkerboard visual stimulus. While carefully excluding possible non-physiological causes of left-to-right bias, we compared the activation of the left and the right cerebral hemispheres and related this to grey matter volume, handedness, age, gender, ocular dominance, interocular difference in visual acuity, as well as line-bisection performance. We found a general lateralization of cerebral activation towards the right hemisphere of early visual cortical areas and areas of higher-level visual processing, involved in visuospatial attention, especially in top-down (i.e., goal-oriented) attentional processing. This right hemisphere lateralization was partly, but not completely, explained by an increased grey matter volume in the right hemisphere of the early visual areas. Difference in activation of the superior parietal lobule was correlated with subject age, suggesting a shift towards the left hemisphere with increasing age. Our findings suggest a right-hemispheric dominance of these areas, which could lend support to the generally observed leftward visual attentional bias and to the left hemifield advantage for some visual perception tasks.     

Atypical Balance between Occipital and Fronto-Parietal Activation for Visual Shape Extraction in Dyslexia

Reading requires the extraction of letter shapes from a complex background of text, and an impairment in visual shape extraction would cause difficulty in reading. To investigate the neural mechanisms of visual shape extraction in dyslexia, we used functional magnetic resonance imaging (fMRI) to examine brain activation while adults with or without dyslexia responded to the change of an arrow’s direction in a complex, relative to a simple, visual background. In comparison to adults with typical reading ability, adults with dyslexia exhibited opposite patterns of atypical activation: decreased activation in occipital visual areas associated with visual perception, and increased activation in frontal and parietal regions associated with visual attention. These findings indicate that dyslexia involves atypical brain organization for fundamental processes of visual shape extraction even when reading is not involved. Overengagement in higher-order association cortices, required to compensate for underengagment in lower-order visual cortices, may result in competition for top-down attentional resources helpful for fluent reading.     

Feature-based attention: it is all bottom-up priming

Feature-based attention (FBA) enhances the representation of image characteristics throughout the visual field, a mechanism that is particularly useful when searching for a specific stimulus feature. Even though most theories of visual search implicitly or explicitly assume that FBA is under top-down control, we argue that the role of top-down processing in FBA may be limited. Our review of the literature indicates that all behavioural and neuro-imaging studies investigating FBA suffer from the shortcoming that they cannot rule out an effect of priming. The mere attending to a feature enhances the mandatory processing of that feature across the visual field, an effect that is likely to occur in an automatic, bottom-up way. Studies that have investigated the feasibility of FBA by means of cueing paradigms suggest that the role of top-down processing in FBA is limited (e.g. prepare for red). Instead, the actual processing of the stimulus is needed to cause the mandatory tuning of responses throughout the visual field. We conclude that it is likely that all FBA effects reported previously are the result of bottom-up priming.     

Hemispheric Asymmetry in the Auditory Facilitation Effect in Dual-Stream Rapid Serial Visual Presentation Tasks

Even though auditory stimuli do not directly convey information related to visual stimuli, they often improve visual detection and identification performance. Auditory stimuli often alter visual perception depending on the reliability of the sensory input, with visual and auditory information reciprocally compensating for ambiguity in the other sensory domain. Perceptual processing is characterized by hemispheric asymmetry. While the left hemisphere is more involved in linguistic processing, the right hemisphere dominates spatial processing. In this context, we hypothesized that an auditory facilitation effect in the right visual field for the target identification task, and a similar effect would be observed in the left visual field for the target localization task. In the present study, we conducted target identification and localization tasks using a dual-stream rapid serial visual presentation. When two targets are embedded in a rapid serial visual presentation stream, the target detection or discrimination performance for the second target is generally lower than for the first target; this deficit is well known as attentional blink. Our results indicate that auditory stimuli improved target identification performance for the second target within the stream when visual stimuli were presented in the right, but not the left visual field. In contrast, auditory stimuli improved second target localization performance when visual stimuli were presented in the left visual field. An auditory facilitation effect was observed in perceptual processing, depending on the hemispheric specialization. Our results demonstrate a dissociation between the lateral visual hemifield in which a stimulus is projected and the kind of visual judgment that may benefit from the presentation of an auditory cue.     

How does the cerebral cortex work? Learning, attention, and grouping by the laminar circuits of visual cortex

The organization of neocortex into layers is one of its most salient anatomical features. These layers include circuits that form functional columns in cortical maps. A major unsolved problem concerns how bottom-up, top-down, and horizontal interactions are organized within cortical layers to generate adaptive behaviors. This article models how these interactions help visual cortex to realize: (i) the binding process whereby cortex groups distributed data into coherent object representations; (ii) the attentional process whereby cortex selectively processes important events; and (iii) the developmental and learning processes whereby cortex shapes its circuits to match environmental constraints. New computational ideas about feedback systems suggest how neocortex develops and learns in a stable way, and why top-down attention requires converging bottom-up inputs to fully activate cortical cells, whereas perceptual groupings do not.     

No evidence for early modulation of evoked responses in primary visual cortex to irrelevant probe stimuli presented during the attentional blink

Background

During rapid serial visual presentation (RSVP), observers often miss the second of two targets if it appears within 500 ms of the first. This phenomenon, called the attentional blink (AB), is widely held to reflect a bottleneck in the processing of rapidly sequential stimuli that arises after initial sensory registration is complete (i.e., at a relatively late, post-perceptual stage of processing). Contrary to this view, recent fMRI studies have found that activity in the primary visual area (V1), which represents the earliest cortical stage of visual processing, is attenuated during the AB. Here we asked whether such changes in V1 activity during the AB arise in the initial feedforward sweep of stimulus input, or instead reflect the influence of feedback signals from higher cortical areas.

Methodology/Principal Findings

EEG signals were recorded while participants monitored a sequential stream of distractor letters for two target digits (T1 and T2). Neural responses associated with an irrelevant probe stimulus presented simultaneously with T2 were measured using an ERP marker – the C1 component – that reflects initial perceptual processing of visual information in V1. As expected, T2 accuracy was compromised when the inter-target interval was brief, reflecting an AB deficit. Critically, however, the magnitude of the early C1 component evoked by the probe was not reduced during the AB.

Conclusions/Significance

Our finding that early sensory processing of irrelevant probe stimuli is not suppressed during the AB is consistent with theoretical models that assume that the bottleneck underlying the AB arises at a post-perceptual stage of processing. This suggests that reduced neural activity in V1 during the AB is driven by re-entrant signals from extrastriate areas that regulate early cortical activity via feedback connections with V1.     

Neural representation of objects in space: a dual coding account.

I present evidence on the nature of object coding in the brain and discuss the implications of this coding for models of visual selective attention. Neuropsychological studies of task-based constraints on: (i) visual neglect; and (ii) reading and counting, reveal the existence of parallel forms of spatial representation for objects: within-object representations, where elements are coded as parts of objects, and between-object representations, where elements are coded as independent objects. Aside from these spatial codes for objects, however, the coding of visual space is limited. We are extremely poor at remembering small spatial displacements across eye movements, indicating (at best) impoverished coding of spatial position per se. Also, effects of element separation on spatial extinction can be eliminated by filling the space with an occluding object, indicating that spatial effects on visual selection are moderated by object coding. Overall, there are separate limits on visual processing reflecting: (i) the competition to code parts within objects; (ii) the small number of independent objects that can be coded in parallel; and (iii) task-based selection of whether within- or between-object codes determine behaviour. Between-object coding may be linked to the dorsal visual system while parallel coding of parts within objects takes place in the ventral system, although there may additionally be some dorsal involvement either when attention must be shifted within objects or when explicit spatial coding of parts is necessary for object identification.     

Attentional weighting: a possible account of visual field asymmetries in visual search?

Several previous visual search studies measuring reaction times have demonstrated scanning biases across the visual field (i.e. a tendency to begin a serial search in a particular region of space). In the present study, we measured visual discrimination thresholds for a target presented amongst distractors using displays that were short enough to greatly reduce the potential for serial (i.e. scanning) search. For both a motion and orientation task, subjects' performance was significantly better when the target appeared in the inferior, as compared to the superior, visual field (no differences were observed between left and right visual fields). These findings suggest that subjects may divide attention unevenly across the visual field when searching for a target amongst distractors, a phenomenon we refer to as 'attentional weighting'. To rule out the possibility that these visual field asymmetries were sensory in nature, thresholds were also measured for conditions in which subjects' attention was directed to the location of the target stimulus, either because it was presented alone in the display or because a spatial cue directed subjects' attention to the location of that target presented amongst distractors. Under these conditions, visual field asymmetries were smaller (or non-existent), suggesting that sensory factors (such as crowding) are unlikely to account for our results. In addition, analyses of set-size effects (obtained by comparing thresholds for a single target vs. the target presented amongst distractors) could be accounted for by an unlimited capacity model, suggesting that multiple stimuli can be processed simultaneously without any limitations at an early stage of sensory processing. Taken together, these findings suggest the possible existence of biases in attentional weighting at a late stage of processing. The bias appears to favor the inferior visual field, which may arise from the fact that there is more ecologically-relevant information in this region of space.