Scene segmentation by spike synchronization in reciprocally connected visual areas. I. Local effects of cortical feedback

 To investigate scene segmentation in the visual system we present a model of two reciprocally connected visual areas using spiking neurons. Area P corresponds to the orientation-selective subsystem of the primary visual cortex, while the central visual area C is modeled as associative memory representing stimulus objects according to Hebbian learning. Without feedback from area C, a single stimulus results in relatively slow and irregular activity, synchronized only for neighboring patches (slow state), while in the complete model activity is faster with an enlarged synchronization range (fast state). When presenting a superposition of several stimulus objects, scene segmentation happens on a time scale of hundreds of milliseconds by alternating epochs of the slow and fast states, where neurons representing the same object are simultaneously in the fast state. Correlation analysis reveals synchronization on different time scales as found in experiments (designated as tower, castle, and hill peaks). On the fast time scale (tower peaks, gamma frequency range), recordings from two sites coding either different or the same object lead to correlograms that are either flat or exhibit oscillatory modulations with a central peak. This is in agreement with experimental findings, whereas standard phase-coding models would predict shifted peaks in the case of different objects. Received: 22 August 2001 / Accepted in revised form: 8 April 2002     

Neural plasticity in high-level visual cortex underlying object perceptual learning

With intensive training, human can achieve impressive behavioral improvement on various perceptual tasks. This phenomenon, termed perceptual learning, has long been considered as a hallmark of the plasticity of sensory neural system. Not surprisingly, high-level vision, such as object perception, can also be improved by perceptual learning. Here we review recent psychophysical, electrophysiological, and neuroimaging studies investigating the effects of training on object selective cortex, such as monkey inferior temporal cortex and human lateral occipital area. Evidences show that learning leads to an increase in object selectivity at the single neuron level and/or the neuronal population level. These findings indicate that high-level visual cortex in humans is highly plastic and visual experience can strongly shape neural functions of these areas. At the end of the review, we discuss several important future directions in this area.     

The intrinsic, association and commissural connections of area 17 on the visual cortex.

An experimental neurohistological study has been made of the intrinsic connections of the cortex of area 17 of the monkey, of the commissural connections of the visual cortex of the cat and monkey and of the association fibres passing into area 17 of the cat. In light microscopic studies the axonal degeneration method of Nauta has been used, and the site and mode of termination of the degenerating fibres has also been determined with the electron microscope...     

Neural correlates of visual motion prediction

Predicting the trajectories of moving objects in our surroundings is important for many life scenarios, such as driving, walking, reaching, hunting and combat. We determined human subjects' performance and task-related brain activity in a motion trajectory prediction task. The task required spatial and motion working memory as well as the ability to extrapolate motion information in time to predict future object locations. We showed that the neural circuits associated with motion prediction included frontal, parietal and insular cortex, as well as the thalamus and the visual cortex. Interestingly, deactivation of many of these regions seemed to be more closely related to task performance. The differential activity during motion prediction vs. direct observation was also correlated with task performance. The neural networks involved in our visual motion prediction task are significantly different from those that underlie visual motion memory and imagery. Our results set the stage for the examination of the effects of deficiencies in these networks, such as those caused by aging and mental disorders, on visual motion prediction and its consequences on mobility related daily activities.     

Medial axis shape coding in macaque inferotemporal cortex

The basic, still unanswered question about visual object representation is this: what specific information is encoded by neural signals? Theorists have long predicted that neurons would encode medial axis or skeletal object shape, yet recent studies reveal instead neural coding of boundary or surface shape. Here, we addressed this theoretical/experimental disconnect, using adaptive shape sampling to demonstrate explicit coding of medial axis shape in high-level object cortex (macaque monkey inferotemporal cortex or IT). Our metric shape analyses revealed a coding continuum, along which most neurons represent a configuration of both medial axis and surface components. Thus, IT response functions embody a rich basis set for simultaneously representing skeletal and external shape of complex objects. This would be especially useful for representing biological shapes, which are often characterized by both complex, articulated skeletal structure and specific surface features.     

Crossmodal processing of object features in human anterior intraparietal cortex: an fMRI study implies equivalencies between humans and monkeys

The organization of macaque posterior parietal cortex (PPC) reflects its functional specialization in integrating polymodal sensory information for object recognition and manipulation. Neuropsychological and recent human imaging studies imply equivalencies between human and macaque PPC, and in particular, the cortex buried in the intraparietal sulcus (IPS). Using functional MRI, we tested the hypothesis that an area in human anterior intraparietal cortex is activated when healthy subjects perform a crossmodal visuo-tactile delayed matching-to-sample task with objects. Tactile or visual object presentation (encoding and recognition) both significantly activated anterior intraparietal cortex. As hypothesized, neural activity in this area was further enhanced when subjects transferred object information between modalities (crossmodal matching). Based on both the observed functional properties and the anatomical location, we suggest that this area in anterior IPS is the human equivalent of macaque area AIP.     

Invariant visual object recognition: a model, with lighting invariance.

How are invariant representations of objects formed in the visual cortex? We describe a neurophysiological and computational approach which focusses on a feature hierarchy model in which invariant representations can be built by self-organizing learning based on the statistics of the visual input. The model can use temporal continuity in an associative synaptic learning rule with a short term memory trace, and/or it can use spatial continuity in Continuous Transformation learning. The model of visual processing in the ventral cortical stream can build representations of objects that are invariant with respect to translation, view, size, and in this paper we show also lighting. The model has been extended to provide an account of invariant representations in the dorsal visual system of the global motion produced by objects such as looming, rotation, and object-based movement. The model has been extended to incorporate top-down feedback connections to model the control of attention by biased competition in for example spatial and object search tasks. The model has also been extended to account for how the visual system can select single objects in complex visual scenes, and how multiple objects can be represented in a scene.     

Toward a unified theory of visual area V4

Visual area V4 is a midtier cortical area in the ventral visual pathway. It is crucial for visual object recognition and has been a focus of many studies on visual attention. However, there is no unifying view of V4's role in visual processing. Neither is there an understanding of how its role in feature processing interfaces with its role in visual attention. This review captures our current knowledge of V4, largely derived from electrophysiological and imaging studies in the macaque monkey. Based on recent discovery of functionally specific domains in V4, we propose that the unifying function of V4 circuitry is to enable selective extraction of specific functional domain-based networks, whether it be by bottom-up specification of object features or by top-down attentionally driven selection.     

Patchy organization and asymmetric distribution of the neural correlates of face processing in monkey inferotemporal cortex

BACKGROUND: It is believed that a face-specific system exists within the primate ventral visual pathway that is separate from a domain-general nonface object coding system. In addition, it is believed that hemispheric asymmetry, which was long held to be a distinct feature of the human brain, can be found in the brains of other primates as well. We show here for the first time by way of a functional imaging technique that face- and object-selective neurons form spatially distinct clusters at the cellular level in monkey inferotemporal cortex. We have used a novel functional mapping technique that simultaneously generates two separate activity profiles by exploiting the differential time course of zif268 mRNA and protein expression. RESULTS: We show that neurons activated by face stimulation can be visualized at cellular resolution and distinguished from those activated by nonface complex objects. Our dual-activity maps of face and object selectivity show that face-selective patches of various sizes (mean, 22.30 mm2; std, 32.76 mm2) exist throughout the IT cortex in the context of a large expanse of cortical territory that is responsive to visual objects. CONCLUSIONS: These results add to recent findings that face-selective patches of various sizes exist throughout area IT and provide the first direct anatomical evidence at cellular resolution for a hemispheric asymmetry in favor of the right hemisphere. Together, our results support the notion that human and monkey brains share a similarity in both anatomical organization and distribution of function with respect to high-level visual processing.     

Differential effects of viewpoint on object-driven activation in dorsal and ventral streams

Using fMRI, we showed that an area in the ventral temporo-occipital cortex (area vTO), which is part of the human homolog of the ventral stream of visual processing, exhibited priming for both identical and depth-rotated images of objects. This pattern of activation in area vTO corresponded to performance in a behavioral matching task. An area in the caudal part of the intraparietal sulcus (area cIPS) also showed priming, but only with identical images of objects. This dorsal-stream area treated rotated images as new objects. The difference in the pattern of priming-related activation in the two areas may reflect the respective roles of the ventral and dorsal streams in object recognition and object-directed action.     

Computations in the early visual cortex.

This paper reviews some of the recent neurophysiological studies that explore the variety of visual computations in the early visual cortex in relation to geometric inference, i.e. the inference of contours, surfaces and shapes. It attempts to draw connections between ideas from computational vision and findings from awake primate electrophysiology. In the classical feed-forward, modular view of visual processing, the early visual areas (LGN, V1 and V2) are modules that serve to extract local features, while higher extrastriate areas are responsible for shape inference and invariant object recognition. However, recent findings in primate early visual systems reveal that the computations in the early visual cortex are rather complex and dynamic, as well as interactive and plastic, subject to influence from global context, higher order perceptual inference, task requirement and behavioral experience. The evidence argues that the early visual cortex does not merely participate in the first stage of visual processing, but is involved in many levels of visual computation.     

From visual affordances in monkey parietal cortex to hippocampo-parietal interactions underlying rat navigation.

This paper explores the hypothesis that various subregions (but by no means all) of the posterior parietal cortex are specialized to process visual information to extract a variety of affordances for behaviour. Two biologically based models of regions of the posterior parietal cortex of the monkey are introduced. The model of the lateral intraparietal area (LIP) emphasizes its roles in dynamic remapping of the representation of targets during a double saccade task, and in combining stored, updated input with current visual input. The model of the anterior intraparietal area (AIP) addresses parietal-premotor interactions involved in grasping, and analyses the interaction between the AIP and premotor area F5. The model represents the role of other intraparietal areas working in concert with the inferotemporal cortex as well as with corollary discharge from F5 to provide and augment the affordance information in the AIP, and suggests how various constraints may resolve the action opportunities provided by multiple affordances. Finally, a systems-level model of hippocampo parietal interactions underlying rat navigation is developed, motivated by the monkey data used in developing the above two models as well as by data on neurones in the posterior parietal cortex of the monkey that are sensitive to visual motion. The formal similarity between dynamic remapping (primate saccades) and path integration (rat navigation) is noted, and certain available data on rat posterior parietal cortex in terms of affordances for locomotion are explained. The utility of further modelling, linking the World Graph model of cognitive maps for motivated behaviour with hippocampal-parietal interactions involved in navigation, is also suggested. These models demonstrate that posterior parietal cortex is not only itself a network of interacting subsystems, but functions through cooperative computation with many other brain regions.     

Effects of visual experience on the representation of objects in the prefrontal cortex

The perception and recognition of objects are improved by experience. Here, we show that monkeys' ability to recognize degraded objects was improved by several days of practice with these objects. This improvement was reflected in the activity of neurons in the prefrontal (PF) cortex, a brain region critical for a wide range of visual behaviors. Familiar objects activated fewer neurons than did novel objects, but these neurons were more narrowly tuned, and the object representation was more resistant to the effects of degradation, after experience. These results demonstrate a neural correlate of visual learning in the PF cortex of adult monkeys.     

A network that uses few active neurones to code visual input predicts the diverse shapes of cortical receptive fields

Computational models of primary visual cortex have demonstrated that principles of efficient coding and neuronal sparseness can explain the emergence of neurones with localised oriented receptive fields. Yet, existing models have failed to predict the diverse shapes of receptive fields that occur in nature. The existing models used a particular "soft" form of sparseness that limits average neuronal activity. Here we study models of efficient coding in a broader context by comparing soft and "bard" forms of neuronal sparseness. As a result of our analyses, we propose a novel network model for visual cortex. The model forms efficient visual representations in which the number of active neurones, rather than mean neuronal activity, is limited. This form of hard sparseness also economises cortical resources like synaptic memory and metabolic energy. Furthermore, our model accurately predicts the distribution of receptive field shapes found in the primary visual cortex of cat and monkey.     

Human Left Ventral Premotor Cortex Mediates Matching of Hand Posture to Object Use

Visuomotor transformations for grasping have been associated with a fronto-parietal network in the monkey brain. The human homologue of the parietal monkey region (AIP) has been identified as the anterior part of the intraparietal sulcus (aIPS), whereas the putative human equivalent of the monkey frontal region (F5) is located in the ventral part of the premotor cortex (vPMC). Results from animal studies suggest that monkey F5 is involved in the selection of appropriate hand postures relative to the constraints of the task. In humans, the functional roles of aIPS and vPMC appear to be more complex and the relative contribution of each region to grasp selection remains uncertain. The present study aimed to identify modulation in brain areas sensitive to the difficulty level of tool object - hand posture matching. Seventeen healthy right handed participants underwent fMRI while observing pictures of familiar tool objects followed by pictures of hand postures. The task was to decide whether the hand posture matched the functional use of the previously shown object. Conditions were manipulated for level of difficulty. Compared to a picture matching control task, the tool object – hand posture matching conditions conjointly showed increased modulation in several left hemispheric regions of the superior and inferior parietal lobules (including aIPS), the middle occipital gyrus, and the inferior temporal gyrus. Comparison of hard versus easy conditions selectively modulated the left inferior frontal gyrus with peak activity located in its opercular part (Brodmann area (BA) 44). We suggest that in the human brain, vPMC/BA44 is involved in the matching of hand posture configurations in accordance with visual and functional demands.     

The representation of erroneously perceived stimuli in the primary visual cortex

In order to attain a correct interpretation of an ambiguous visual stimulus, the brain may have to elaborate on the sensory evidence. Are the neurons that carry the sensory evidence also involved in generating an interpretation? To address this question, we studied the activity of neurons in the primary visual cortex of macaque monkeys involved in a task in which they have to trace a curve mentally, without moving their eyes. On a percentage of trials, the monkeys made errors and traced the wrong curve. Here, we show that these errors are predicted by activity in area V1. Thus, neurons in the primary visual cortex do not only represent sensory events, but also the way in which they are interpreted by the monkey.     

fMRI reveals how pain modulates visual object processing in the ventral visual stream

It is well known that pain attracts attention and interferes with cognition. Given that the mechanisms behind this phenomenon are largely unknown, we used functional magnetic resonance imaging and presented visual objects with or without concomitant pain stimuli. To test for the specificity of pain, we compared this modulatory effect with a previously established modulatory effect of working memory on visual object processing. Our data showed a comparable behavioral effect of both types of modulation and identified the lateral occipital complex (LOC) as the site of modulation in the ventral visual stream, for both pain and working memory. However, the sources of these modulatory effects differed for the two processes. Whereas the source of modulation for working memory could be attributed to the parietal cortex, the modulatory effect of pain was observed in the rostral anterior cingulate cortex (rACC), an area ideally suited to link pain perception and attentional control.     

Visual saliency and spike timing in the ventral visual pathway.

Visual saliency is a fundamental yet hard to define property of objects or locations in the visual world. In a context where objects and their representations compete to dominate our perception, saliency can be thought of as the "juice" that makes objects win the race. It is often assumed that saliency is extracted and represented in an explicit saliency map, which serves to determine the location of spatial attention at any given time. It is then by drawing attention to a salient object that it can be recognized or categorized. I argue against this classical view that visual "bottom-up" saliency automatically recruits the attentional system prior to object recognition. A number of visual processing tasks are clearly performed too fast for such a costly strategy to be employed. Rather, visual attention could simply act by biasing a saliency-based object recognition system. Under natural conditions of stimulation, saliency can be represented implicitly throughout the ventral visual pathway, independent of any explicit saliency map. At any given level, the most activated cells of the neural population simply represent the most salient locations. The notion of saliency itself grows increasingly complex throughout the system, mostly based on luminance contrast until information reaches visual cortex, gradually incorporating information about features such as orientation or color in primary visual cortex and early extrastriate areas, and finally the identity and behavioral relevance of objects in temporal cortex and beyond. Under these conditions the object that dominates perception, i.e. the object yielding the strongest (or the first) selective neural response, is by definition the one whose features are most "salient"--without the need for any external saliency map. In addition, I suggest that such an implicit representation of saliency can be best encoded in the relative times of the first spikes fired in a given neuronal population. In accordance with our subjective experience that saliency and attention do not modify the appearance of objects, the feed-forward propagation of this first spike wave could serve to trigger saliency-based object recognition outside the realm of awareness, while conscious perceptions could be mediated by the remaining discharges of longer neuronal spike trains.     

Fucntional specialization and binocular interaction in the visual areas of rhesus monkey prestriate cortex.

If is is believed that neural mechanisms mediating stereoscopic vision may be localized in specific areas of the visual cortex, then it becomes necessary to be able to define these areas adequately. This is no easy matter in the rhesus monkey, an animal close to man, where the cytoarchitecturally uniform prestriate cortex is folded into deep sulci with secondary gyri. One way around this awkward problem is to use the callosal connections of the prestriate cortex as the anatomical landmarks. Callosal connections are restricted to regions at which the vertical meridian is represented. Since the visual fields, including the vertical meridian, are separately represented in each area, each has its own callosal connections. These are of great help in defining some of the boundaries of these areas, since the boundaries often coincide with the representation of the vertical meridian. With the visual areas thus defined anatomically, it becomes relatively easy to assign recordings to particular areas. Studies of binocular interactions in these areas reveal that most cells in all prestriate areas are binocularly driven. Hence, theoretically, all of the prestriate areas are candidates for stereoscopic mechanisms. The degree of binocular interaction varies from cell to cell. At the two extremes are cells which either respond to monocular stimulation only and are inhibited by binocular stimulation or ones which respond to binocular stimulation only. Changing, as opposed to fixed, disparity is signalled by two types of cells. In one category are cells activated in opposite directions for the two eyes. Such cells are always binocularly driven. In the other category are cells, some of which are monocularly activated, that are capable of responding to changing image size. In the monkey, both these categories of cells have so far been found in the motion area of the superior temporal sulcus only.