Distributed neural plasticity for shape learning in the human visual cortex

Expertise in recognizing objects in cluttered scenes is a critical skill for our interactions in complex environments and is thought to develop with learning. However, the neural implementation of object learning across stages of visual analysis in the human brain remains largely unknown. Using combined psychophysics and functional magnetic resonance imaging (fMRI), we show a link between shape-specific learning in cluttered scenes and distributed neuronal plasticity in the human visual cortex. We report stronger fMRI responses for trained than untrained shapes across early and higher visual areas when observers learned to detect low-salience shapes in noisy backgrounds. However, training with high-salience pop-out targets resulted in lower fMRI responses for trained than untrained shapes in higher occipitotemporal areas. These findings suggest that learning of camouflaged shapes is mediated by increasing neural sensitivity across visual areas to bolster target segmentation and feature integration. In contrast, learning of prominent pop-out shapes is mediated by associations at higher occipitotemporal areas that support sparser coding of the critical features for target recognition. We propose that the human brain learns novel objects in complex scenes by reorganizing shape processing across visual areas, while taking advantage of natural image correlations that determine the distinctiveness of target shapes.     

Northern bats, Eptesicus nilssonii, use vision but not flutter-detection when searching for prey in clutter

We investigated the detection cues used by the aerial-hawking bat Eptesicus nilssonii foraging in a cluttered environment. The bats can detect and attack rapidly moving targets within the clutter, i.e. below grass panicles, by using prey motion as a cue. Stationary objects are attacked only above the grass, but still within the clutter overlap zone. To test if the bats were guided by flutter from moth wings or by vision when searching for stationary targets, they were presented with male ghost swifts mounted on top of steel wires. There was no difference in attack frequency on live, fluttering moths compared to dead and spread ones. However, when comparing white and dark moths, we found a significantly higher attack frequency on white ones. As the attacks always were guided by echolocation calls, we hypothesize that northern bats, at least in the initial search phase, use visual cues as a complement to detect stationary ghost swifts.     

Attention facilitates multiple stimulus features in parallel in human visual cortex

Successfully locating a dangerous or desirable object within a cluttered visual scene is a commonplace yet highly adaptive skill. In the laboratory, this ability is modeled by visual search experiments in which subjects try to find a target item surrounded by an array of distracting stimuli. Under certain conditions, targets that are distinguishable from distractors by virtue of having a particular combination of shared sensory features (e.g., a particular color and orientation) can be found rapidly regardless of the number of distractors. To explain this highly efficient localization of feature-conjunction targets, "guided search" theories propose that attention is directed in parallel to the individual features that define the target, which then stands out from the distractors because of additive facilitation of its feature signals. Here we recorded frequency-tagged potentials evoked in human visual cortex and found that color and orientation features of target stimuli are indeed facilitated by attention in a parallel and additive manner. This additive feature-enhancement mechanism, reported here for the first time, not only enables rapid guided search but also plays a broader role in directing and sustaining attention to multi-feature objects and keeping them perceptually distinct from background clutter.     

Different processing phases for features, figures, and selective attention in the primary visual cortex

Our visual system imposes structure onto images that usually contain a diversity of surfaces, contours, and colors. Psychological theories propose that there are multiple steps in this process that occur in hierarchically organized regions of the cortex: early visual areas register basic features, higher areas bind them into objects, and yet higher areas select the objects that are relevant for behavior. Here we test these theories by recording from the primary visual cortex (area V1) of monkeys. We demonstrate that the V1 neurons first register the features (at a latency of 48 ms), then segregate figures from the background (after 57 ms), and finally select relevant figures over irrelevant ones (after 137 ms). We conclude that the psychological processing stages map onto distinct time episodes that unfold in the visual cortex after the presentation of a new stimulus, so that area V1 may contribute to all these processing steps.     

Coding the presence of visual objects in a recurrent neural network of visual cortex

Before we can recognize a visual object, our visual system has to segregate it from its background. This requires a fast mechanism for establishing the presence and location of objects independently of their identity. Recently, border-ownership neurons were recorded in monkey visual cortex which might be involved in this task [Zhou, H., Friedmann, H., von der Heydt, R., 2000. Coding of border ownership in monkey visual cortex. J. Neurosci. 20 (17), 6594-6611]. In order to explain the basic mechanisms required for fast coding of object presence, we have developed a neural network model of visual cortex consisting of three stages. Feed-forward and lateral connections support coding of Gestalt properties, including similarity, good continuation, and convexity. Neurons of the highest area respond to the presence of an object and encode its position, invariant of its form. Feedback connections to the lowest area facilitate orientation detectors activated by contours belonging to potential objects, and thus generate the experimentally observed border-ownership property. This feedback control acts fast and significantly improves the figure-ground segregation required for the consecutive task of object recognition.     

Computational modeling and exploration of contour integration for visual saliency

We propose a computational model of contour integration for visual saliency. The model uses biologically plausible devices to simulate how the representations of elements aligned collinearly along a contour in an image are enhanced. Our model adds such devices as a dopamine-like fast plasticity, local GABAergic inhibition and multi-scale processing of images. The fast plasticity addresses the problem of how neurons in visual cortex seem to be able to influence neurons they are not directly connected to, for instance, as observed in contour closure effect. Local GABAergic inhibition is used to control gain in the system without using global mechanisms which may be non-plausible given the limited reach of axonal arbors in visual cortex. The model is then used to explore not only its validity in real and artificial images, but to discover some of the mechanisms involved in processing of complex visual features such as junctions and end-stops as well as contours. We present evidence for the validity of our model in several phases, starting with local enhancement of only a few collinear elements. We then test our model on more complex contour integration images with a large number of Gabor elements. Sections of the model are also extracted and used to discover how the model might relate contour integration neurons to neurons that process end-stops and junctions. Finally, we present results from real world images. Results from the model suggest that it is a good current approximation of contour integration in human vision. As well, it suggests that contour integration mechanisms may be strongly related to mechanisms for detecting end-stops and junction points. Additionally, a contour integration mechanism may be involved in finding features for objects such as faces. This suggests that visual cortex may be more information efficient and that neural regions may have multiple roles.     

A sparse object coding scheme in area V4

Sparse coding has long been recognized as a primary goal of image transformation in the visual system. Sparse coding in early visual cortex is achieved by abstracting local oriented spatial frequencies and by excitatory/inhibitory surround modulation. Object responses are thought to be sparse at subsequent processing stages, but neural mechanisms for higher-level sparsification are not known. Here, convergent results from macaque area V4 neural recording and simulated V4 populations trained on natural object contours suggest that sparse coding is achieved in midlevel visual cortex by emphasizing representation of acute convex and concave curvature. We studied 165 V4 neurons with a random, adaptive stimulus strategy to minimize bias and explore an unlimited range of contour shapes. V4 responses were strongly weighted toward contours containing acute convex or concave curvature. In contrast, the tuning distribution in nonsparse simulated V4 populations was strongly weighted toward low curvature. But as sparseness constraints increased, the simulated tuning distribution shifted progressively toward more acute convex and concave curvature, matching the neural recording results. These findings indicate a sparse object coding scheme in midlevel visual cortex based on uncommon but diagnostic regions of acute contour curvature.     

Optimality of human contour integration

For processing and segmenting visual scenes, the brain is required to combine a multitude of features and sensory channels. It is neither known if these complex tasks involve optimal integration of information, nor according to which objectives computations might be performed. Here, we investigate if optimal inference can explain contour integration in human subjects. We performed experiments where observers detected contours of curvilinearly aligned edge configurations embedded into randomly oriented distractors. The key feature of our framework is to use a generative process for creating the contours, for which it is possible to derive a class of ideal detection models. This allowed us to compare human detection for contours with different statistical properties to the corresponding ideal detection models for the same stimuli. We then subjected the detection models to realistic constraints and required them to reproduce human decisions for every stimulus as well as possible. By independently varying the four model parameters, we identify a single detection model which quantitatively captures all correlations of human decision behaviour for more than 2000 stimuli from 42 contour ensembles with greatly varying statistical properties. This model reveals specific interactions between edges closely matching independent findings from physiology and psychophysics. These interactions imply a statistics of contours for which edge stimuli are indeed optimally integrated by the visual system, with the objective of inferring the presence of contours in cluttered scenes. The recurrent algorithm of our model makes testable predictions about the temporal dynamics of neuronal populations engaged in contour integration, and it suggests a strong directionality of the underlying functional anatomy.     

Invariant visual object recognition: a model, with lighting invariance.

How are invariant representations of objects formed in the visual cortex? We describe a neurophysiological and computational approach which focusses on a feature hierarchy model in which invariant representations can be built by self-organizing learning based on the statistics of the visual input. The model can use temporal continuity in an associative synaptic learning rule with a short term memory trace, and/or it can use spatial continuity in Continuous Transformation learning. The model of visual processing in the ventral cortical stream can build representations of objects that are invariant with respect to translation, view, size, and in this paper we show also lighting. The model has been extended to provide an account of invariant representations in the dorsal visual system of the global motion produced by objects such as looming, rotation, and object-based movement. The model has been extended to incorporate top-down feedback connections to model the control of attention by biased competition in for example spatial and object search tasks. The model has also been extended to account for how the visual system can select single objects in complex visual scenes, and how multiple objects can be represented in a scene.     

Illusory contours over pathological retinal scotomas

Our visual percepts are not fully determined by physical stimulus inputs. Thus, in visual illusions such as the Kanizsa figure, inducers presented at the corners allow one to perceive the bounding contours of the figure in the absence of luminance-defined borders. We examined the discrimination of the curvature of these illusory contours that pass across retinal scotomas caused by macular degeneration. In contrast with previous studies with normal-sighted subjects that showed no perception of these illusory contours in the region of physiological scotomas at the optic nerve head, we demonstrated perfect discrimination of the curvature of the illusory contours over the pathological retinal scotoma. The illusion occurred despite the large scar around the macular lesion, strongly reducing discrimination of whether the inducer openings were acute or obtuse and suggesting that the coarse information in the inducers (low spatial frequency) sufficed. The result that subjective contours can pass through the pathological retinal scotoma suggests that the visual cortex, despite the loss of bottom-up input, can use low-spatial frequency information from the inducers to form a neural representation of new complex geometrical shapes inside the scotoma.     

A visual model for object detection based on active contours and level-set method

A visual model for object detection is proposed. In order to make the detection ability comparable with existing technical methods for object detection, an evolution equation of neurons in the model is derived from the computational principle of active contours. The hierarchical structure of the model emerges naturally from the evolution equation. One drawback involved with initial values of active contours is alleviated by introducing and formulating convexity, which is a visual property. Numerical experiments show that the proposed model detects objects with complex topologies and that it is tolerant of noise. A visual attention model is introduced into the proposed model. Other simulations show that the visual properties of the model are consistent with the results of psychological experiments that disclose the relation between figure–ground reversal and visual attention. We also demonstrate that the model tends to perceive smaller regions as figures, which is a characteristic observed in human visual perception.This work was partially supported by Grants-in-Aid for Scientific Research (#14780254) from Japan Society of Promotion of Science.     

Motion detection, noise reduction, texture suppression, and contour enhancement by spatiotemporal Gabor filters with surround inhibition

We study the orientation and speed tuning properties of spatiotemporal three-dimensional (3D) Gabor and motion energy filters as models of time-dependent receptive fields of simple and complex cells in the primary visual cortex (V1). We augment the motion energy operator with surround suppression to model the inhibitory effect of stimuli outside the classical receptive field. We show that spatiotemporal integration and surround suppression lead to substantial noise reduction. We propose an effective and straightforward motion detection computation that uses the population code of a set of motion energy filters tuned to different velocities. We also show that surround inhibition leads to suppression of texture and thus improves the visibility of object contours and facilitates figure/ground segregation and the detection and recognition of objects.     

Visual Cortex: How Are Faces and Objects Represented?

The way in which information about complex objects and faces is represented in visual cortex is controversial. One model posits that information is processed in modules, highly specialized for different categories of objects; an opposing model appeals to a distributed representation across a large network of visual areas. A recent paper uses a novel imaging technique to address this controversy.     

Deriving behavioural receptive fields for visually completed contours

The visual system is constantly faced with the problem of identifying partially occluded objects from incomplete images cast on the retinae. Phenomenologically, the visual system seems to fill in missing information by interpolating illusory and occluded contours at points of occlusion, so that we perceive complete objects. Previous behavioural [1] [2] [3] [4] [5] [6] [7] and physiological [8] [9] [10] [11] [12] studies suggest that the visual system treats illusory and occluded contours like luminance-defined contours in many respects. None of these studies has, however, directly shown that illusory and occluded contours are actually used to perform perceptual tasks. Here, we use a response-classification technique [13] [14] [15] [16] [17] [18] [19] [20] to answer this question directly. This technique provides pictorial representations - 'classification images' - that show which parts of a stimulus observers use to make perceptual decisions, effectively deriving behavioural receptive fields. Here we show that illusory and occluded contours appear in observers' classification images, providing the first direct evidence that observers use perceptually interpolated contours to recognize objects. These results offer a compelling demonstration of how visual processing acts on completed representations, and illustrate a powerful new technique for constraining models of visual completion.     

Inhomogeneous retino-cortical mapping is supported and stabilized with correlation-learning during self-motion

In primates, the area of primary visual cortex representing a fixed area of visual space decreases with increasing eccentricity. We identify visual situations to which this inhomogeneous retino-cortical mapping is well adapted and study their relevance during natural vision and development. We assume that cortical activations caused by stationary objects during self-motion along the direction of gaze travel on average with constant speed across the cortical surface, independent of retinal eccentricity. This is the case if the distribution of objects corresponds to an ellipsoid with the observer in its center. We apply the resulting flow field to train a simple network of pulse coding neurons with Hebbian learning and demonstrate that the density of learned receptive field centers is in close agreement with primate retino-cortical magnification. In addition, the model reproduces the increase of receptive field size and the decrease of its peak sensitivity with increasing eccentricity. Our results suggest that self-motion may have played an important role in the evolution of the visual system and that cortical magnification can be refined and stabilized by Hebbian learning mechanisms in ontogenesis under natural viewing conditions.     

Category selectivity in the ventral visual pathway confers robustness to clutter and diverted attention

Are objects coded by a small number of neurons or cortical regions that respond preferentially to the object in question, or by more distributed patterns of responses, including neurons or regions that respond only weakly? Distributed codes can represent a larger number of alternative items than sparse codes but produce ambiguities when multiple items are represented simultaneously (the "superposition" problem). Recent studies found category information in the distributed pattern of response across the ventral visual pathway, including in regions that do not "prefer" the object in question. However, these studies measured neural responses to isolated objects, a situation atypical of real-world vision, where multiple objects are usually present simultaneously ("clutter"). We report that information in the spatial pattern of fMRI response about standard object categories is severely disrupted by clutter and eliminated when attention is diverted. However, information about preferred categories in category-specific regions is undiminished by clutter and partly preserved under diverted attention. These findings indicate that in natural conditions, the pattern of fMRI response provides robust category information only for objects coded in selective cortical regions and highlight the vulnerability of distributed representations to clutter and the advantages of sparse cortical codes in mitigating clutter costs.     

Globally consistent depth sorting of overlapping 2D surfaces in a model using local recurrent interactions

The human visual system utilizes depth information as a major cue to group together visual items constituting an object and to segregate them from items belonging to other objects in the visual scene. Depth information can be inferred from a variety of different visual cues, such as disparity, occlusions and perspective. Many of these cues provide only local and relative information about the depth of objects. For example, at occlusions, T-junctions indicate the local relative depth precedence of surface patches. However, in order to obtain a globally consistent interpretation of the depth relations between the surfaces and objects in a visual scene, a mechanism is necessary that globally propagates such local and relative information. We present a computational framework in which depth information derived from T-junctions is propagated along surface contours using local recurrent interactions between neighboring neurons. We demonstrate that within this framework a globally consistent depth sorting of overlapping surfaces can be obtained on the basis of local interactions. Unlike previous approaches in which locally restricted cell interactions could merely distinguish between two depths (figure and ground), our model can also represent several intermediate depth positions. Our approach is an extension of a previous model of recurrent V1–V2 interaction for contour processing and illusory contour formation. Based on the contour representation created by this model, a recursive scheme of local interactions subsequently achieves a globally consistent depth sorting of several overlapping surfaces. Within this framework, the induction of illusory contours by the model of recurrent V1–V2 interaction gives rise to the figure-ground segmentation of illusory figures such as a Kanizsa square.     

Learning and neural plasticity in visual object recognition

The capability of the adult primate visual system for rapid and accurate recognition of targets in cluttered, natural scenes far surpasses the abilities of state-of-the-art artificial vision systems. Understanding this capability remains a fundamental challenge in visual neuroscience. Recent experimental evidence suggests that adaptive coding strategies facilitated by underlying neural plasticity enable the adult brain to learn from visual experience and shape its ability to integrate and recognize coherent visual objects.     

Patchy organization and asymmetric distribution of the neural correlates of face processing in monkey inferotemporal cortex

BACKGROUND: It is believed that a face-specific system exists within the primate ventral visual pathway that is separate from a domain-general nonface object coding system. In addition, it is believed that hemispheric asymmetry, which was long held to be a distinct feature of the human brain, can be found in the brains of other primates as well. We show here for the first time by way of a functional imaging technique that face- and object-selective neurons form spatially distinct clusters at the cellular level in monkey inferotemporal cortex. We have used a novel functional mapping technique that simultaneously generates two separate activity profiles by exploiting the differential time course of zif268 mRNA and protein expression. RESULTS: We show that neurons activated by face stimulation can be visualized at cellular resolution and distinguished from those activated by nonface complex objects. Our dual-activity maps of face and object selectivity show that face-selective patches of various sizes (mean, 22.30 mm2; std, 32.76 mm2) exist throughout the IT cortex in the context of a large expanse of cortical territory that is responsive to visual objects. CONCLUSIONS: These results add to recent findings that face-selective patches of various sizes exist throughout area IT and provide the first direct anatomical evidence at cellular resolution for a hemispheric asymmetry in favor of the right hemisphere. Together, our results support the notion that human and monkey brains share a similarity in both anatomical organization and distribution of function with respect to high-level visual processing.