A sparse object coding scheme in area V4

Sparse coding has long been recognized as a primary goal of image transformation in the visual system. Sparse coding in early visual cortex is achieved by abstracting local oriented spatial frequencies and by excitatory/inhibitory surround modulation. Object responses are thought to be sparse at subsequent processing stages, but neural mechanisms for higher-level sparsification are not known. Here, convergent results from macaque area V4 neural recording and simulated V4 populations trained on natural object contours suggest that sparse coding is achieved in midlevel visual cortex by emphasizing representation of acute convex and concave curvature. We studied 165 V4 neurons with a random, adaptive stimulus strategy to minimize bias and explore an unlimited range of contour shapes. V4 responses were strongly weighted toward contours containing acute convex or concave curvature. In contrast, the tuning distribution in nonsparse simulated V4 populations was strongly weighted toward low curvature. But as sparseness constraints increased, the simulated tuning distribution shifted progressively toward more acute convex and concave curvature, matching the neural recording results. These findings indicate a sparse object coding scheme in midlevel visual cortex based on uncommon but diagnostic regions of acute contour curvature.     

Perceptual organization of local elements into global shapes in the human visual cortex

The question of how local image features on the retina are integrated into perceived global shapes is central to our understanding of human visual perception. Psychophysical investigations have suggested that the emergence of a coherent visual percept, or a "good-Gestalt", is mediated by the perceptual organization of local features based on their similarity. However, the neural mechanisms that mediate unified shape perception in the human brain remain largely unknown. Using human fMRI, we demonstrate that not only higher occipitotemporal but also early retinotopic areas are involved in the perceptual organization and detection of global shapes. Specifically, these areas showed stronger fMRI responses to global contours consisting of collinear elements than to patterns of randomly oriented local elements. More importantly, decreased detection performance and fMRI activations were observed when misalignment of the contour elements disturbed the perceptual coherence of the contours. However, grouping of the misaligned contour elements by disparity resulted in increased performance and fMRI activations, suggesting that similar neural mechanisms may underlie grouping of local elements to global shapes by different visual features (orientation or disparity). Thus, these findings provide novel evidence for the role of both early feature integration processes and higher stages of visual analysis in coherent visual perception.     

Visual shape representation with geometrically characterized contour partitions

This paper proposes a biologically plausible matching method to recognize general shapes based on contour curvature information. The human visual system recognizes general shapes flexibly in real-world scenes through the ventral pathway. The pathway is typically modeled using artificial neural networks. These network models, however, do not construct a shape representation that satisfies the following required constraints: (1) The original shape should be represented by a group of partitioned contours in order to retrieve the whole shape (global information) from the partial contours (local information). (2) Coarse and fine structures of the original shapes should be individually represented in order for the visual system to respond to shapes as quickly as possible based on the least number of their features, and to discriminate between shapes based on detailed information. (3) The shape recognition realized with an artificial visual system should be invariant to geometric transformation such as expansion, rotation, or shear. In this paper, we propose a visual shape representation with geometrically characterized contour partitions described on multiple spatial scales.     

An fMRI study of the selective activation of human extrastriate form vision areas by radial and concentric gratings

The ventral form vision pathway of the primate brain comprises a sequence of areas that include V1, V2, V4 and the inferior temporal cortex (IT) [1]. Although contour extraction in the V1 area and responses to complex images, such as faces, in the IT have been studied extensively, much less is known about shape extraction at intermediate cortical levels such as V4. Here, we used functional magnetic resonance imaging (fMRI) to demonstrate that the human V4 is more strongly activated by concentric and radial patterns than by conventional sinusoidal gratings. This is consistent with global pooling of local V1 orientations to extract concentric and radial shape information in V4. Furthermore, concentric patterns were found to be effective in activating the fusiform face area. These findings support recent psychophysical [2,3] and physiological [4,5] data indicating that analysis of concentric and radial structure represents an important aspect of processing at intermediate levels of form vision.     

Integration of local features into global shapes: monkey and human FMRI studies

The integration of local image features into global shapes was investigated in monkeys and humans using fMRI. An adaptation paradigm was used, in which stimulus selectivity was deduced by changes in the course of adaptation of a pattern of randomly oriented elements. Accordingly, we observed stronger activity when orientation changes in the adapting stimulus resulted in a collinear contour than a different random pattern. This selectivity to collinear contours was observed not only in higher visual areas that are implicated in shape processing, but also in early visual areas where selectivity depended on the receptive field size. These findings suggest that unified shape perception in both monkeys and humans involves multiple visual areas that may integrate local elements to global shapes at different spatial scales.     

Distributed neural plasticity for shape learning in the human visual cortex

Expertise in recognizing objects in cluttered scenes is a critical skill for our interactions in complex environments and is thought to develop with learning. However, the neural implementation of object learning across stages of visual analysis in the human brain remains largely unknown. Using combined psychophysics and functional magnetic resonance imaging (fMRI), we show a link between shape-specific learning in cluttered scenes and distributed neuronal plasticity in the human visual cortex. We report stronger fMRI responses for trained than untrained shapes across early and higher visual areas when observers learned to detect low-salience shapes in noisy backgrounds. However, training with high-salience pop-out targets resulted in lower fMRI responses for trained than untrained shapes in higher occipitotemporal areas. These findings suggest that learning of camouflaged shapes is mediated by increasing neural sensitivity across visual areas to bolster target segmentation and feature integration. In contrast, learning of prominent pop-out shapes is mediated by associations at higher occipitotemporal areas that support sparser coding of the critical features for target recognition. We propose that the human brain learns novel objects in complex scenes by reorganizing shape processing across visual areas, while taking advantage of natural image correlations that determine the distinctiveness of target shapes.     

The structures of letters and symbols throughout human history are selected to match those found in objects in natural scenes

Are there empirical regularities in the shapes of letters and other human visual signs, and if so, what are the selection pressures underlying these regularities? To examine this, we determined a wide variety of topologically distinct contour configurations and examined the relative frequency of these configuration types across writing systems, Chinese writing, and nonlinguistic symbols. Our first result is that these three classes of human visual sign possess a similar signature in their configuration distribution, suggesting that there are underlying principles governing the shapes of human visual signs. Second, we provide evidence that the shapes of visual signs are selected to be easily seen at the expense of the motor system. Finally, we provide evidence to support an ecological hypothesis that visual signs have been culturally selected to match the kinds of conglomeration of contours found in natural scenes because that is what we have evolved to be good at visually processing.     

Hierarchical processing of tactile shape in the human brain

It is not known exactly which cortical areas compute somatosensory representations of shape. This was investigated using positron emission tomography and cytoarchitectonic mapping. Volunteers discriminated shapes by passive or active touch, brush velocity, edge length, curvature, and roughness. Discrimination of shape by active touch, as opposed to passive touch, activated the right anterior lobe of cerebellum only. Areas 3b and 1 were activated by all stimuli. Area 2 was activated with preference for surface curvature changes and shape stimuli. The anterior part of the supramarginal gyrus (ASM) and the cortex lining the intraparietal sulcus (IPA) were activated by active and passive shape discrimination, but not by other mechanical stimuli. We suggest, based on these findings, that somatosensory representations of shape are computed by areas 3b, 1, 2, IPA, and ASM in this hierarchical fashion.     

Contour saliency in primary visual cortex

Contour integration is an important intermediate stage of object recognition, in which line segments belonging to an object boundary are perceptually linked and segmented from complex backgrounds. Contextual influences observed in primary visual cortex (V1) suggest the involvement of V1 in contour integration. Here, we provide direct evidence that, in monkeys performing a contour detection task, there was a close correlation between the responses of V1 neurons and the perceptual saliency of contours. Receiver operating characteristic analysis showed that single neuronal responses encode the presence or absence of a contour as reliably as the animal's behavioral responses. We also show that the same visual contours elicited significantly weaker neuronal responses when they were not detected in the detection task, or when they were unattended. Our results demonstrate that contextual interactions in V1 play a pivotal role in contour integration and saliency.     

Computations in the early visual cortex.

This paper reviews some of the recent neurophysiological studies that explore the variety of visual computations in the early visual cortex in relation to geometric inference, i.e. the inference of contours, surfaces and shapes. It attempts to draw connections between ideas from computational vision and findings from awake primate electrophysiology. In the classical feed-forward, modular view of visual processing, the early visual areas (LGN, V1 and V2) are modules that serve to extract local features, while higher extrastriate areas are responsible for shape inference and invariant object recognition. However, recent findings in primate early visual systems reveal that the computations in the early visual cortex are rather complex and dynamic, as well as interactive and plastic, subject to influence from global context, higher order perceptual inference, task requirement and behavioral experience. The evidence argues that the early visual cortex does not merely participate in the first stage of visual processing, but is involved in many levels of visual computation.     

See me, hear me, touch me: multisensory integration in lateral occipital-temporal cortex

Our understanding of multisensory integration has advanced because of recent functional neuroimaging studies of three areas in human lateral occipito-temporal cortex: superior temporal sulcus, area LO and area MT (V5). Superior temporal sulcus is activated strongly in response to meaningful auditory and visual stimuli, but responses to tactile stimuli have not been well studied. Area LO shows strong activation in response to both visual and tactile shape information, but not to auditory representations of objects. Area MT, an important region for processing visual motion, also shows weak activation in response to tactile motion, and a signal that drops below resting baseline in response to auditory motion. Within superior temporal sulcus, a patchy organization of regions is activated in response to auditory, visual and multisensory stimuli. This organization appears similar to that observed in polysensory areas in macaque superior temporal sulcus, suggesting that it is an anatomical substrate for multisensory integration. A patchy organization might also be a neural mechanism for integrating disparate representations within individual sensory modalities, such as representations of visual form and visual motion.     

Contrast sensitivity functions to stimuli defined in Cartesian, polar and hyperbolic coordinates

Recent electrophysiological studies indicate that cells in the LGN, V1, V2, and V4 areas in monkeys are specifically sensitive to Cartesian, polar and hyperbolic stimuli. We have characterized the contrast sensitivity functions (CSF) to stimuli defined in these coordinates with the two-alternatives forced-choice paradigm. CSFs to Cartesian, concentric, and hyperbolic stimuli have had similar shapes, with peak sensitivity at approximately 3 c/deg. However, the Cartesian CSF peak sensitivity has been at least 0.1 log units higher than that to stimuli in any other coordinate system. The concentric-Bessel CSF has a low-pass shape, peaking at 1.5 c/deg or below. The radial CSF has a bell shape with maximum sensitivity at 8 c/360 degrees. Only the concentric-Bessel CSF could be explained in terms of the components of maximum amplitude of the Fourier transform. Neural models, which in previous studies predicted the responses to Cartesian and polar Glass patterns, failed to account for the full CSFs data.     

Dependence of V2 illusory contour response on V1 cell properties and topographic organization

An illusory contour is an image that is perceived as a contour in the absence of typical contour characteristics, such as a change in luminance or chromaticity across the stimulus. In cats and primates, cells that respond to illusory contours are sparse in cortical area V1, but are found in greater numbers in cortical area V2. We propose a model capable of illusory contour detection that is based on a realistic topographic organization of V1 cells, which reproduces the responses of individual cell types measured experimentally. The model allows us to explain several experimentally observed properties of V2 cells including variability in orientation tuning and inducer spacing preference. As a practical application, the model can be used to estimate the relationship between the severity of a cortical injury in the primary visual cortex and the deterioration of V2 cell responses to real and illusory contours.     

Judgments of shape similarity in the barbary dove (Streptopelia risoria)

Sixteen experimentally naive adult Barbary doves (S. risoria) learned a successive discrimination between a simple and a complex shape and were then tested for transfer of training in a generalization test with seven unfamiliar shapes. It was found that the degree of equivalence between shapes was correlated with two physical measures of shape based on contour length and on the number of sides of the shape. These results are in general agreement with other comparative data including those for humans. Results of the generalization test showed an ‘asymmetrical’ response in that judgments of shape similarity were not elicited with equal frequency in all subjects. This was interpreted as evidence for an internal standard in that subjects compared test stimuli to the positive training shape.     

Local non-linear interactions in the visual cortex may reflect global decorrelation

The classical receptive field in the primary visual cortex have been successfully explained by sparse activation of relatively independent units, whose tuning properties reflect the statistical dependencies in the natural environment. Robust surround modulation, emerging from stimulation beyond the classical receptive field, has been associated with increase of lifetime sparseness in the V1, but the system-wide modulation of response strength have currently no theoretical explanation. We measured fMRI responses from human visual cortex and quantified the contextual modulation with a decorrelation coefficient (d), derived from a subtractive normalization model. All active cortical areas demonstrated local non-linear summation of responses, which were in line with hypothesis of global decorrelation of voxels responses. In addition, we found sensitivity to surrounding stimulus structure across the ventral stream, and large-scale sensitivity to the number of simultaneous objects. Response sparseness across voxel population increased consistently with larger stimuli. These data suggest that contextual modulation for a stimulus event reflect optimization of the code and perhaps increase in energy efficiency throughout the ventral stream hierarchy. Our model provides a novel prediction that average suppression of response amplitude for simultaneous stimuli across the cortical network is a monotonic function of similarity of response strengths in the network when the stimuli are presented alone.     

Multivariate patterns in object-selective cortex dissociate perceptual and physical shape similarity

Prior research has identified the lateral occipital complex (LOC) as a critical cortical region for the representation of object shape in humans. However, little is known about the nature of the representations contained in the LOC and their relationship to the perceptual experience of shape. We used human functional MRI to measure the physical, behavioral, and neural similarity between pairs of novel shapes to ask whether the representations of shape contained in subregions of the LOC more closely reflect the physical stimuli themselves, or the perceptual experience of those stimuli. Perceptual similarity measures for each pair of shapes were obtained from a psychophysical same-different task; physical similarity measures were based on stimulus parameters; and neural similarity measures were obtained from multivoxel pattern analysis methods applied to anterior LOC (pFs) and posterior LOC (LO). We found that the pattern of pairwise shape similarities in LO most closely matched physical shape similarities, whereas shape similarities in pFs most closely matched perceptual shape similarities. Further, shape representations were similar across participants in LO but highly variable across participants in pFs. Together, these findings indicate that activation patterns in subregions of object-selective cortex encode objects according to a hierarchy, with stimulus-based representations in posterior regions and subjective and observer-specific representations in anterior regions.     

Dynamic shape synthesis in posterior inferotemporal cortex

How does the brain synthesize low-level neural signals for simple shape parts into coherent representations of complete objects? Here, we present evidence for a dynamic process of object part integration in macaque posterior inferotemporal cortex (IT). Immediately after stimulus onset, neural responses carried information about individual object parts (simple contour fragments) only. Subsequently, information about specific multipart configurations emerged, building gradually over the course of approximately 60 ms, producing a sparser and more explicit representation of object shape. We show that this gradual transformation can be explained by a recurrent network process that effectively compares parts signals across neurons to generate inferences about multipart shape configurations.     

Erythrocyte shape simulation by numerical optimization

In a recent paper we examined the morphology of erythrocytes in terms of the mean mean curvature (MMC) of their cell membranes. A computer simulation of these shapes based on the different geometries showed that the MMC increased from the sphero-stomatocyte to the spheroechinocyte via the discocyte. In this work we extend this analysis by using a numerical optimization method based on importance sampling and the principle of adiabatic cooling. The erythrocyte membrane is treated as a single closed fluid lamina exhibiting viscoelastic characteristics. The energy function of the lamina includes the following terms: (i) Curvature-elastic energy terms which depend on both local and global curvature. (ii) A term describing the compression elasticity of the lamina. (iii) A term which depends on the volume of the cell and which is related to the osmotic pressure across the membrane. In the simulation the cell is assumed to have axial symmetry and it can therefore be described by a finite set of conic sections. So far we have been able to obtain an energy minimum corresponding to a discocyte shape using a sphere as the initial configuration. Our results therefore imply that the well-known sequence of erythrocyte shapes could solely be governed by the above mentioned properties of an ideal fluid forming a closed singly connected lamina.