Learning to link visual contours

In complex visual scenes, linking related contour elements is important for object recognition. This process, thought to be stimulus driven and hard wired, has substrates in primary visual cortex (V1). Here, however, we find contour integration in V1 to depend strongly on perceptual learning and top-down influences that are specific to contour detection. In naive monkeys, the information about contours embedded in complex backgrounds is absent in V1 neuronal responses and is independent of the locus of spatial attention. Training animals to find embedded contours induces strong contour-related responses specific to the trained retinotopic region. These responses are most robust when animals perform the contour detection task but disappear under anesthesia. Our findings suggest that top-down influences dynamically adapt neural circuits according to specific perceptual tasks. This may serve as a general neuronal mechanism of perceptual learning and reflect top-down mediated changes in cortical states.     

Orientation-Cue Invariant Population Responses to Contrast-Modulated and Phase-Reversed Contour Stimuli in Macaque V1 and V2

Visual scenes can be readily decomposed into a variety of oriented components, the processing of which is vital for object segregation and recognition. In primate V1 and V2, most neurons have small spatio-temporal receptive fields responding selectively to oriented luminance contours (first order), while only a subgroup of neurons signal non-luminance defined contours (second order). So how is the orientation of second-order contours represented at the population level in macaque V1 and V2? Here we compared the population responses in macaque V1 and V2 to two types of second-order contour stimuli generated either by modulation of contrast or phase reversal with those to first-order contour stimuli. Using intrinsic signal optical imaging, we found that the orientation of second-order contour stimuli was represented invariantly in the orientation columns of both macaque V1 and V2. A physiologically constrained spatio-temporal energy model of V1 and V2 neuronal populations could reproduce all the recorded population responses. These findings suggest that, at the population level, the primate early visual system processes the orientation of second-order contours initially through a linear spatio-temporal filter mechanism. Our results of population responses to different second-order contour stimuli support the idea that the orientation maps in primate V1 and V2 can be described as a spatial-temporal energy map.     

Dependence of V2 illusory contour response on V1 cell properties and topographic organization

An illusory contour is an image that is perceived as a contour in the absence of typical contour characteristics, such as a change in luminance or chromaticity across the stimulus. In cats and primates, cells that respond to illusory contours are sparse in cortical area V1, but are found in greater numbers in cortical area V2. We propose a model capable of illusory contour detection that is based on a realistic topographic organization of V1 cells, which reproduces the responses of individual cell types measured experimentally. The model allows us to explain several experimentally observed properties of V2 cells including variability in orientation tuning and inducer spacing preference. As a practical application, the model can be used to estimate the relationship between the severity of a cortical injury in the primary visual cortex and the deterioration of V2 cell responses to real and illusory contours.     

Transgenically encoded protein photoinactivation (FlAsH-FALI): acute inactivation of synaptotagmin I

Perceptual completion can link widely separated contour fragments and interpolate illusory contours (ICs) between them. The mechanisms underlying such long-range linking are not well understood. Here we report that completion is much poorer when ICs cross the vertical meridian than when they reside entirely within the left or right visual hemifield. This deficit reflects limitations in cross-hemispheric integration. We also show that the sensitivity to the interhemispheric divide is unique to perceptual completion: a comparable task which did not require completion showed no across-meridian impairment. We propose that these findings support the existence of specialized completion mechanisms in early visual cortical areas (V1/V2), since those areas are likely to be more sensitive to the interhemispheric divide.     

Seeing more than meets the eye: processing of illusory contours in animals

This review article illustrates that mammals, birds and insects are able to perceive illusory contours. Illusory contours lack a physical counterpart, but monkeys, cats, owls and bees perceive them as if they were real borders. In all of these species, a neural correlate for such perceptual completion phenomena has been described. The robustness of neuronal responses and the abundance of cells argue that such neurons might indeed represent a neural correlate for illusory contour perception. The internal state of an animal subject (i.e., alert and behaving) seems to be an important factor when correlating neural activity with perceptual phenomena. The fact that the neural network necessary for illusory contour perception has been found in relatively early visual brain areas in all tested animals suggests that bottom-up processing is largely sufficient to explain such perceptual abilities. However, recent findings in monkeys indicate that feedback loops within the visual system may provide additional modulation. The detection of illusory contours by independently evolved visual systems argues that processing of edges in the absence of contrast gradients reflects fundamental visual constraints and not just an artifact of visual processing.     

Motion-boundary illusions and their regularization

Humans use various cues to understand the structure of the world from images. One such cue is the contours of an object formed by occlusion or from surface discontinuities. It is known that contours in the image of an object provide various amounts of information about the shape of the object in view, depending on assumptions that the observer makes. Another powerful cue is motion. The ability of the human visual system to discern structure from a motion stimulus is well known and has a solid theoretical and experimental foundation. However, when humans interpret a visual scene they use various cues to understand what they observe, and the interpretation comes from combining the information acquired from the various modules devoted to specific cues. In such an integration of modules it seems that each cue carries a different weight and importance. We performed several experiments where we made sure that the only cues available to the observer were contour and motion. It turns out that when humans combine information from contour and motion to reconstruct the shape of an object in view, if the results of the two modules--shape from contour and structure from motion--are inconsistent, they experience a perceptual result which is due to the combination of the two modules, with the influence of the contour dominating, thus giving rise to the illusion. We describe here examples of such illusions and identify the conditions under which they happen. Finally, we introduce a computational theory for combining contour and motion using the theory of regularization. The theory explains such illusions and predicts many more.(ABSTRACT TRUNCATED AT 250 WORDS)     

A sparse object coding scheme in area V4

Sparse coding has long been recognized as a primary goal of image transformation in the visual system. Sparse coding in early visual cortex is achieved by abstracting local oriented spatial frequencies and by excitatory/inhibitory surround modulation. Object responses are thought to be sparse at subsequent processing stages, but neural mechanisms for higher-level sparsification are not known. Here, convergent results from macaque area V4 neural recording and simulated V4 populations trained on natural object contours suggest that sparse coding is achieved in midlevel visual cortex by emphasizing representation of acute convex and concave curvature. We studied 165 V4 neurons with a random, adaptive stimulus strategy to minimize bias and explore an unlimited range of contour shapes. V4 responses were strongly weighted toward contours containing acute convex or concave curvature. In contrast, the tuning distribution in nonsparse simulated V4 populations was strongly weighted toward low curvature. But as sparseness constraints increased, the simulated tuning distribution shifted progressively toward more acute convex and concave curvature, matching the neural recording results. These findings indicate a sparse object coding scheme in midlevel visual cortex based on uncommon but diagnostic regions of acute contour curvature.     

EEG Findings of Reduced Neural Synchronization during Visual Integration in Schizophrenia

Schizophrenia patients exhibit well-documented visual processing deficits. One area of disruption is visual integration, the ability to form global objects from local elements. However, most studies of visual integration in schizophrenia have been conducted in the context of an active attention task, which may influence the findings. In this study we examined visual integration using electroencephalography (EEG) in a passive task to elucidate neural mechanisms associated with poor visual integration. Forty-six schizophrenia patients and 30 healthy controls had EEG recorded while passively viewing figures comprised of real, illusory, or no contours. We examined visual P100, N100, and P200 event-related potential (ERP) components, as well as neural synchronization in the gamma (30-60 Hz) band assessed by the EEG phase locking factor (PLF). The N100 was significantly larger to illusory vs. no contour, and illusory vs. real contour stimuli while the P200 was larger only to real vs. illusory stimuli; there were no significant interactions with group. Compared to controls, patients failed to show increased phase locking to illusory versus no contours between 40-60 Hz. Also, controls, but not patients, had larger PLF between 30-40 Hz when viewing real vs. illusory contours. Finally, the positive symptom factor of the BPRS was negatively correlated with PLF values between 40-60 Hz to illusory stimuli, and with PLF between 30-40 Hz to real contour stimuli. These results suggest that the pattern of results across visual processing conditions is similar in patients and controls. However, patients have deficits in neural synchronization in the gamma range during basic processing of illusory contours when attentional demand is limited.     

Computational modeling and exploration of contour integration for visual saliency

We propose a computational model of contour integration for visual saliency. The model uses biologically plausible devices to simulate how the representations of elements aligned collinearly along a contour in an image are enhanced. Our model adds such devices as a dopamine-like fast plasticity, local GABAergic inhibition and multi-scale processing of images. The fast plasticity addresses the problem of how neurons in visual cortex seem to be able to influence neurons they are not directly connected to, for instance, as observed in contour closure effect. Local GABAergic inhibition is used to control gain in the system without using global mechanisms which may be non-plausible given the limited reach of axonal arbors in visual cortex. The model is then used to explore not only its validity in real and artificial images, but to discover some of the mechanisms involved in processing of complex visual features such as junctions and end-stops as well as contours. We present evidence for the validity of our model in several phases, starting with local enhancement of only a few collinear elements. We then test our model on more complex contour integration images with a large number of Gabor elements. Sections of the model are also extracted and used to discover how the model might relate contour integration neurons to neurons that process end-stops and junctions. Finally, we present results from real world images. Results from the model suggest that it is a good current approximation of contour integration in human vision. As well, it suggests that contour integration mechanisms may be strongly related to mechanisms for detecting end-stops and junction points. Additionally, a contour integration mechanism may be involved in finding features for objects such as faces. This suggests that visual cortex may be more information efficient and that neural regions may have multiple roles.     

End-stopping and the aperture problem: two-dimensional motion signals in macaque V1

Our perception of fine visual detail relies on small receptive fields at early stages of visual processing. However, small receptive fields tend to confound the orientation and velocity of moving edges, leading to ambiguous or inaccurate motion measurements (the aperture problem). Thus, it is often assumed that neurons in primary visual cortex (V1) carry only ambiguous motion information. Here we show that a subpopulation of V1 neurons is capable of signaling motion direction in a manner that is independent of contour orientation. Specifically, end-stopped V1 neurons obtain accurate motion measurements by responding only to the endpoints of long contours, a strategy which renders them largely immune to the aperture problem. Furthermore, the time course of end-stopping is similar to the time course of motion integration by MT neurons. These results suggest that cortical neurons might represent object motion by responding selectively to two-dimensional discontinuities in the visual scene.     

Optimality of human contour integration

For processing and segmenting visual scenes, the brain is required to combine a multitude of features and sensory channels. It is neither known if these complex tasks involve optimal integration of information, nor according to which objectives computations might be performed. Here, we investigate if optimal inference can explain contour integration in human subjects. We performed experiments where observers detected contours of curvilinearly aligned edge configurations embedded into randomly oriented distractors. The key feature of our framework is to use a generative process for creating the contours, for which it is possible to derive a class of ideal detection models. This allowed us to compare human detection for contours with different statistical properties to the corresponding ideal detection models for the same stimuli. We then subjected the detection models to realistic constraints and required them to reproduce human decisions for every stimulus as well as possible. By independently varying the four model parameters, we identify a single detection model which quantitatively captures all correlations of human decision behaviour for more than 2000 stimuli from 42 contour ensembles with greatly varying statistical properties. This model reveals specific interactions between edges closely matching independent findings from physiology and psychophysics. These interactions imply a statistics of contours for which edge stimuli are indeed optimally integrated by the visual system, with the objective of inferring the presence of contours in cluttered scenes. The recurrent algorithm of our model makes testable predictions about the temporal dynamics of neuronal populations engaged in contour integration, and it suggests a strong directionality of the underlying functional anatomy.     

Psychophysical tests of the hypothesis of a bottom-up saliency map in primary visual cortex

A unique vertical bar among horizontal bars is salient and pops out perceptually. Physiological data have suggested that mechanisms in the primary visual cortex (V1) contribute to the high saliency of such a unique basic feature, but indicated little regarding whether V1 plays an essential or peripheral role in input-driven or bottom-up saliency. Meanwhile, a biologically based V1 model has suggested that V1 mechanisms can also explain bottom-up saliencies beyond the pop-out of basic features, such as the low saliency of a unique conjunction feature such as a red vertical bar among red horizontal and green vertical bars, under the hypothesis that the bottom-up saliency at any location is signaled by the activity of the most active cell responding to it regardless of the cell's preferred features such as color and orientation. The model can account for phenomena such as the difficulties in conjunction feature search, asymmetries in visual search, and how background irregularities affect ease of search. In this paper, we report nontrivial predictions from the V1 saliency hypothesis, and their psychophysical tests and confirmations. The prediction that most clearly distinguishes the V1 saliency hypothesis from other models is that task-irrelevant features could interfere in visual search or segmentation tasks which rely significantly on bottom-up saliency. For instance, irrelevant colors can interfere in an orientation-based task, and the presence of horizontal and vertical bars can impair performance in a task based on oblique bars. Furthermore, properties of the intracortical interactions and neural selectivities in V1 predict specific emergent phenomena associated with visual grouping. Our findings support the idea that a bottom-up saliency map can be at a lower visual area than traditionally expected, with implications for top-down selection mechanisms.     

The role of attention in figure-ground segregation in areas V1 and V4 of the visual cortex

Our visual system segments images into objects and background. Figure-ground segregation relies on the detection of feature discontinuities that signal boundaries between the figures and the background and on a complementary region-filling process that groups together image regions with similar features. The neuronal mechanisms for these processes are not well understood and it is unknown how they depend on visual attention. We measured neuronal activity in V1 and V4 in a task where monkeys either made an eye movement to texture-defined figures or ignored them. V1 activity predicted the timing and the direction of the saccade if the figures were task relevant. We found that boundary detection is an early process that depends little on attention, whereas region filling occurs later and is facilitated by visual attention, which acts in an object-based manner. Our findings are explained by a model with local, bottom-up computations for boundary detection and feedback processing for region filling.     

Perceptual organization of local elements into global shapes in the human visual cortex

The question of how local image features on the retina are integrated into perceived global shapes is central to our understanding of human visual perception. Psychophysical investigations have suggested that the emergence of a coherent visual percept, or a "good-Gestalt", is mediated by the perceptual organization of local features based on their similarity. However, the neural mechanisms that mediate unified shape perception in the human brain remain largely unknown. Using human fMRI, we demonstrate that not only higher occipitotemporal but also early retinotopic areas are involved in the perceptual organization and detection of global shapes. Specifically, these areas showed stronger fMRI responses to global contours consisting of collinear elements than to patterns of randomly oriented local elements. More importantly, decreased detection performance and fMRI activations were observed when misalignment of the contour elements disturbed the perceptual coherence of the contours. However, grouping of the misaligned contour elements by disparity resulted in increased performance and fMRI activations, suggesting that similar neural mechanisms may underlie grouping of local elements to global shapes by different visual features (orientation or disparity). Thus, these findings provide novel evidence for the role of both early feature integration processes and higher stages of visual analysis in coherent visual perception.     

Effects of Spatial Frequency Similarity and Dissimilarity on Contour Integration

We examined the effects of spatial frequency similarity and dissimilarity on human contour integration under various conditions of uncertainty. Participants performed a temporal 2AFC contour detection task. Spatial frequency jitter up to 3.0 octaves was applied either to background elements, or to contour and background elements, or to none of both. Results converge on four major findings. (1) Contours defined by spatial frequency similarity alone are only scarcely visible, suggesting the absence of specialized cortical routines for shape detection based on spatial frequency similarity. (2) When orientation collinearity and spatial frequency similarity are combined along a contour, performance amplifies far beyond probability summation when compared to the fully heterogenous condition but only to a margin compatible with probability summation when compared to the fully homogenous case. (3) Psychometric functions are steeper but not shifted for homogenous contours in heterogenous backgrounds indicating an advantageous signal-to-noise ratio. The additional similarity cue therefore not so much improves contour detection performance but primarily reduces observer uncertainty about whether a potential candidate is a contour or just a false positive. (4) Contour integration is a broadband mechanism which is only moderately impaired by spatial frequency dissimilarity.     

Visual saliency and spike timing in the ventral visual pathway.

Visual saliency is a fundamental yet hard to define property of objects or locations in the visual world. In a context where objects and their representations compete to dominate our perception, saliency can be thought of as the "juice" that makes objects win the race. It is often assumed that saliency is extracted and represented in an explicit saliency map, which serves to determine the location of spatial attention at any given time. It is then by drawing attention to a salient object that it can be recognized or categorized. I argue against this classical view that visual "bottom-up" saliency automatically recruits the attentional system prior to object recognition. A number of visual processing tasks are clearly performed too fast for such a costly strategy to be employed. Rather, visual attention could simply act by biasing a saliency-based object recognition system. Under natural conditions of stimulation, saliency can be represented implicitly throughout the ventral visual pathway, independent of any explicit saliency map. At any given level, the most activated cells of the neural population simply represent the most salient locations. The notion of saliency itself grows increasingly complex throughout the system, mostly based on luminance contrast until information reaches visual cortex, gradually incorporating information about features such as orientation or color in primary visual cortex and early extrastriate areas, and finally the identity and behavioral relevance of objects in temporal cortex and beyond. Under these conditions the object that dominates perception, i.e. the object yielding the strongest (or the first) selective neural response, is by definition the one whose features are most "salient"--without the need for any external saliency map. In addition, I suggest that such an implicit representation of saliency can be best encoded in the relative times of the first spikes fired in a given neuronal population. In accordance with our subjective experience that saliency and attention do not modify the appearance of objects, the feed-forward propagation of this first spike wave could serve to trigger saliency-based object recognition outside the realm of awareness, while conscious perceptions could be mediated by the remaining discharges of longer neuronal spike trains.     

Contour integration and cortical processing.

Our understanding of visual processing in general, and contour integration in particular, has undergone great change over the last 10 years. There is now an accumulation of psychophysical and neurophysiological evidence that the outputs of cells with conjoint orientation preference and spatial position are integrated in the process of explication of rudimentary contours. Recent neuroanatomical and neurophysiological results suggest that this process takes place at the cortical level V1. The code for contour integration may be a temporal one in that it may only manifest itself in the latter part of the spike train as a result of feedback and lateral interactions. Here we review some of the properties of contour integration from a psychophysical perspective and we speculate on their underlying neurophysiological substrate.     

基于初级视觉皮层的同步振荡目标检测模型

提出一种基于初级视觉皮层的目标检测模型,该模型只采用方位选择性细胞和皮层内水平连接等Ｖ１基本单元,它以链码表示的目标轮廓作为知识,允许该知识以时间脉冲的形式控制Ｖ１区内神经细胞的动态活动,使与知识轮廓形状相符合的轮廓内的细胞进入同步振荡状态,实现对视野中特定目标轮廓的识别。计算机仿真结果表明,在较高级皮层的“知识”控制之下,初级视觉皮层结构上实现简单的目标检测是可行的。     

Feature-based image segmentation in human vision

Experiments are described which indicate that the integration of high-precision shape information along a bright line is blocked by the presence of certain image features. All the features involved have three properties: (1) they are points where contours are not smooth (i.e. not twice differentiable) within the limits set by the finite space constants of visual processes; (2) they are all points that are emphasized in the responses of certain classes of circularly symmetric bandpass spatial filter; and (3) they are all significant for three-dimensional shape analysis. The results are interpreted as implying an inflexible segmentation of the contour image before detailed shape analysis.     

Properties of V1 neurons tuned to conjunctions of visual features: application of the V1 saliency hypothesis to visual search behavior

From a computational theory of V1, we formulate an optimization problem to investigate neural properties in the primary visual cortex (V1) from human reaction times (RTs) in visual search. The theory is the V1 saliency hypothesis that the bottom-up saliency of any visual location is represented by the highest V1 response to it relative to the background responses. The neural properties probed are those associated with the less known V1 neurons tuned simultaneously or conjunctively in two feature dimensions. The visual search is to find a target bar unique in color (C), orientation (O), motion direction (M), or redundantly in combinations of these features (e.g., CO, MO, or CM) among uniform background bars. A feature singleton target is salient because its evoked V1 response largely escapes the iso-feature suppression on responses to the background bars. The responses of the conjunctively tuned cells are manifested in the shortening of the RT for a redundant feature target (e.g., a CO target) from that predicted by a race between the RTs for the two corresponding single feature targets (e.g., C and O targets). Our investigation enables the following testable predictions. Contextual suppression on the response of a CO-tuned or MO-tuned conjunctive cell is weaker when the contextual inputs differ from the direct inputs in both feature dimensions, rather than just one. Additionally, CO-tuned cells and MO-tuned cells are often more active than the single feature tuned cells in response to the redundant feature targets, and this occurs more frequently for the MO-tuned cells such that the MO-tuned cells are no less likely than either the M-tuned or O-tuned neurons to be the most responsive neuron to dictate saliency for an MO target.