Top-down modulation of lateral interactions in early vision: does attention affect integration of the whole or just perception of the parts?

Attention can modulate sensitivity to local stimuli in early vision. But, can attention also modulate integration of local stimuli into global visual patterns? We recently measured effects of attention on the phenomenon of lateral interactions between collinear elements, commonly thought to reflect long-range mechanisms in early visual cortex underlying contour integration. We showed improved detection of low-contrast central Gabor targets in the context of collinear flankers, but only when the collinear flankers were attended for a secondary task rather than ignored in favor of an orthogonal flanker pair. Here, we contrast two hypotheses for how attention might modulate flanker influences on the target: by changing just local sensitivity to the flankers themselves (flanker-modulation-only hypothesis), or by weighting integrative connections between flanker and target (connection-weighting hypothesis). Modeled on the known nonlinear dependence of target visibility on collinear flanker contrast, the first hypothesis predicts that an increase in physical flanker contrast should readily offset any reduction in their effective contrast when ignored, thus eliminating attentional modulation. Conversely, the second hypothesis predicts that attentional modulation should persist even for the highest flanker contrasts. Our results showed the latter outcome and indicated that attention modulates flanker-target integration, rather than just processing of local flanker elements.     

Perceptual organization of local elements into global shapes in the human visual cortex

The question of how local image features on the retina are integrated into perceived global shapes is central to our understanding of human visual perception. Psychophysical investigations have suggested that the emergence of a coherent visual percept, or a "good-Gestalt", is mediated by the perceptual organization of local features based on their similarity. However, the neural mechanisms that mediate unified shape perception in the human brain remain largely unknown. Using human fMRI, we demonstrate that not only higher occipitotemporal but also early retinotopic areas are involved in the perceptual organization and detection of global shapes. Specifically, these areas showed stronger fMRI responses to global contours consisting of collinear elements than to patterns of randomly oriented local elements. More importantly, decreased detection performance and fMRI activations were observed when misalignment of the contour elements disturbed the perceptual coherence of the contours. However, grouping of the misaligned contour elements by disparity resulted in increased performance and fMRI activations, suggesting that similar neural mechanisms may underlie grouping of local elements to global shapes by different visual features (orientation or disparity). Thus, these findings provide novel evidence for the role of both early feature integration processes and higher stages of visual analysis in coherent visual perception.     

Integration of local features into global shapes: monkey and human FMRI studies

The integration of local image features into global shapes was investigated in monkeys and humans using fMRI. An adaptation paradigm was used, in which stimulus selectivity was deduced by changes in the course of adaptation of a pattern of randomly oriented elements. Accordingly, we observed stronger activity when orientation changes in the adapting stimulus resulted in a collinear contour than a different random pattern. This selectivity to collinear contours was observed not only in higher visual areas that are implicated in shape processing, but also in early visual areas where selectivity depended on the receptive field size. These findings suggest that unified shape perception in both monkeys and humans involves multiple visual areas that may integrate local elements to global shapes at different spatial scales.     

The effect of contour closure on shape perception

We studied psychophysically whether 'contour closure' enhances the accuracy of shape perception. Stimulus configurations (presented on a blank background) always consisted of identical pattern elements, but the positions of the local elements were varied: the global contour shape either contained closure or not. In the first two stimulus conditions (Closure), the oriented pattern elements (Gabor patches) formed a 'closed' rectangular shape composed of either four long lines or four corners. In the third condition (No closure), the global shape was composed of the four corners, but they were outward oriented, and hence they did not form the outline of a closed contour. We measured the precision of shape perception using a discrimination task in which observers judged the aspect ratio of the outline shape i.e. whether the rectangular shape was tall or wide. We found that: (i) shape discrimination was better (more precise) for Closed contours than for Non-closed contours, i.e. the aspect ratio discrimination thresholds were lower for the Closed than Non-closed configurations. The improved performance could not be explained by differences in visibility of the local elements in the two conditions. (ii) For closed contours, shape discrimination was more precise when the local elements were aligned with the global shape, than when the local elements were orthogonal to it.     

Functional architecture of long-range perceptual interactions

The pattern of lateral interactions in the primary visual cortex, which has emerged from recent studies, conforms to the grouping rules of similarity, proximity, smoothness and closure. The goal of this paper is to understand the perceptual salience of oriented elements that are specifically organized to form a smooth contour. An overview of recent studies, in combination with new experimental results, is presented here to emphasis the idea that visual responses depend on input from both the center and the surround of the classical receptive field (CRF). It is assumed that normal lateral interactions produce a neuronal network that is formed by two antagonistic mechanisms: (i) excitation, that is spatially organized along the optimal orientation (collinear), and is predominant near the contrast threshold of the neuron, and (ii) inhibition, that is less selective and is distributed diffusely around the cell's response field. Thus, the inputs from the CRF and the anisotropic surround are summated non-linearly. The specificity of the facilitation and suppression along the collinear direction suggests the existence of second-order elongated collinear filters, which may increase the response similarity between neurons responding to elongated stimulus, thus may enhance the perceptual salience of anisotropic configurations such as contours. This causal connection is particularly evident in amblyopes, where abnormal development of the network results in the abnormal perception of contours.     

Contour detection threshold: repeatability and learning with 'contour cards'.

Human observers are able to locate contours that are defined solely on the basis of long-range, orientation-domain correlations. The integrity of the mechanisms responsible for second-order contour detection is disrupted by amblyopia (Kovacs et al., 1996; Hess et al., 1997) and it is therefore of interest to develop methods for assessing pediatric patients undergoing treatment for amblyopia. In this study, we have determined the inter-observer and test-retest reliability of a card-based test of second-order contour integration. The magnitude of practice effects was also assessed in both adult and pediatric patient groups. Contour detection thresholds were measured for a closed contour, defined by Gabor patches, embedded in a randomly oriented Gabor-patch background. The visibility of the contour was controlled by varying the density of the background elements. Thresholds, defined in terms of the ratio of contour element spacing to average background spacing were measured with a clinical staircase procedure. Thresholds measured by two observers differed on average by 0.023 +/- 0.075 or about one half the increment between cards. Children and adults showed only small practice effects (0.022 +/- 0.051 vs 0.053 +/- 0.077, respectively) and average unsigned differences between repeated measures were equivalent to approximately 1 card across groups. A card-based test of second-order contour integration produces reliable estimates of contour integration performance in normal and amblyopic observers, including children.     

Computational modeling and exploration of contour integration for visual saliency

We propose a computational model of contour integration for visual saliency. The model uses biologically plausible devices to simulate how the representations of elements aligned collinearly along a contour in an image are enhanced. Our model adds such devices as a dopamine-like fast plasticity, local GABAergic inhibition and multi-scale processing of images. The fast plasticity addresses the problem of how neurons in visual cortex seem to be able to influence neurons they are not directly connected to, for instance, as observed in contour closure effect. Local GABAergic inhibition is used to control gain in the system without using global mechanisms which may be non-plausible given the limited reach of axonal arbors in visual cortex. The model is then used to explore not only its validity in real and artificial images, but to discover some of the mechanisms involved in processing of complex visual features such as junctions and end-stops as well as contours. We present evidence for the validity of our model in several phases, starting with local enhancement of only a few collinear elements. We then test our model on more complex contour integration images with a large number of Gabor elements. Sections of the model are also extracted and used to discover how the model might relate contour integration neurons to neurons that process end-stops and junctions. Finally, we present results from real world images. Results from the model suggest that it is a good current approximation of contour integration in human vision. As well, it suggests that contour integration mechanisms may be strongly related to mechanisms for detecting end-stops and junction points. Additionally, a contour integration mechanism may be involved in finding features for objects such as faces. This suggests that visual cortex may be more information efficient and that neural regions may have multiple roles.     

Reduced Crowding and Poor Contour Detection in Schizophrenia Are Consistent with Weak Surround Inhibition

Background

Detection of visual contours (strings of small oriented elements) is markedly poor in schizophrenia. This has previously been attributed to an inability to group local information across space into a global percept. Here, we show that this failure actually originates from a combination of poor encoding of local orientation and abnormal processing of visual context.

Methods

We measured the ability of observers with schizophrenia to localise contours embedded in backgrounds of differently oriented elements (either randomly oriented, near-parallel or near-perpendicular to the contour). In addition, we measured patients’ ability to process local orientation information (i.e., report the orientation of an individual element) for both isolated and crowded elements (i.e., presented with nearby distractors).

Results

While patients are poor at detecting contours amongst randomly oriented elements, they are proportionally less disrupted (compared to unaffected controls) when contour and surrounding elements have similar orientations (near-parallel condition). In addition, patients are poor at reporting the orientation of an individual element but, again, are less prone to interference from nearby distractors, a phenomenon known as visual crowding.

Conclusions

We suggest that patients’ poor performance at contour perception arises not as a consequence of an “integration deficit” but from a combination of reduced sensitivity to local orientation and abnormalities in contextual processing. We propose that this is a consequence of abnormal gain control, a phenomenon that has been implicated in orientation-selectivity as well as surround suppression.     

Experience shapes the utility of natural statistics for perceptual contour integration

Segmenting meaningful targets from cluttered scenes is a fundamental function of the visual system. Evolution and development have been suggested to optimize the brain's solution to this computationally challenging task by tuning the visual system to features that co-occur frequently in natural scenes (e.g., collinear edges) [1, 2, 3]. However, the role of shorter-term experience in shaping the utility of scene statistics remains largely unknown. Here, we ask whether collinearity is a specialized case, or whether the brain can learn to recruit any image regularity for the purpose of target identification. Consistent with long-term optimization for typical scene statistics, observers were better at detecting collinear contours than configurations of elements oriented at orthogonal or acute angles to the contour path. However, training resulted in improved detection of orthogonal contours that lasted for several months, suggesting retuning rather than transient changes of visual sensitivity. Improvement was also observed for acute contours but only after longer training. These results demonstrate that the brain flexibly exploits image regularities and learns to use discontinuities typically associated with surface boundaries (orthogonal, acute alignments) for contour linking and target identification. Thus, short-term experience in adulthood shapes the interpretation of scenes by assigning new statistical utility to image regularities.     

Collinear effects on 3-Gabor alignment as a function of spacing, orientation and detectability.

Our ability to align three Gabor patches depends upon their internal carrier orientation; we are better at aligning vertical or horizontal patches than oblique patches (Keeble and Hess, 1998). However, the tuning of alignment to patch orientation has not studied in detail. We measured the alignment of a vertical target with reference patches varying in orientation and found it tuned to vertical (collinear) patches at centre-to-centre separation of three carrier periods, with a steep increase for oblique references and slight downturn for horizontal (orthogonal) references. Next, we increased separation between the patches, testing collinear, side-by-side, orthogonal and oblique configurations. Surprisingly, we found that the tuning for collinear patches was preserved. All ten observers tested had lower alignment thresholds for collinear patches. This effect extended to an inter-patch separation of 10 carrier periods (20 envelope standard deviations). Additionally, we measured contrast detection thresholds for the reference patches using the same stimuli. The collinear facilitation of alignment was even greater than the collinear facilitation of detection.     

Do alignment thresholds define a critical boundary in long-range detection facilitation with co-linear lines?

Thresholds for line contrast detection (experiment 2) were measured with a two-alternative temporal forced-choice procedure as a function of the spatial position of a vertical target line with regard to two co-linear context lines. The different spatial positions of the target line corresponded to values near the position discrimination threshold (experiment 1) reflecting the just detectable lateral offset, or non-co-linearity, between the context lines which were vertically separated by about 100 minutes of visual arc. Target and context lines were vertically separated by about 30 minutes of arc, had equal contrast polarity in one case, and opposite contrast polarity in the other. Strong line contrast detection facilitation is found at perceptually co-linear target locations. This facilitation decreases noticeably at a horizontal target offset that corresponds to the alignment threshold measured with the context lines. The effects are independent of the relative contrast polarity of target and context and, as shown in a third experiment, also independent of both the relative length or number of lines, and the magnitude of their absolute co-axial separation. This independence seems to hold, provided individual line length and co-axial distance between lines are larger than what appears to be the lower limit of the long-range spatial domain for orientation or contour integration (i.e. 20 minutes of arc), as determined by previous studies. The findings reported here suggest that alignment thresholds are likely to define a critical lateral boundary in long-range detection facilitation with co-linear lines. They support models of contour integration based on interactions between neural mechanisms that integrate local signals of contrast, orientation, and relative position or end-to-end alignment. Such mechanisms may help to explain the formation of representations of virtual contours and object contours in human perception.     

Detection of a gabor patch superimposed on an illusory contour

Interactions between visual stimuli have been found to be specific to the spatial frequency, orientation and phase of the interacting stimuli. We asked if there are any interactions between luminance-defined Gabor patches and Kanizsa-type illusory contours. In psychophysical experiments we studied whether induction of a vertical illusory line affects detection thresholds for a Gabor patch superimposed on this line and whether these effects depend on the orientation, spatial frequency and phase of the Gabor elements. Employing a 2AFC method with a staircase procedure we measured contrast detection thresholds and varied the orientation, spatial frequency and phase of the test Gabor patch and the separation between the two pacmen in four experimental series. The results show that in a situation where the two inducers generate perception of an illusory line, the contrast detection of the Gabor patch is facilitated relative to a control condition where the rotated pacmen do not induce illusory contours. This facilitation was more pronounced for test Gabor signals that were collinear to the illusory line, but the observer's performance was not altered by changes in the spatial frequency or phase of the Gabor stimuli. With increasing spatial separation of the two pacmen (and, consequently, with a decreasing support ratio), the difference between performance in the test and control conditions diminished. From the data obtained we cannot infer that we have measured some neural interactions between Gabor patches and Kanizsa-type illusory contours, and nor can we draw a unique conclusion about what causes the facilitation of detection of the test Gabor patch in the experimental situation that allows induction of the illusory line. We discuss possible mechanisms of the facilitation, such as contextual influences or a reduction of uncertainty about spatial location of the test Gabor patch.     

Optimality of human contour integration

For processing and segmenting visual scenes, the brain is required to combine a multitude of features and sensory channels. It is neither known if these complex tasks involve optimal integration of information, nor according to which objectives computations might be performed. Here, we investigate if optimal inference can explain contour integration in human subjects. We performed experiments where observers detected contours of curvilinearly aligned edge configurations embedded into randomly oriented distractors. The key feature of our framework is to use a generative process for creating the contours, for which it is possible to derive a class of ideal detection models. This allowed us to compare human detection for contours with different statistical properties to the corresponding ideal detection models for the same stimuli. We then subjected the detection models to realistic constraints and required them to reproduce human decisions for every stimulus as well as possible. By independently varying the four model parameters, we identify a single detection model which quantitatively captures all correlations of human decision behaviour for more than 2000 stimuli from 42 contour ensembles with greatly varying statistical properties. This model reveals specific interactions between edges closely matching independent findings from physiology and psychophysics. These interactions imply a statistics of contours for which edge stimuli are indeed optimally integrated by the visual system, with the objective of inferring the presence of contours in cluttered scenes. The recurrent algorithm of our model makes testable predictions about the temporal dynamics of neuronal populations engaged in contour integration, and it suggests a strong directionality of the underlying functional anatomy.     

Collinear Stimuli Induce Local and Cross-Areal Coherence in the Visual Cortex of Behaving Monkeys

Background

Collinear patterns of local visual stimuli are used to study contextual effects in the visual system. Previous studies have shown that proximal collinear flankers, unlike orthogonal, can enhance the detection of a low contrast central element. However, the direct neural interactions between cortical populations processing the individual flanker elements and the central element are largely unknown.

Methodology/Principal Findings

Using voltage-sensitive dye imaging (VSDI) we imaged neural population responses in V1 and V2 areas in fixating monkeys while they were presented with collinear or orthogonal arrays of Gabor patches. We then studied the spatio-temporal interactions between neuronal populations processing individual Gabor patches in the two conditions. Time-frequency analysis of the stimulus-evoked VSDI signal showed power increase mainly in low frequencies, i.e., the alpha band (α; 7–14 Hz). Power in the α-band was more discriminative at a single trial level than other neuronal population measures. Importantly, the collinear condition showed an increased intra-areal (V1-V1 and V2-V2) and inter-areal (V1-V2) α-coherence with shorter latencies than the orthogonal condition, both before and after the removal of the stimulus contribution. α-coherence appeared between discrete neural populations processing the individual Gabor patches: the central element and the flankers.

Conclusions/Significance

Our findings suggest that collinear effects are mediated by synchronization in a distributed network of proximal and distant neuronal populations within and across V1 and V2.     

Chromatic collinear facilitation, further evidence for chromatic form perception

Collinear facilitation of contrast detection of achromatic stimuli has been studied over the past decade by different groups. We measured collinear facilitation of chromatic contrast detection under equal-luminance (photometric quantity) and under isoluminance (minimum motion technique) conditions, as two different controls. The facilitation was tested for chromatic contrast detection of a foveal Gabor signal flanked by two high chromatic-contrast Gabor signals. The results indicated a significant facilitation in the presence of spatial adjacent collinear chromatic contrast signals, when the flankers were located at a short distance, across all observers for three chromatic channels. The facilitation was compared to a non-collinear flanker configuration. The results indicated no facilitation effect at the opposing phase configuration, at a short flanker distance, whereas a small facilitation was observed with a configuration at a longer flanker distance. The findings suggest that the performance and specificity of chromatic collinear facilitation is not impaired with regard to achromatic mechanisms.     

Psychometric curves of lateral facilitation

Visibility of an oriented stimulus may be enhanced by nearby stimuli that are co-aligned with the target. However, the underlying mechanism governing this facilitation is controversial. Here we measured the dependence of percent correct detection on the target's contrast (psychometric curve) with and without flankers, where flankers were either collinear with or orthogonal to the target. We find that the effect of collinear flankers can be described as a translation of the psychometric function along the linear contrast axis. This behavior is consistent, within experimental error, with two types of models: (1) non-linear transduction of target contrast with collinear flankers having additive effects on contrast, and (2) uncertainty reduction (Pelli, 1985) by collinear flankers. We discuss properties of collinear facilitation that can help deciding between these two models.     

Electrophysiological investigation of the texture discrimination mechanisms

In electrophysiological and psychophysical experiments, we investigated mechanisms of the visual system underlying local and global texture processing. Textures included rectangular matrixes composed of Gabor patches (sine wave grating windowed by a Gaussian envelope). Orientation of each grating varied from 0 to 165 degrees with the step of 15 degrees. Matrixes differed by the amount of Gabor patches with vertical or horizontal orientation. The observers' task was to discriminate the dominant orientation. The advantage of such stimuli involved a possibility to calculate global statistics of the textures, which we considered as the difference between whole amount of vertical and horizontal orientations in the stimulus irrespective of their location. The local statistics was calculated as relative amount of spatially organized nearby gratings (i. e. collinear contours). The subjects' accuracy was low in discriminating less organized textures and gradually improved with the amount of vertically of horizontally oriented Gabor patches, while the reaction time decreased. Visual evoked potentials (VEPs) recorded from occipital lobes revealed different dependencies of their components' magnitude on the amount of equally oriented gratings. Amplitude of the late positive component P3 with latency 400 ms directly depended on the texture discriminability, and N2 wave with latency 180 ms had an S-like dependence. Opposite to that, the magnitude of P2 wave with latency 260 ms was maximal in response to less organized textures and gradually decreased with the amount of equally oriented gratings. The dependencies received were compared with the textures' statistics. Data analysis allowed us to suppose that, in the conditions of our experimental paradigm, two mechanisms were involved in discrimination of the textures--the local and the global processing. We believe that by recording VEPs one can separately investigate activity of these two processes.     

Abnormal Contextual Modulation of Visual Contour Detection in Patients with Schizophrenia

Schizophrenia patients demonstrate perceptual deficits consistent with broad dysfunction in visual context processing. These include poor integration of segments forming visual contours, and reduced visual contrast effects (e.g. weaker orientation-dependent surround suppression, ODSS). Background image context can influence contour perception, as stimuli near the contour affect detection accuracy. Because of ODSS, this contextual modulation depends on the relative orientation between the contour and flanking elements, with parallel flankers impairing contour perception. However in schizophrenia, the impact of abnormal ODSS during contour perception is not clear. It is also unknown whether deficient contour perception marks genetic liability for schizophrenia, or is strictly associated with clinical expression of this disorder. We examined contour detection in 25 adults with schizophrenia, 13 unaffected first-degree biological relatives of schizophrenia patients, and 28 healthy controls. Subjects performed a psychophysics experiment designed to quantify the effect of flanker orientation during contour detection. Overall, patients with schizophrenia showed poorer contour detection performance than relatives or controls. Parallel flankers suppressed and orthogonal flankers enhanced contour detection performance for all groups, but parallel suppression was relatively weaker for schizophrenia patients than healthy controls. Relatives of patients showed equivalent performance with controls. Computational modeling suggested that abnormal contextual modulation in schizophrenia may be explained by suppression that is more broadly tuned for orientation. Abnormal flanker suppression in schizophrenia is consistent with weaker ODSS and/or broader orientation tuning. This work provides the first evidence that such perceptual abnormalities may not be associated with a genetic liability for schizophrenia.     

Context modulates the ERP signature of contour integration

We investigated how the electrophysiological signature of contour integration is changed by the context in which a contour is embedded. Specifically, we manipulated the orientations of Gabor elements surrounding an embedded shape outline. The amplitudes of early visual components over posterior scalp regions were changed by the presence of a contour, and by the orientation of elements surrounding the contour. Differences in context type had an effect on the early P1 and N1 components, but not on the later P2 component. The presence of a contour had an effect on the N1 and P2 components, but not on the earlier P1 component. A modulatory effect of context on contour integration was observed on the N1 component. These results highlight the importance of the context in which contour integration takes place.