Reduced Crowding and Poor Contour Detection in Schizophrenia Are Consistent with Weak Surround Inhibition

Background

Detection of visual contours (strings of small oriented elements) is markedly poor in schizophrenia. This has previously been attributed to an inability to group local information across space into a global percept. Here, we show that this failure actually originates from a combination of poor encoding of local orientation and abnormal processing of visual context.

Methods

We measured the ability of observers with schizophrenia to localise contours embedded in backgrounds of differently oriented elements (either randomly oriented, near-parallel or near-perpendicular to the contour). In addition, we measured patients’ ability to process local orientation information (i.e., report the orientation of an individual element) for both isolated and crowded elements (i.e., presented with nearby distractors).

Results

While patients are poor at detecting contours amongst randomly oriented elements, they are proportionally less disrupted (compared to unaffected controls) when contour and surrounding elements have similar orientations (near-parallel condition). In addition, patients are poor at reporting the orientation of an individual element but, again, are less prone to interference from nearby distractors, a phenomenon known as visual crowding.

Conclusions

We suggest that patients’ poor performance at contour perception arises not as a consequence of an “integration deficit” but from a combination of reduced sensitivity to local orientation and abnormalities in contextual processing. We propose that this is a consequence of abnormal gain control, a phenomenon that has been implicated in orientation-selectivity as well as surround suppression.     

Optimality of human contour integration

For processing and segmenting visual scenes, the brain is required to combine a multitude of features and sensory channels. It is neither known if these complex tasks involve optimal integration of information, nor according to which objectives computations might be performed. Here, we investigate if optimal inference can explain contour integration in human subjects. We performed experiments where observers detected contours of curvilinearly aligned edge configurations embedded into randomly oriented distractors. The key feature of our framework is to use a generative process for creating the contours, for which it is possible to derive a class of ideal detection models. This allowed us to compare human detection for contours with different statistical properties to the corresponding ideal detection models for the same stimuli. We then subjected the detection models to realistic constraints and required them to reproduce human decisions for every stimulus as well as possible. By independently varying the four model parameters, we identify a single detection model which quantitatively captures all correlations of human decision behaviour for more than 2000 stimuli from 42 contour ensembles with greatly varying statistical properties. This model reveals specific interactions between edges closely matching independent findings from physiology and psychophysics. These interactions imply a statistics of contours for which edge stimuli are indeed optimally integrated by the visual system, with the objective of inferring the presence of contours in cluttered scenes. The recurrent algorithm of our model makes testable predictions about the temporal dynamics of neuronal populations engaged in contour integration, and it suggests a strong directionality of the underlying functional anatomy.     

EEG Findings of Reduced Neural Synchronization during Visual Integration in Schizophrenia

Schizophrenia patients exhibit well-documented visual processing deficits. One area of disruption is visual integration, the ability to form global objects from local elements. However, most studies of visual integration in schizophrenia have been conducted in the context of an active attention task, which may influence the findings. In this study we examined visual integration using electroencephalography (EEG) in a passive task to elucidate neural mechanisms associated with poor visual integration. Forty-six schizophrenia patients and 30 healthy controls had EEG recorded while passively viewing figures comprised of real, illusory, or no contours. We examined visual P100, N100, and P200 event-related potential (ERP) components, as well as neural synchronization in the gamma (30-60 Hz) band assessed by the EEG phase locking factor (PLF). The N100 was significantly larger to illusory vs. no contour, and illusory vs. real contour stimuli while the P200 was larger only to real vs. illusory stimuli; there were no significant interactions with group. Compared to controls, patients failed to show increased phase locking to illusory versus no contours between 40-60 Hz. Also, controls, but not patients, had larger PLF between 30-40 Hz when viewing real vs. illusory contours. Finally, the positive symptom factor of the BPRS was negatively correlated with PLF values between 40-60 Hz to illusory stimuli, and with PLF between 30-40 Hz to real contour stimuli. These results suggest that the pattern of results across visual processing conditions is similar in patients and controls. However, patients have deficits in neural synchronization in the gamma range during basic processing of illusory contours when attentional demand is limited.     

Effects of Spatial Frequency Similarity and Dissimilarity on Contour Integration

We examined the effects of spatial frequency similarity and dissimilarity on human contour integration under various conditions of uncertainty. Participants performed a temporal 2AFC contour detection task. Spatial frequency jitter up to 3.0 octaves was applied either to background elements, or to contour and background elements, or to none of both. Results converge on four major findings. (1) Contours defined by spatial frequency similarity alone are only scarcely visible, suggesting the absence of specialized cortical routines for shape detection based on spatial frequency similarity. (2) When orientation collinearity and spatial frequency similarity are combined along a contour, performance amplifies far beyond probability summation when compared to the fully heterogenous condition but only to a margin compatible with probability summation when compared to the fully homogenous case. (3) Psychometric functions are steeper but not shifted for homogenous contours in heterogenous backgrounds indicating an advantageous signal-to-noise ratio. The additional similarity cue therefore not so much improves contour detection performance but primarily reduces observer uncertainty about whether a potential candidate is a contour or just a false positive. (4) Contour integration is a broadband mechanism which is only moderately impaired by spatial frequency dissimilarity.     

Integration of local features into global shapes: monkey and human FMRI studies

The integration of local image features into global shapes was investigated in monkeys and humans using fMRI. An adaptation paradigm was used, in which stimulus selectivity was deduced by changes in the course of adaptation of a pattern of randomly oriented elements. Accordingly, we observed stronger activity when orientation changes in the adapting stimulus resulted in a collinear contour than a different random pattern. This selectivity to collinear contours was observed not only in higher visual areas that are implicated in shape processing, but also in early visual areas where selectivity depended on the receptive field size. These findings suggest that unified shape perception in both monkeys and humans involves multiple visual areas that may integrate local elements to global shapes at different spatial scales.     

Computational modeling and exploration of contour integration for visual saliency

We propose a computational model of contour integration for visual saliency. The model uses biologically plausible devices to simulate how the representations of elements aligned collinearly along a contour in an image are enhanced. Our model adds such devices as a dopamine-like fast plasticity, local GABAergic inhibition and multi-scale processing of images. The fast plasticity addresses the problem of how neurons in visual cortex seem to be able to influence neurons they are not directly connected to, for instance, as observed in contour closure effect. Local GABAergic inhibition is used to control gain in the system without using global mechanisms which may be non-plausible given the limited reach of axonal arbors in visual cortex. The model is then used to explore not only its validity in real and artificial images, but to discover some of the mechanisms involved in processing of complex visual features such as junctions and end-stops as well as contours. We present evidence for the validity of our model in several phases, starting with local enhancement of only a few collinear elements. We then test our model on more complex contour integration images with a large number of Gabor elements. Sections of the model are also extracted and used to discover how the model might relate contour integration neurons to neurons that process end-stops and junctions. Finally, we present results from real world images. Results from the model suggest that it is a good current approximation of contour integration in human vision. As well, it suggests that contour integration mechanisms may be strongly related to mechanisms for detecting end-stops and junction points. Additionally, a contour integration mechanism may be involved in finding features for objects such as faces. This suggests that visual cortex may be more information efficient and that neural regions may have multiple roles.     

Perceptual organization of local elements into global shapes in the human visual cortex

The question of how local image features on the retina are integrated into perceived global shapes is central to our understanding of human visual perception. Psychophysical investigations have suggested that the emergence of a coherent visual percept, or a "good-Gestalt", is mediated by the perceptual organization of local features based on their similarity. However, the neural mechanisms that mediate unified shape perception in the human brain remain largely unknown. Using human fMRI, we demonstrate that not only higher occipitotemporal but also early retinotopic areas are involved in the perceptual organization and detection of global shapes. Specifically, these areas showed stronger fMRI responses to global contours consisting of collinear elements than to patterns of randomly oriented local elements. More importantly, decreased detection performance and fMRI activations were observed when misalignment of the contour elements disturbed the perceptual coherence of the contours. However, grouping of the misaligned contour elements by disparity resulted in increased performance and fMRI activations, suggesting that similar neural mechanisms may underlie grouping of local elements to global shapes by different visual features (orientation or disparity). Thus, these findings provide novel evidence for the role of both early feature integration processes and higher stages of visual analysis in coherent visual perception.     

Radial Frequency Analysis of Contour Shapes in the Visual Cortex

Cumulative psychophysical evidence suggests that the shape of closed contours is analysed by means of their radial frequency components (RFC). However, neurophysiological evidence for RFC-based representations is still missing. We investigated the representation of radial frequency in the human visual cortex with functional magnetic resonance imaging. We parametrically varied the radial frequency, amplitude and local curvature of contour shapes. The stimuli evoked clear responses across visual areas in the univariate analysis, but the response magnitude did not depend on radial frequency or local curvature. Searchlight-based, multivariate representational similarity analysis revealed RFC specific response patterns in areas V2d, V3d, V3AB, and IPS0. Interestingly, RFC-specific representations were not found in hV4 or LO, traditionally associated with visual shape analysis. The modulation amplitude of the shapes did not affect the responses in any visual area. Local curvature, SF-spectrum and contrast energy related representations were found across visual areas but without similar specificity for visual area that was found for RFC. The results suggest that the radial frequency of a closed contour is one of the cortical shape analysis dimensions, represented in the early and mid-level visual areas.     

The tactile perception of stimulus orientation

Studies of the visual system suggest that, at an early stage of form processing, a stimulus is represented as a set of contours and that a critical feature of these local contours is their orientation. Here, we characterize the ability of human observers to identify or discriminate the orientation of bars and edges presented to the distal fingerpad. The experiments were performed using a 400-probe stimulator that allowed us to flexibly deliver stimuli across a wide range of conditions. Orientation thresholds, approximately 20 degrees on average, varied only slightly across modes of stimulus presentation (scanned or indented), stimulus amplitudes, scanning speeds, and different stimulus types (bars or edges). The tactile orientation acuity was found to be poorer than its visual counterpart for stimuli of similar aspect ratio, contrast, and size. This result stands in contrast to the equivalent spatial acuity of the two systems (at the limit set by peripheral innervation density) and to the results of studies of tactile and visual letter recognition, which show that the two modalities yield comparable performance when stimuli are scaled appropriately.     

Learning to link visual contours

In complex visual scenes, linking related contour elements is important for object recognition. This process, thought to be stimulus driven and hard wired, has substrates in primary visual cortex (V1). Here, however, we find contour integration in V1 to depend strongly on perceptual learning and top-down influences that are specific to contour detection. In naive monkeys, the information about contours embedded in complex backgrounds is absent in V1 neuronal responses and is independent of the locus of spatial attention. Training animals to find embedded contours induces strong contour-related responses specific to the trained retinotopic region. These responses are most robust when animals perform the contour detection task but disappear under anesthesia. Our findings suggest that top-down influences dynamically adapt neural circuits according to specific perceptual tasks. This may serve as a general neuronal mechanism of perceptual learning and reflect top-down mediated changes in cortical states.     

The effect of contour closure on shape perception

We studied psychophysically whether 'contour closure' enhances the accuracy of shape perception. Stimulus configurations (presented on a blank background) always consisted of identical pattern elements, but the positions of the local elements were varied: the global contour shape either contained closure or not. In the first two stimulus conditions (Closure), the oriented pattern elements (Gabor patches) formed a 'closed' rectangular shape composed of either four long lines or four corners. In the third condition (No closure), the global shape was composed of the four corners, but they were outward oriented, and hence they did not form the outline of a closed contour. We measured the precision of shape perception using a discrimination task in which observers judged the aspect ratio of the outline shape i.e. whether the rectangular shape was tall or wide. We found that: (i) shape discrimination was better (more precise) for Closed contours than for Non-closed contours, i.e. the aspect ratio discrimination thresholds were lower for the Closed than Non-closed configurations. The improved performance could not be explained by differences in visibility of the local elements in the two conditions. (ii) For closed contours, shape discrimination was more precise when the local elements were aligned with the global shape, than when the local elements were orthogonal to it.     

A model of human pattern perception: association fields for adaptive spatial filters.

Visual neurons in the primary visual cortex 'look' at the retinal image through a four-dimensional array of spatial receptive fields (filter-elements): two spatial dimensions and, at each spatial location, two Fourier dimensions of spatial frequency and orientation. In general, visual objects activate filter-elements along each of these dimensions, suggesting a need for some kind of linking mechanism that determines whether two or more filter-elements are responding to the same or different contours or objects. In the spatial domain, a (spatial) association field between filter-elements, arranged to form first-order curves, has been inferred as a flexible method by which different parts of extended (luminance) contours become associated (Field et al., 1993). Linking has also been explored between filters selective for different regions in Fourier space (e.g. Georgeson and Meese, 1997). Perceived structure of stationary plaids suggests that spatial filtering is adaptive: synthetic filters can be created by the linear summation of basis-filters across orientation or spatial frequency in a stimulus-dependent way. For example, a plaid with a pair of sine-wave components at +/-45 deg looks like a blurred checkerboard; a structure that can be understood if features are derived after linear summation of spatial filters at different orientations. However, the addition of an oblique third-harmonic component causes the plaid to perceptually segment into overlapping oblique contours. This result can be understood if filters are summed across spatial frequency, but, in this case, treated independently across orientation. In the present paper, the architecture of an association field is proposed to permit linking and segmentation of filter-elements across spatial frequency and orientation. Three types of link are proposed: (1) A chain of constructive links around sites of common spatial frequency but different orientation, to promote binding of filters across orientation; (2) Constructive links between sites with common orientation but different spatial frequency, to promote binding of filters across spatial frequency; (3) Long-range links between sites of common spatial frequency but different orientation, whose activation and role are determined by activity in a higher spatial frequency band. A model employing the proposed network of links is consistent with at least six previously reported effects on the perception of briefly presented stationary plaids.     

Methods of analysis of neural nets

The properties of isotropy, smoothness, minimum curvature and locality suggest the shape of filled-in contours between two boundary edges. The contours are composed of the arcs of two circles tangent to the given edges, meeting smoothly, and minimizing the total curvature. It is shown that shapes meeting all the above requirement can be generated by a network which performs simple, local computations. It is suggested that the filling-in process plays an important role in the early processing of visual information.     

Feature-based image segmentation in human vision

Experiments are described which indicate that the integration of high-precision shape information along a bright line is blocked by the presence of certain image features. All the features involved have three properties: (1) they are points where contours are not smooth (i.e. not twice differentiable) within the limits set by the finite space constants of visual processes; (2) they are all points that are emphasized in the responses of certain classes of circularly symmetric bandpass spatial filter; and (3) they are all significant for three-dimensional shape analysis. The results are interpreted as implying an inflexible segmentation of the contour image before detailed shape analysis.     

Contour integration and cortical processing.

Our understanding of visual processing in general, and contour integration in particular, has undergone great change over the last 10 years. There is now an accumulation of psychophysical and neurophysiological evidence that the outputs of cells with conjoint orientation preference and spatial position are integrated in the process of explication of rudimentary contours. Recent neuroanatomical and neurophysiological results suggest that this process takes place at the cortical level V1. The code for contour integration may be a temporal one in that it may only manifest itself in the latter part of the spike train as a result of feedback and lateral interactions. Here we review some of the properties of contour integration from a psychophysical perspective and we speculate on their underlying neurophysiological substrate.     

Visual shape representation with geometrically characterized contour partitions

This paper proposes a biologically plausible matching method to recognize general shapes based on contour curvature information. The human visual system recognizes general shapes flexibly in real-world scenes through the ventral pathway. The pathway is typically modeled using artificial neural networks. These network models, however, do not construct a shape representation that satisfies the following required constraints: (1) The original shape should be represented by a group of partitioned contours in order to retrieve the whole shape (global information) from the partial contours (local information). (2) Coarse and fine structures of the original shapes should be individually represented in order for the visual system to respond to shapes as quickly as possible based on the least number of their features, and to discriminate between shapes based on detailed information. (3) The shape recognition realized with an artificial visual system should be invariant to geometric transformation such as expansion, rotation, or shear. In this paper, we propose a visual shape representation with geometrically characterized contour partitions described on multiple spatial scales.     

Experience shapes the utility of natural statistics for perceptual contour integration

Segmenting meaningful targets from cluttered scenes is a fundamental function of the visual system. Evolution and development have been suggested to optimize the brain's solution to this computationally challenging task by tuning the visual system to features that co-occur frequently in natural scenes (e.g., collinear edges) [1, 2, 3]. However, the role of shorter-term experience in shaping the utility of scene statistics remains largely unknown. Here, we ask whether collinearity is a specialized case, or whether the brain can learn to recruit any image regularity for the purpose of target identification. Consistent with long-term optimization for typical scene statistics, observers were better at detecting collinear contours than configurations of elements oriented at orthogonal or acute angles to the contour path. However, training resulted in improved detection of orthogonal contours that lasted for several months, suggesting retuning rather than transient changes of visual sensitivity. Improvement was also observed for acute contours but only after longer training. These results demonstrate that the brain flexibly exploits image regularities and learns to use discontinuities typically associated with surface boundaries (orthogonal, acute alignments) for contour linking and target identification. Thus, short-term experience in adulthood shapes the interpretation of scenes by assigning new statistical utility to image regularities.     

Energy filters,motion uncertainty,and motion sensitive cells in the visual cortex: a mathematical analysis

Energy filters are tuned to space-time frequency orientations. In order to compute velocity it is necessary to use a collection of filters, each tuned to a different space-time frequency. Here we analyze, in a probabilistic framework, the properties of the motion uncertainty. Its lower bound, which can be explicitly computed through the Cramér-Rao inequality, will have different values depending on the filter parameters. We show for the Gabor filter that, in order to minimize the motion uncertainty, the spatial and temporal filter sizes cannot be arbitrarily chosen; they are only allowed to vary over a limited range of values such that the temporal filter bandwidth is larger than the spatial bandwidth. This property is shared by motion sensitive cells in the primary visual cortex of the cat, which are known to be direction selective and are tuned to spacetime frequency orientations. We conjecture that these cells have larger temporal bandwidth relative to their spatial bandwidth because they compute velocity with maximum efficiency, that is, with a minimum motion uncertainty.     

Visual mechanisms of recognition of vanishing figures

The acuity test Landolt C's with a complex contour imitating various kinds of high-pass spatial frequency filtering generated as a printed visual acuity charts with a grey background were studied with regard to recognition thresholds. Comparisons were made for recognition thresholds with different complex contours. When the vanishing optotypes are out of focus, their image on the retina fades rapidly into the grey background, rendering them invisible rather than merely blurred as in the standard chart. The data obtained confirm that the recognition distance depends on physical property of complex contour profile and show the importance of the Landolt C spatial frequency spectrum for recognition of different types of the complex contour.     

Simulation of the effect of local adaption with nonlinear spatial filters

The visual system of vertebrates is capable of processing pattern signals over a wide range of intensity reaching from nearly absolute darkness to very bright sunlight. Typically the visual system of humans extracts fine contours of patterns of sufficiently high intensity or at high background intensity level, showing signal processing properties which can be explained by a bandpass system. Conversely, at very low intensity levels that system shows low-pass response: only coarse contours of patterns are recognized, however, the amplification of the signals has increased. The effect is called local adaption. A model is shown on the basis of a one-stage nonlinear spatial filter which, controlled by the local distribution of pattern intensity, can alter its frequency characteristic between low-pass response and bandpass response. Results are stated for computer-modelled filters. The investigation is restricted to one-dimensional filters, however, the results can be used to explain the function of two-dimensional filters qualitatively.