Monotonic coding of numerosity in macaque lateral intraparietal area

As any child knows, the first step in counting is summing up individual elements, yet the brain mechanisms responsible for this process remain obscure. Here we show, for the first time, that a population of neurons in the lateral intraparietal area of monkeys encodes the total number of elements within their classical receptive fields in a graded fashion, across a wide range of numerical values (2-32). Moreover, modulation of neuronal activity by visual quantity developed rapidly, within 100 ms of stimulus onset, and was independent of attention, reward expectations, or stimulus attributes such as size, density, or color. The responses of these neurons resemble the outputs of "accumulator neurons" postulated in computational models of number processing. Numerical accumulator neurons may provide inputs to neurons encoding specific cardinal values, such as "4," that have been described in previous work. Our findings may explain the frequent association of visuospatial and numerical deficits following damage to parietal cortex in humans.     

Computational modeling and exploration of contour integration for visual saliency

We propose a computational model of contour integration for visual saliency. The model uses biologically plausible devices to simulate how the representations of elements aligned collinearly along a contour in an image are enhanced. Our model adds such devices as a dopamine-like fast plasticity, local GABAergic inhibition and multi-scale processing of images. The fast plasticity addresses the problem of how neurons in visual cortex seem to be able to influence neurons they are not directly connected to, for instance, as observed in contour closure effect. Local GABAergic inhibition is used to control gain in the system without using global mechanisms which may be non-plausible given the limited reach of axonal arbors in visual cortex. The model is then used to explore not only its validity in real and artificial images, but to discover some of the mechanisms involved in processing of complex visual features such as junctions and end-stops as well as contours. We present evidence for the validity of our model in several phases, starting with local enhancement of only a few collinear elements. We then test our model on more complex contour integration images with a large number of Gabor elements. Sections of the model are also extracted and used to discover how the model might relate contour integration neurons to neurons that process end-stops and junctions. Finally, we present results from real world images. Results from the model suggest that it is a good current approximation of contour integration in human vision. As well, it suggests that contour integration mechanisms may be strongly related to mechanisms for detecting end-stops and junction points. Additionally, a contour integration mechanism may be involved in finding features for objects such as faces. This suggests that visual cortex may be more information efficient and that neural regions may have multiple roles.     

Noise effect on the recognition of fragmented contour images

We compared the results of recognition of fragmented contour images in the presence of noise and without noise. Both the contour images and the visual noise were synthesized with Gabor elements. The spacing between fragments in contour images and between noise elements, as well as the sizes of images, varied irrespective of one another. The percentage of recognition did not depend on the size of stimuli, but it differed for various objects in the presence and absence of noise. The percentage of recognition was higher for images with lots of turns in the absence of noise and, on the contrary, for images with lengthy contours with a lightly varying curvature in the presence of noise. The thresholds of recognition in noise depended, in general, on the ratio of the spacing between the elements in noise to the spacing between contour fragments.     

Adult visual cortical plasticity

The visual cortex has the capacity for experience-dependent change, or cortical plasticity, that is retained throughout life. Plasticity is invoked for encoding information during perceptual learning, by internally representing the regularities of the visual environment, which is useful for facilitating intermediate-level vision-contour integration and surface segmentation. The same mechanisms have adaptive value for functional recovery after CNS damage, such as that associated with stroke or neurodegenerative disease. A common feature to plasticity in primary visual cortex (V1) is an association field that links contour elements across the visual field. The circuitry underlying the association field includes a plexus of long-range horizontal connections formed by cortical pyramidal cells. These connections undergo rapid and exuberant sprouting and pruning in response to removal of sensory input, which can account for the topographic reorganization following retinal lesions. Similar alterations in cortical circuitry may be involved in perceptual learning, and the changes observed in V1 may be representative of how learned information is encoded throughout the cerebral cortex.     

A multi-stage model for fundamental functional properties in primary visual cortex

Many neurons in mammalian primary visual cortex have properties such as sharp tuning for contour orientation, strong selectivity for motion direction, and insensitivity to stimulus polarity, that are not shared with their sub-cortical counterparts. Successful models have been developed for a number of these properties but in one case, direction selectivity, there is no consensus about underlying mechanisms. We here define a model that accounts for many of the empirical observations concerning direction selectivity. The model describes a single column of cat primary visual cortex and comprises a series of processing stages. Each neuron in the first cortical stage receives input from a small number of on-centre and off-centre relay cells in the lateral geniculate nucleus. Consistent with recent physiological evidence, the off-centre inputs to cortex precede the on-centre inputs by a small (～4 ms) interval, and it is this difference that confers direction selectivity on model neurons. We show that the resulting model successfully matches the following empirical data: the proportion of cells that are direction selective; tilted spatiotemporal receptive fields; phase advance in the response to a stationary contrast-reversing grating stepped across the receptive field. The model also accounts for several other fundamental properties. Receptive fields have elongated subregions, orientation selectivity is strong, and the distribution of orientation tuning bandwidth across neurons is similar to that seen in the laboratory. Finally, neurons in the first stage have properties corresponding to simple cells, and more complex-like cells emerge in later stages. The results therefore show that a simple feed-forward model can account for a number of the fundamental properties of primary visual cortex.     

The peri-saccadic perception of objects and space

Eye movements affect object localization and object recognition. Around saccade onset, briefly flashed stimuli appear compressed towards the saccade target, receptive fields dynamically change position, and the recognition of objects near the saccade target is improved. These effects have been attributed to different mechanisms. We provide a unifying account of peri-saccadic perception explaining all three phenomena by a quantitative computational approach simulating cortical cell responses on the population level. Contrary to the common view of spatial attention as a spotlight, our model suggests that oculomotor feedback alters the receptive field structure in multiple visual areas at an intermediate level of the cortical hierarchy to dynamically recruit cells for processing a relevant part of the visual field. The compression of visual space occurs at the expense of this locally enhanced processing capacity.     

Recurrent V1–V2 interaction in early visual boundary processing

A majority of cortical areas are connected via feedforward and feedback fiber projections. In feedforward pathways we mainly observe stages of feature detection and integration. The computational role of the descending pathways at different stages of processing remains mainly unknown. Based on empirical findings we suggest that the top-down feedback pathways subserve a context-dependent gain control mechanism. We propose a new computational model for recurrent contour processing in which normalized activities of orientation selective contrast cells are fed forward to the next processing stage. There, the arrangement of input activation is matched against local patterns of contour shape. The resulting activities are subsequently fed back to the previous stage to locally enhance those initial measurements that are consistent with the top-down generated responses. In all, we suggest a computational theory for recurrent processing in the visual cortex in which the significance of local measurements is evaluated on the basis of a broader visual context that is represented in terms of contour code patterns. The model serves as a framework to link physiological with perceptual data gathered in psychophysical experiments. It handles a variety of perceptual phenomena, such as the local grouping of fragmented shape outline, texture surround and density effects, and the interpolation of illusory contours. Received: 28 October 1998 / Accepted in revised form: 19 March 1999     

Extraction of Surface-Related Features in a Recurrent Model of V1-V2 Interactions

Background

Humans can effortlessly segment surfaces and objects from two-dimensional (2D) images that are projections of the 3D world. The projection from 3D to 2D leads partially to occlusions of surfaces depending on their position in depth and on viewpoint. One way for the human visual system to infer monocular depth cues could be to extract and interpret occlusions. It has been suggested that the perception of contour junctions, in particular T-junctions, may be used as cue for occlusion of opaque surfaces. Furthermore, X-junctions could be used to signal occlusion of transparent surfaces.

Methodology/Principal Findings

In this contribution, we propose a neural model that suggests how surface-related cues for occlusion can be extracted from a 2D luminance image. The approach is based on feedforward and feedback mechanisms found in visual cortical areas V1 and V2. In a first step, contours are completed over time by generating groupings of like-oriented contrasts. Few iterations of feedforward and feedback processing lead to a stable representation of completed contours and at the same time to a suppression of image noise. In a second step, contour junctions are localized and read out from the distributed representation of boundary groupings. Moreover, surface-related junctions are made explicit such that they are evaluated to interact as to generate surface-segmentations in static images. In addition, we compare our extracted junction signals with a standard computer vision approach for junction detection to demonstrate that our approach outperforms simple feedforward computation-based approaches.

Conclusions/Significance

A model is proposed that uses feedforward and feedback mechanisms to combine contextually relevant features in order to generate consistent boundary groupings of surfaces. Perceptually important junction configurations are robustly extracted from neural representations to signal cues for occlusion and transparency. Unlike previous proposals which treat localized junction configurations as 2D image features, we link them to mechanisms of apparent surface segregation. As a consequence, we demonstrate how junctions can change their perceptual representation depending on the scene context and the spatial configuration of boundary fragments.     

A model of the ventral visual system based on temporal stability and local memory

The cerebral cortex is a remarkably homogeneous structure suggesting a rather generic computational machinery. Indeed, under a variety of conditions, functions attributed to specialized areas can be supported by other regions. However, a host of studies have laid out an ever more detailed map of functional cortical areas. This leaves us with the puzzle of whether different cortical areas are intrinsically specialized, or whether they differ mostly by their position in the processing hierarchy and their inputs but apply the same computational principles. Here we show that the computational principle of optimal stability of sensory representations combined with local memory gives rise to a hierarchy of processing stages resembling the ventral visual pathway when it is exposed to continuous natural stimuli. Early processing stages show receptive fields similar to those observed in the primary visual cortex. Subsequent stages are selective for increasingly complex configurations of local features, as observed in higher visual areas. The last stage of the model displays place fields as observed in entorhinal cortex and hippocampus. The results suggest that functionally heterogeneous cortical areas can be generated by only a few computational principles and highlight the importance of the variability of the input signals in forming functional specialization.     

Axonal processes and neural plasticity. III. Competition for dendrites.

In previous work we have developed a computational framework for topographic map formation and plasticity based on axonal process sprouting and retraction, in which sprouting and retraction are governed by competition for neurotrophic support. Here we show that such an approach can account for certain aspects of the dendritic morphology of cortical maps. In particular, we model the development of ocular dominance columns in the primary visual cortex and show that cortical cells near to column boundaries prefer to elaborate dendritic fields which avoid crossing the boundaries. This emerges as different functional inputs are spatially separated. We predict that afferent segregation occurs before or simultaneously with, but not after, the emergence of dendritic bias. We predict that animals reared with complete but asynchronous stimulation of the optic nerves do not develop a dendritic bias. We suggest that the emergence of a dendritic bias might provide a partial account for the critical period for a response to monocular deprivation. In particular, we predict that animals reared with asynchronous optic nerve stimulation might exhibit an extended critical period. Our results also indicate that the number of synapses supported by cortical cells depends on the intra-ocular image correlations used in our simulations. This suggests that inter-ocular image correlations, and thus strabismic rearing of kittens, may also affect the innervation density.     

End-stopping and the aperture problem: two-dimensional motion signals in macaque V1

Our perception of fine visual detail relies on small receptive fields at early stages of visual processing. However, small receptive fields tend to confound the orientation and velocity of moving edges, leading to ambiguous or inaccurate motion measurements (the aperture problem). Thus, it is often assumed that neurons in primary visual cortex (V1) carry only ambiguous motion information. Here we show that a subpopulation of V1 neurons is capable of signaling motion direction in a manner that is independent of contour orientation. Specifically, end-stopped V1 neurons obtain accurate motion measurements by responding only to the endpoints of long contours, a strategy which renders them largely immune to the aperture problem. Furthermore, the time course of end-stopping is similar to the time course of motion integration by MT neurons. These results suggest that cortical neurons might represent object motion by responding selectively to two-dimensional discontinuities in the visual scene.     

Lateral spike conduction velocity in the visual cortex affects spatial range of synchronization and receptive field size without visual experience: a learning model with spiking neurons

Classical receptive fields (cRF) increase in size from the retina to higher visual centers. The present work shows how temporal properties, in particular lateral spike velocity and spike input correlation, can affect cRF size and position without visual experience. We demonstrate how these properties are related to the spatial range of cortical synchronization if Hebbian learning dominates early development. For this, a largely reduced model of two successive levels of the visual cortex is developed (e.g., areas V1 and V2). It consists of retinotopic networks of spiking neurons with constant spike velocity in lateral connections. Feedforward connections between level 1 and 2 are additive and determine cRF size and shape, while lateral connections within level 1 are modulatory and affect the cortical range of synchronization. Input during development is mimicked by spike trains with spatially homogeneous properties and a confined temporal correlation width. During learning, the homogeneous lateral coupling shrinks to limited coupling structures defining synchronization and related association fields (AF). The size of level-1 synchronization fields determines the lateral coupling range of developing level-1-to-2 connections and, thus, the size of level-2 cRFs, even if the feedforward connections have distance-independent delays. AFs and cRFs increase with spike velocity in the lateral network and temporal correlation width of the input. Our results suggest that AF size of V1 and cRF size of V2 neurons are confined during learning by the temporal width of input correlations and the spike velocity in lateral connections without the need of visual experience. During learning from visual experience, a similar influence of AF size on the cRF size may be operative at successive levels of processing, including other parts of the visual system.     

Abnormal Contextual Modulation of Visual Contour Detection in Patients with Schizophrenia

Schizophrenia patients demonstrate perceptual deficits consistent with broad dysfunction in visual context processing. These include poor integration of segments forming visual contours, and reduced visual contrast effects (e.g. weaker orientation-dependent surround suppression, ODSS). Background image context can influence contour perception, as stimuli near the contour affect detection accuracy. Because of ODSS, this contextual modulation depends on the relative orientation between the contour and flanking elements, with parallel flankers impairing contour perception. However in schizophrenia, the impact of abnormal ODSS during contour perception is not clear. It is also unknown whether deficient contour perception marks genetic liability for schizophrenia, or is strictly associated with clinical expression of this disorder. We examined contour detection in 25 adults with schizophrenia, 13 unaffected first-degree biological relatives of schizophrenia patients, and 28 healthy controls. Subjects performed a psychophysics experiment designed to quantify the effect of flanker orientation during contour detection. Overall, patients with schizophrenia showed poorer contour detection performance than relatives or controls. Parallel flankers suppressed and orthogonal flankers enhanced contour detection performance for all groups, but parallel suppression was relatively weaker for schizophrenia patients than healthy controls. Relatives of patients showed equivalent performance with controls. Computational modeling suggested that abnormal contextual modulation in schizophrenia may be explained by suppression that is more broadly tuned for orientation. Abnormal flanker suppression in schizophrenia is consistent with weaker ODSS and/or broader orientation tuning. This work provides the first evidence that such perceptual abnormalities may not be associated with a genetic liability for schizophrenia.     

Predicting Cortical Dark/Bright Asymmetries from Natural Image Statistics and Early Visual Transforms

The nervous system has evolved in an environment with structure and predictability. One of the ubiquitous principles of sensory systems is the creation of circuits that capitalize on this predictability. Previous work has identified predictable non-uniformities in the distributions of basic visual features in natural images that are relevant to the encoding tasks of the visual system. Here, we report that the well-established statistical distributions of visual features -- such as visual contrast, spatial scale, and depth -- differ between bright and dark image components. Following this analysis, we go on to trace how these differences in natural images translate into different patterns of cortical input that arise from the separate bright (ON) and dark (OFF) pathways originating in the retina. We use models of these early visual pathways to transform natural images into statistical patterns of cortical input. The models include the receptive fields and non-linear response properties of the magnocellular (M) and parvocellular (P) pathways, with their ON and OFF pathway divisions. The results indicate that there are regularities in visual cortical input beyond those that have previously been appreciated from the direct analysis of natural images. In particular, several dark/bright asymmetries provide a potential account for recently discovered asymmetries in how the brain processes visual features, such as violations of classic energy-type models. On the basis of our analysis, we expect that the dark/bright dichotomy in natural images plays a key role in the generation of both cortical and perceptual asymmetries.     

A multi-differential neuromorphic approach to motion detection

This paper presents a multi-differential neuromorphic approach to motion detection. The model is based evidence for a differential operators interpretation of the properties of the cortical motion pathway. We discuss how this strategy, which provides a robust measure of speed for a range of types of image motion using a single computational mechanism, forms a useful framework in which to develop future neuromorphic motion systems. We also discuss both our approaches to developing computational motion models, and constraints in the design strategy for transferring motion models to other domains of early visual processing.     

Emotion Separation Is Completed Early and It Depends on Visual Field Presentation

It is now apparent that the visual system reacts to stimuli very fast, with many brain areas activated within 100 ms. It is, however, unclear how much detail is extracted about stimulus properties in the early stages of visual processing. Here, using magnetoencephalography we show that the visual system separates different facial expressions of emotion well within 100 ms after image onset, and that this separation is processed differently depending on where in the visual field the stimulus is presented. Seven right-handed males participated in a face affect recognition experiment in which they viewed happy, fearful and neutral faces. Blocks of images were shown either at the center or in one of the four quadrants of the visual field. For centrally presented faces, the emotions were separated fast, first in the right superior temporal sulcus (STS; 35–48 ms), followed by the right amygdala (57–64 ms) and medial pre-frontal cortex (83–96 ms). For faces presented in the periphery, the emotions were separated first in the ipsilateral amygdala and contralateral STS. We conclude that amygdala and STS likely play a different role in early visual processing, recruiting distinct neural networks for action: the amygdala alerts sub-cortical centers for appropriate autonomic system response for fight or flight decisions, while the STS facilitates more cognitive appraisal of situations and links appropriate cortical sites together. It is then likely that different problems may arise when either network fails to initiate or function properly.     

Solution of visuo-spatial tasks during masking

The images of two fragments of simple geometrical figures (square, triangle, etc.) were successively presented to healthy adult subjects in the left and right visual fields with the interval of 20, 80 and 380 ms; the subjects had to compare them mentally and decide whether they formed a geometrical figure. The correctness of reaction was controlled by a computer which lightened on the screen the word "correct" or "error". The number of correct decisions was significantly greater in response to the stimuli, forming a regular figure and increased with the increase of interstimuli interval. At the interval of 120 ms, when no regular figure could be formed from two fragments, the number of correct decisions was greater if the stimuli were presented in the left visual field. The reaction time did not depend on the hemisphere to which information was addressed; it was less in response to the stimuli forming a regular figure, and became shorter with the increase of the interstimuli interval.     

Running as fast as it can: How spiking dynamics form object groupings in the laminar circuits of visual cortex

How spiking neurons cooperate to control behavioral processes is a fundamental problem in computational neuroscience. Such cooperative dynamics are required during visual perception when spatially distributed image fragments are grouped into emergent boundary contours. Perceptual grouping is a challenge for spiking cells because its properties of collinear facilitation and analog sensitivity occur in response to binary spikes with irregular timing across many interacting cells. Some models have demonstrated spiking dynamics in recurrent laminar neocortical circuits, but not how perceptual grouping occurs. Other models have analyzed the fast speed of certain percepts in terms of a single feedforward sweep of activity, but cannot explain other percepts, such as illusory contours, wherein perceptual ambiguity can take hundreds of milliseconds to resolve by integrating multiple spikes over time. The current model reconciles fast feedforward with slower feedback processing, and binary spikes with analog network-level properties, in a laminar cortical network of spiking cells whose emergent properties quantitatively simulate parametric data from neurophysiological experiments, including the formation of illusory contours; the structure of non-classical visual receptive fields; and self-synchronizing gamma oscillations. These laminar dynamics shed new light on how the brain resolves local informational ambiguities through the use of properly designed nonlinear feedback spiking networks which run as fast as they can, given the amount of uncertainty in the data that they process.     

A neural network model for texture discrimination

A model of texture discrimination in visual cortex was built using a feedforward network with lateral interactions among relatively realistic spiking neural elements. The elements have various membrane currents, equilibrium potentials and time constants, with action potentials and synapses. The model is derived from the modified programs of MacGregor (1987). Gabor-like filters are applied to overlapping regions in the original image; the neural network with lateral excitatory and inhibitory interactions then compares and adjusts the Gabor amplitudes in order to produce the actual texture discrimination. Finally, a combination layer selects and groups various representations in the output of the network to form the final transformed image material. We show that both texture segmentation and detection of texture boundaries can be represented in the firing activity of such a network for a wide variety of synthetic to natural images. Performance details depend most strongly on the global balance of strengths of the excitatory and inhibitory lateral interconnections. The spatial distribution of lateral connective strengths has relatively little effect. Detailed temporal firing activities of single elements in the lateral connected network were examined under various stimulus conditions. Results show (as in area 17 of cortex) that a single element's response to image features local to its receptive field can be altered by changes in the global context.