Computational model for perception of objects and motions

Perception of objects and motions in the visual scene is one of the basic problems in the visual system. There exist ‘What’ and ‘Where’ pathways in the superior visual cortex, starting from the simple cells in the primary visual cortex. The former is able to perceive objects such as forms, color, and texture, and the latter perceives ‘where’, for example, velocity and direction of spatial movement of objects. This paper explores brain-like computational architectures of visual information processing. We propose a visual perceptual model and computational mechanism for training the perceptual model. The computational model is a three-layer network. The first layer is the input layer which is used to receive the stimuli from natural environments. The second layer is designed for representing the internal neural information. The connections between the first layer and the second layer, called the receptive fields of neurons, are self-adaptively learned based on principle of sparse neural representation. To this end, we introduce Kullback-Leibler divergence as the measure of independence between neural responses and derive the learning algorithm based on minimizing the cost function. The proposed algorithm is applied to train the basis functions, namely receptive fields, which are localized, oriented, and bandpassed. The resultant receptive fields of neurons in the second layer have the characteristics resembling that of simple cells in the primary visual cortex. Based on these basis functions, we further construct the third layer for perception of what and where in the superior visual cortex. The proposed model is able to perceive objects and their motions with a high accuracy and strong robustness against additive noise. Computer simulation results in the final section show the feasibility of the proposed perceptual model and high efficiency of the learning algorithm.     

Perisaccadic Remapping and Rescaling of Visual Responses in Macaque Superior Colliculus

Visual neurons have spatial receptive fields that encode the positions of objects relative to the fovea. Because foveate animals execute frequent saccadic eye movements, this position information is constantly changing, even though the visual world is generally stationary. Interestingly, visual receptive fields in many brain regions have been found to exhibit changes in strength, size, or position around the time of each saccade, and these changes have often been suggested to be involved in the maintenance of perceptual stability. Crucial to the circuitry underlying perisaccadic changes in visual receptive fields is the superior colliculus (SC), a brainstem structure responsible for integrating visual and oculomotor signals. In this work we have studied the time-course of receptive field changes in the SC. We find that the distribution of the latencies of SC responses to stimuli placed outside the fixation receptive field is bimodal: The first mode is comprised of early responses that are temporally locked to the onset of the visual probe stimulus and stronger for probes placed closer to the classical receptive field. We suggest that such responses are therefore consistent with a perisaccadic rescaling, or enhancement, of weak visual responses within a fixed spatial receptive field. The second mode is more similar to the remapping that has been reported in the cortex, as responses are time-locked to saccade onset and stronger for stimuli placed in the postsaccadic receptive field location. We suggest that these two temporal phases of spatial updating may represent different sources of input to the SC.     

Adaptation of oriented and unoriented color-selective neurons in human visual areas

Primary visual cortex contains at least two distinct populations of color-selective cells: neurons in one have circularly symmetric receptive fields and respond best to reddish and greenish light, while neurons in another have oriented receptive fields and a variety of color preferences. The relative prevalence and perceptual roles of the two kinds of neurons remain controversial, however. We used fMRI and a selective adaptation technique to measure responses attributable to these two populations. The technique revealed evidence of adaptation in both populations and indicated that they each produced strong signals in V1 and other human visual areas. The activity of both sets of neurons was also reflected in color appearance measurements made with the same stimuli. Thus, both oriented and unoriented color-selective cells in V1 are important components of the neural pathways that underlie perception of color.     

Neural processes in symmetry perception: a parallel spatio-temporal model

Symmetry is usually computationally expensive to detect reliably, while it is relatively easy to perceive. In spite of many attempts to understand the neurofunctional properties of symmetry processing, no symmetry-specific activation was found in earlier cortical areas. Psychophysical evidence relating to the processing mechanisms suggests that the basic processes of symmetry perception would not perform a serial, point-by-point comparison of structural features but rather operate in parallel. Here, modeling of neural processes in psychophysical detection of bilateral texture symmetry is considered. A simple fine-grained algorithm that is capable of performing symmetry estimation without explicit comparison of remote elements is introduced. A computational model of symmetry perception is then described to characterize the underlying mechanisms as one-dimensional spatio-temporal neural processes, each of which is mediated by intracellular horizontal connections in primary visual cortex and adopts the proposed algorithm for the neural computation. Simulated experiments have been performed to show the efficiency and the dynamics of the model. Model and human performances are comparable for symmetry perception of intensity images. Interestingly, the responses of V1 neurons to propagation activities reflecting higher-order perceptual computations have been reported in neurophysiologic experiments.     

Neural network model of visual cortex for determining surface curvature from images of shaded surfaces

The visual system can extract information about shape from the pattern of light and dark surface shading on an object. Very little is known about how this is accomplished. We have used a learning algorithm to construct a neural network model that computes the principal curvatures and orientation of elliptic paraboloids independently of the illumination direction. Our chief finding is that receptive fields developed by units of such model network are surprisingly similar to some found in the visual cortex. It appears that neurons that can make use of the continuous gradations of shading have receptive fields similar to those previously interpreted as dealing with contours (i.e. 'bar' detectors or 'edge' detectors). This study illustrates the difficulty of deducing neuronal function within a network solely from receptive fields. It is also important to consider the pattern of connections a neuron makes with subsequent stages, which we call the 'projective field'.     

Learning receptive fields using predictive feedback.

Previously, it was suggested that feedback connections from higher- to lower-level areas carry predictions of lower-level neural activities, whereas feedforward connections carry the residual error between the predictions and the actual lower-level activities [Rao, R.P.N., Ballard, D.H., 1999. Nature Neuroscience 2, 79-87.]. A computational model implementing the hypothesis learned simple cell receptive fields when exposed to natural images. Here, we use predictive feedback to explain tuning properties in medial superior temporal area (MST). We implement the hypothesis using a new, biologically plausible, algorithm based on matching pursuit, which retains all the features of the previous implementation, including its ability to efficiently encode input. When presented with natural images, the model developed receptive field properties as found in primary visual cortex. In addition, when exposed to visual motion input resulting from movements through space, the model learned receptive field properties resembling those in MST. These results corroborate the idea that predictive feedback is a general principle used by the visual system to efficiently encode natural input.     

Two-dimensional substructure of stereo and motion interactions in macaque visual cortex

The analysis of object motion and stereoscopic depth are important tasks that are begun at early stages of the primate visual system. Using sparse white noise, we mapped the receptive field substructure of motion and disparity interactions in neurons in V1 and MT of alert monkeys. Interactions in both regions revealed subunits similar in structure to V1 simple cells. For both motion and stereo, the scale and shape of the receptive field substructure could be predicted from conventional tuning for bars or dot-field stimuli, indicating that the small-scale interactions were repeated across the receptive fields. We also found neurons in V1 and in MT that were tuned to combinations of spatial and temporal binocular disparities, suggesting a possible neural substrate for the perceptual Pulfrich phenomenon. Our observations constrain computational and developmental models of motion-stereo integration.     

Shaping of Receptive Fields in the Visual Cortex During Retinal Maturation

We present a computational study of the formation of simple-cell receptive field patterns in the primary visual cortex. Based on the observation that the spatial frequency of the retinal filter increases postnatally, our results explain differences in the time course of the development of orientation selectivity in binocularly deprived and normally reared kittens. Development after eye-opening in normal animals is modelled by training with natural images, whereas in the case of binocular deprivation noise-like stimulation continues. Further, it is shown that different orientation selectivities are obtained for network models trained with natural images in contrast to random phase images of identical second order statistics. The latter finding suggests that higher-order statistics of the inputs influences development of primary visual cortex. Finally, we search for quantities that identify possible signatures of natural image statistics in order to specify the amount of constructiveness that visual experience has on the formation of receptive fields.     

Does neuronal synchrony underlie visual feature grouping?

Previous research suggests that synchronous neural activity underlies perceptual grouping of visual image features. The generality of this mechanism is unclear, however, as previous studies have focused on pairs of neurons with overlapping or collinear receptive fields. By sampling more broadly and employing stimuli that contain partially occluded objects, we have conducted a more incisive test of the binding by synchrony hypothesis in area MT. We find that synchrony in spiking activity shows little dependence on feature grouping, whereas gamma band synchrony in field potentials can be significantly stronger when features are grouped. However, these changes in gamma band synchrony are small relative to the variability of synchrony across recording sites and do not provide a robust population signal for feature grouping. Moreover, these effects are reduced when stimulus differences nearby the receptive fields are eliminated using partial occlusion. Our findings suggest that synchrony does not constitute a general mechanism of visual feature binding.     

Construction of complex receptive fields in cat primary visual cortex.

In primary visual cortex, neurons are classified into simple cells and complex cells based on their response properties. Although the role of these two cell types in vision is still unknown, an attractive hypothesis is that simple cells are necessary to construct complex receptive fields. This hierarchical model puts forward two main predictions. First, simple cells should connect monosynaptically to complex cells. Second, complex cells should become silent when simple cells are inactivated. We have recently provided evidence for the first prediction, and here we do the same for the second. In summary, our results suggest that the receptive fields of most layer 2+3 complex cells are generated by a mechanism that requires simple cell inputs.     

A Network Model of Motion Processing in Area MT of Primates

A simple and biologically plausible model is proposed to simulatethe visual motion processing taking place in the middle temporal (MT) areaof the visual cortex in the primate brain. The model is ahierarchical neural network composed of multiple competitive learninglayers. The input layer of the network simulates the neurons in the primaryvisual cortex (V1), which are sensitive to the orientation and motionvelocity of the visual stimuli, and the middle and output layers of thenetwork simulate the component MT and pattern MT neurons, which areselectively responsive to local and global motions, respectively. Thenetwork model was tested with various simulated motion patterns (random dotsof different direction correlations, transparent motion, grating and plaidpatterns, and so on). The response properties of the model closely resemblemany of the known features of the MT neurons found neurophysiologically.These results show that the sophisticated response behaviors of the MTneurons can emerge naturally from some very simple models, such as acompetitive learning network.     

综述: 初级视皮层的亮度信息加工

亮度(luminance)是最基本的视觉信息.与其他视觉特征相比,由于视神经元对亮度刺激的反应较弱,并且许多神经元对均匀亮度无反应,对亮度信息编码的神经机制知之甚少.初级视皮层部分神经元对亮度的反应要慢于对比度反应,被认为是由边界对比度诱导的亮度知觉(brightness)的神经基础.我们的研究表明,初级视皮层许多神经元的亮度反应要快于对比度反应,并且这些神经元偏好低的空间频率、高的时间频率和高的运动速度,提示皮层下具有低空间频率和高运动速度通路的信息输入对产生初级视皮层神经元的亮度反应有贡献.已经知道初级视皮层神经元对空间频率反应的时间过程是从低空间频率到高空间频率,我们发现的早期亮度反应是对极低空间频率的反应,与这一时间过程是一致的,是这一从粗到细的视觉信息加工过程的第一步,揭示了处理最早的粗的视觉信息的神经基础.另外,初级视皮层含有偏好亮度下降和高运动速度的神经元,这群神经元的活动有助于在光照差的环境中检测高速运动的低亮度物体.     

Receptive fields in the rat piriform cortex.

Current models of odor discrimination in mammals involve molecular feature detection by a large family of diverse olfactory receptors, refinement of molecular feature extraction through precise projections of olfactory receptor neurons to the olfactory bulb to form an odor-specific spatial map of molecular features across glomerular layer, and synthesis of these features into odor objects within the piriform cortex. This review describes our recent work on odor and spatial receptive fields within the anterior piriform cortex and compares these fields with receptive fields of their primary afferent, olfactory bulb mitral/tufted cells. The results suggest that receptive fields in the piriform cortex are ensemble in nature, highly dynamic, and may contribute to odor discrimination and odor memory.     

Predictive Feedback Can Account for Biphasic Responses in the Lateral Geniculate Nucleus

Biphasic neural response properties, where the optimal stimulus for driving a neural response changes from one stimulus pattern to the opposite stimulus pattern over short periods of time, have been described in several visual areas, including lateral geniculate nucleus (LGN), primary visual cortex (V1), and middle temporal area (MT). We describe a hierarchical model of predictive coding and simulations that capture these temporal variations in neuronal response properties. We focus on the LGN-V1 circuit and find that after training on natural images the model exhibits the brain's LGN-V1 connectivity structure, in which the structure of V1 receptive fields is linked to the spatial alignment and properties of center-surround cells in the LGN. In addition, the spatio-temporal response profile of LGN model neurons is biphasic in structure, resembling the biphasic response structure of neurons in cat LGN. Moreover, the model displays a specific pattern of influence of feedback, where LGN receptive fields that are aligned over a simple cell receptive field zone of the same polarity decrease their responses while neurons of opposite polarity increase their responses with feedback. This phase-reversed pattern of influence was recently observed in neurophysiology. These results corroborate the idea that predictive feedback is a general coding strategy in the brain.     

The Gaussian derivative model for spatial-temporal vision: I. Cortical model

How do we see the motion of objects as well as their shapes? The Gaussian Derivative (GD) spatial model is extended to time to help answer this question. The GD spatio-temporal model requires only two numbers to describe the complete three-dimensional space-time shapes of individual receptive fields in primate visual cortex. These two numbers are the derivative numbers along the respective spatial and temporal principal axes of a given receptive field. Nine transformation parameters allow for a standard geometric association of these intrinsic axes with the extrinsic environment. The GD spatio-temporal model describes in one framework the following properties of primate simple cell fields: motion properties, number of lobes in space-time, spatial orientation. location, and size. A discrete difference-of-offset-Gaussians (DOOG) model provides a plausible physiological mechanism to form GD-like model fields in both space and time. The GD model hypothesizes that receptive fields at the first stage of processing in the visual cortex approximate 'derivative analyzers' that estimate local spatial and temporal derivatives of the intensity profile in the visual environment. The receptive fields as modeled provide operators that can allow later stages of processing in either a biological or machine vision system to estimate the motion as well as the shapes of objects in the environment.     

Direct evidence for encoding of motion streaks in human visual cortex

Temporal integration in the visual system causes fast-moving objects to generate static, oriented traces (‘motion streaks’), which could be used to help judge direction of motion. While human psychophysics and single-unit studies in non-human primates are consistent with this hypothesis, direct neural evidence from the human cortex is still lacking. First, we provide psychophysical evidence that faster and slower motions are processed by distinct neural mechanisms: faster motion raised human perceptual thresholds for static orientations parallel to the direction of motion, whereas slower motion raised thresholds for orthogonal orientations. We then used functional magnetic resonance imaging to measure brain activity while human observers viewed either fast (‘streaky’) or slow random dot stimuli moving in different directions, or corresponding static-oriented stimuli. We found that local spatial patterns of brain activity in early retinotopic visual cortex reliably distinguished between static orientations. Critically, a multivariate pattern classifier trained on brain activity evoked by these static stimuli could then successfully distinguish the direction of fast (‘streaky’) but not slow motion. Thus, signals encoding static-oriented streak information are present in human early visual cortex when viewing fast motion. These experiments show that motion streaks are present in the human visual system for faster motion.     

A neural network model on self-organizing emergence of simple-cell receptive field with orientation selectivity in visual cortex

In order to probe into the self-organizing emergence of simple cell orientation selectivity, we tried to construct a neural network model that consists of LGN neurons and simple cells in visual cortex and obeys the Hebbian learning rule. We investigated the neural coding and representation of simple cells to a natural image by means of this model. The results show that the structures of their receptive fields are determined by the preferred orientation selectivity of simple cells. However, they are also decided by the emergence of self-organization in the unsupervision learning process. This kind of orientation selectivity results from dynamic self-organization based on the interactions between LGN and cortex.     

A neural network model on self-organizing emergence of simple-cell receptive field with orientation selectivity in visual cortex

In order to probe into the self-organizing emergence of simple cell orientation selectivity, we tried to construct a neural network model that consists of LGN neurons and simple cells in visual cortex and obeys the Hebbian learning rule. We investigated the neural coding and representation of simple cells to a natural image by means of this model. The results show that the structures of their receptive fields are determined by the preferred orientation selectivity of simple cells. However, they are also decided by the emergence of self-organization in the unsupervision learning process. This kind of orientation selectivity results from dynamic self-organization based on the interactions between LGN and cortex.