Independent component analysis of natural image sequences yields spatio-temporal filters similar to simple cells in primary visual cortex.

Simple cells in the primary visual cortex process incoming visual information with receptive fields localized in space and time, bandpass in spatial and temporal frequency, tuned in orientation, and commonly selective for the direction of movement. It is shown that performing independent component analysis (ICA) on video sequences of natural scenes produces results with qualitatively similar spatio-temporal properties. Whereas the independent components of video resemble moving edges or bars, the independent component filters, i.e. the analogues of receptive fields, resemble moving sinusoids windowed by steady Gaussian envelopes. Contrary to earlier ICA results on static images, which gave only filters at the finest possible spatial scale, the spatio-temporal analysis yields filters at a range of spatial and temporal scales. Filters centred at low spatial frequencies are generally tuned to faster movement than those at high spatial frequencies.     

Are V1 Simple Cells Optimized for Visual Occlusions? A Comparative Study

Simple cells in primary visual cortex were famously found to respond to low-level image components such as edges. Sparse coding and independent component analysis (ICA) emerged as the standard computational models for simple cell coding because they linked their receptive fields to the statistics of visual stimuli. However, a salient feature of image statistics, occlusions of image components, is not considered by these models. Here we ask if occlusions have an effect on the predicted shapes of simple cell receptive fields. We use a comparative approach to answer this question and investigate two models for simple cells: a standard linear model and an occlusive model. For both models we simultaneously estimate optimal receptive fields, sparsity and stimulus noise. The two models are identical except for their component superposition assumption. We find the image encoding and receptive fields predicted by the models to differ significantly. While both models predict many Gabor-like fields, the occlusive model predicts a much sparser encoding and high percentages of ‘globular’ receptive fields. This relatively new center-surround type of simple cell response is observed since reverse correlation is used in experimental studies. While high percentages of ‘globular’ fields can be obtained using specific choices of sparsity and overcompleteness in linear sparse coding, no or only low proportions are reported in the vast majority of studies on linear models (including all ICA models). Likewise, for the here investigated linear model and optimal sparsity, only low proportions of ‘globular’ fields are observed. In comparison, the occlusive model robustly infers high proportions and can match the experimentally observed high proportions of ‘globular’ fields well. Our computational study, therefore, suggests that ‘globular’ fields may be evidence for an optimal encoding of visual occlusions in primary visual cortex.     

Are receptive fields of the visual cortex detectors or spatial frequency filters?

Besides its principal maximum, the spatial frequency characteristic curve of the complex visual cortical receptive field of curarized cats also has additional maxima and also negative regions, as predicted by the theory of piecewise Fourier analysis. Comparison of responses of the complex receptive field to sinusoidal gratings completely and incompletely contained in the field and comparison of responses to sinusoidal and square-wave gratings indicate that the receptive field, as a spatial frequency filter, has linear properties. The response of the complex receptive field rises with an increase in the number of periods of the sinusoidal grating. Several periods of optimal frequency match the complex field. Receptive fields tuned to a broad band of spatial frequencies were found in neuron columns. The results confirm the view that complex receptive fields are spatial frequency filters and not detectors.I. P. Pavlov Institute of Physiology, Academy of Sciences of the USSR, Leningrad. Translated from Neirofiziologiya, Vol. 11, No. 5, pp. 403–411, September–October, 1979.     

一种新的视觉皮层简单细胞工作机制的模型

提出了一种基于独立元分析（ICA）的视觉皮层简单细胞工作机制的模型。用Gabor函数逼近对自然图像进行ICA而获得的基函数,揭示了ICA基函数与视觉皮层简单细胞感受野反应间存在内在的关系。并对水平条纹的图像进行ICA,模拟在特殊视觉环境下生长的幼年动物的视觉皮层发育过程,证实了1970年Blakemore和Cooper在幼猫上的实验结果。从而说明ICA可以模拟动物的视觉皮层简单细胞工作过程。     

Analysis of the two-dimensional receptive fields learned by the Generalized Hebbian Algorithm in response to random input

The Generalized Hebbian Algorithm has been proposed for training linear feedforward neural networks and has been proven to cause the weights to converge to the eigenvectors of the input distribution (Sanger 1989a, b). For an input distribution given by 2D Gaussian smoothed white noise inside a Gaussian window, some of the masks learned by the Generalized Hebbian Algorithm resemble edge and bar detectors. Since these do not match the form of the actual eigenvectors of this distribution (Linsker 1987, 1990), we seek an explanation of the development of the masks prior to complete convergence to the correct solution. Analysis in the spatial and spatial frequency domains sheds light on this development, and shows that the masks which occur tend to be localized in the spatial frequency domain, reminiscent of one of the properties of 2D Gabor filters proposed by Daugman ( 1980, 1985) as a model for the receptive fields of cells in primate visual cortex.     

Spectral characteristics of stimuli and responses in a complex receptive field]

Responses of a complex receptive field of visual cortex of the cat are determined by the energy of stimuli in the band pass of equivalent space frequency characteristic of the receptive field. This is a proof that the complex receptive fields are spatial frequency band pass filters.     

Response histogram shapes and tuning curves: The predicted responses of several cortical cell types to drifting gratings stimuli

The responses to visual stimuli of simple cortical cells show linear spatial summation within and between their receptive field subunits. Complex cortical cells do not show this linearity. We analyzed the simulated responses to drifting sinusoidal grating stimuli of simple and of several types of complex cells. The complex cells, whose responses are seen to be half-wave rectified before pooling, have receptive fields consisting of two or more DOG (difference-of-Gaussians) shaped subunits. In both cases of stimulation by contrast-reversal gratings or drifting gratings, the cells' response as a function of spatial frequency is affected by the subunit distances 2 and the stimulation frequency . Furthermore, an increased number of subunits (a larger receptive field) yields a narrower peak tuning curve with decreased modulation depth for many of the spatial frequencies. The average and the peak response tuning curves are compared for the different receptive field types.     

Receptive field properties of near neighbor orientation selective neurons in the visual cortex: a modeling study

The primary visual cortex is organized into clusters of cells having similar receptive fields (RFs). A purely feedforward model has been shown to produce realistic simple cell receptive fields. The modeled cells capture a wide range of receptive field properties of orientation selective cortical cells. We have analyzed the responses of 78 nearby cell pairs to study which RF properties are clustered. Orientation preference shows strongest clustering. Orientation tuning width (hwhh) and tuning height (spikes/sec) at the preferred orientation are not as tightly clustered. Spatial frequency is also not as tightly clustered and RF phase has the least clustering. Clustering property of orientation preference, orientation tuning height and width depend on the location of cells in the orientation map. No such location dependence is observed for spatial frequency and RF phase. Our results agree well with experimental data.     

Texture discrimination by Gabor functions

A 2D Gabor filter can be realized as a sinusoidal plane wave of some frequency and orientation within a two dimensional Gaussian envelope. Its spatial extent, frequency and orientation preferences as well as bandwidths are easily controlled by the parameters used in generating the filters. However, there is an uncertainty relation associated with linear filters which limits the resolution simultaneously attainable in space and frequency. Daugman (1985) has determined that 2D Gabor filters are members of a class of functions achieving optimal joint resolution in the 2D space and 2D frequency domains. They have also been found to be a good model for two dimensional receptive fields of simple cells in the striate cortex (Jones 1985; Jones et al. 1985).The characteristic of optimal joint resolution in both space and frequency suggests that these filters are appropriate operators for tasks requiring simultaneous measurement in these domains. Texture discrimination is such a task. Computer application of a set of Gabor filters to a variety of textures found to be preattentively discriminable produces results in which differently textured regions are distinguished by firstorder differences in the values measured by the filters. This ability to reduce the statistical complexity distinguishing differently textured region as well as the sensitivity of these filters to certain types of local features suggest that Gabor functions can act as detectors of certain texton types. The performance of the computer models suggests that cortical neurons with Gabor like receptive fields may be involved in preattentive texture discrimination.     

Prediction of orientation selectivity from receptive field architecture in simple cells of cat visual cortex

From the intracellularly recorded responses to small, rapidly flashed spots, we have quantitatively mapped the receptive fields of simple cells in the cat visual cortex. We then applied these maps to a feedforward model of orientation selectivity. Both the preferred orientation and the width of orientation tuning of the responses to oriented stimuli were well predicted by the model. Where tested, the tuning curve was well predicted at different spatial frequencies. The model was also successful in predicting certain features of the spatial frequency selectivity of the cells. It did not successfully predict the amplitude of the responses to drifting gratings. Our results show that the spatial organization of the receptive field can account for a large fraction of the orientation selectivity of simple cells.     

Simple model for encoding natural images by retinal ganglion cells with nonlinear spatial integration

A central goal in sensory neuroscience is to understand the neuronal signal processing involved in the encoding of natural stimuli. A critical step towards this goal is the development of successful computational encoding models. For ganglion cells in the vertebrate retina, the development of satisfactory models for responses to natural visual scenes is an ongoing challenge. Standard models typically apply linear integration of visual stimuli over space, yet many ganglion cells are known to show nonlinear spatial integration, in particular when stimulated with contrast-reversing gratings. We here study the influence of spatial nonlinearities in the encoding of natural images by ganglion cells, using multielectrode-array recordings from isolated salamander and mouse retinas. We assess how responses to natural images depend on first- and second-order statistics of spatial patterns inside the receptive field. This leads us to a simple extension of current standard ganglion cell models. We show that taking not only the weighted average of light intensity inside the receptive field into account but also its variance over space can partly account for nonlinear integration and substantially improve response predictions of responses to novel images. For salamander ganglion cells, we find that response predictions for cell classes with large receptive fields profit most from including spatial contrast information. Finally, we demonstrate how this model framework can be used to assess the spatial scale of nonlinear integration. Our results underscore that nonlinear spatial stimulus integration translates to stimulation with natural images. Furthermore, the introduced model framework provides a simple, yet powerful extension of standard models and may serve as a benchmark for the development of more detailed models of the nonlinear structure of receptive fields.     

Theory of spatial position and spatial frequency relations in the receptive fields of simple cells in the visual cortex

Striate cells showing linear spatial summation obey very general mathematical inequalities relating the size of their receptive fields to the corresponding spatial frequency and orientation tuning characteristics. The experimental data show that, in the preferred direction of stimulus motion, the spatial response profiles of cells in the simple family are well described by the mathematical form of Gabor elementary signals. The product of the uncertainties in signalling spatial position (x) and spatial frequency (f) has, therefore, a theoretical minimum value of xf=1/2. We examine the implications that these conclusions have for the relationship between the spatial response profiles of simple cells and the characteristics of their spatial frequency tuning curves. Examples of the spatial frequency tuning curves and their associated spatial response profiles are discussed and illustrated. The advantages for the operation of the visual system of different relationships between the spatial response profiles and the characteristics of the spatial frequency tuning curves are examined. Two examples are discussed in detail, one system having a constant receptive field size and the other a constant bandwidth.     

Independent Component Analysis of Temporal Sequences Subject to Constraints by Lateral Geniculate Nucleus Inputs Yields All the Three Major Cell Types of the Primary Visual Cortex

Information maximization has long been suggested as the underlying coding strategy of the primary visual cortex (V1). Grouping image sequences into blocks has been shown by others to improve agreement between experiments and theory. We have studied the effect of temporal convolution on the formation of spatiotemporal filters—that is, the analogues of receptive fields—since this temporal feature is characteristic to the response function of lagged and nonlagged cells of the lateral geniculate nucleus. Concatenated input sequences were used to learn the linear transformation that maximizes the information transfer. Learning was accomplished by means of principal component analysis and independent component analysis. Properties of the emerging spatiotemporal filters closely resemble the three major types of V1 cells: simple cells with separable receptive field, simple cells with nonseparable receptive field, and complex cells.     

A functional model of the wiring of the simple cells of visual cortex

A computer model of the simple cells in the mammalian visual cortex was constructed. The model cells received inputs from a great number of isopolar centre/surround cells assumed to be located in the lateral geniculate nucleus (LGN). The distribution of input to the model simple cells was either inhibitory/excitatory or inhibitory/excitatory/inhibitory. Such arrangements produced receptive fields containing four or five consecutively antagonistic subfields. Responses produced by the model cells to different types of stimuli (periodical as well as nonperiodical) were obtained and compared to responses of living cells reported from various laboratories under comparable stimulus conditions. In all the situations tested, the responses of the model cells corresponded qualitatively very well to those of living cells. It was seen that the same wiring mechanism was able to account for orientation selectivity, spatial frequency filtering, various phase relationships between stimulus and response, subfield orientational selectivity, and slight end-inhibition. Furthermore, the receptive fields of the model simple cells closely resemble Gabor functions.     

The Gaussian derivative model for spatial-temporal vision: I. Cortical model

How do we see the motion of objects as well as their shapes? The Gaussian Derivative (GD) spatial model is extended to time to help answer this question. The GD spatio-temporal model requires only two numbers to describe the complete three-dimensional space-time shapes of individual receptive fields in primate visual cortex. These two numbers are the derivative numbers along the respective spatial and temporal principal axes of a given receptive field. Nine transformation parameters allow for a standard geometric association of these intrinsic axes with the extrinsic environment. The GD spatio-temporal model describes in one framework the following properties of primate simple cell fields: motion properties, number of lobes in space-time, spatial orientation. location, and size. A discrete difference-of-offset-Gaussians (DOOG) model provides a plausible physiological mechanism to form GD-like model fields in both space and time. The GD model hypothesizes that receptive fields at the first stage of processing in the visual cortex approximate 'derivative analyzers' that estimate local spatial and temporal derivatives of the intensity profile in the visual environment. The receptive fields as modeled provide operators that can allow later stages of processing in either a biological or machine vision system to estimate the motion as well as the shapes of objects in the environment.     

Features and functions of nonlinear spatial integration by retinal ganglion cells

Ganglion cells in the vertebrate retina integrate visual information over their receptive fields. They do so by pooling presynaptic excitatory inputs from typically many bipolar cells, which themselves collect inputs from several photoreceptors. In addition, inhibitory interactions mediated by horizontal cells and amacrine cells modulate the structure of the receptive field. In many models, this spatial integration is assumed to occur in a linear fashion. Yet, it has long been known that spatial integration by retinal ganglion cells also incurs nonlinear phenomena. Moreover, several recent examples have shown that nonlinear spatial integration is tightly connected to specific visual functions performed by different types of retinal ganglion cells. This work discusses these advances in understanding the role of nonlinear spatial integration and reviews recent efforts to quantitatively study the nature and mechanisms underlying spatial nonlinearities. These new insights point towards a critical role of nonlinearities within ganglion cell receptive fields for capturing responses of the cells to natural and behaviorally relevant visual stimuli. In the long run, nonlinear phenomena of spatial integration may also prove important for implementing the actual neural code of retinal neurons when designing visual prostheses for the eye.     

Natural Image Coding in V1: How Much Use Is Orientation Selectivity?

Orientation selectivity is the most striking feature of simple cell coding in V1 that has been shown to emerge from the reduction of higher-order correlations in natural images in a large variety of statistical image models. The most parsimonious one among these models is linear Independent Component Analysis (ICA), whereas second-order decorrelation transformations such as Principal Component Analysis (PCA) do not yield oriented filters. Because of this finding, it has been suggested that the emergence of orientation selectivity may be explained by higher-order redundancy reduction. To assess the tenability of this hypothesis, it is an important empirical question how much more redundancy can be removed with ICA in comparison to PCA or other second-order decorrelation methods. Although some previous studies have concluded that the amount of higher-order correlation in natural images is generally insignificant, other studies reported an extra gain for ICA of more than 100%. A consistent conclusion about the role of higher-order correlations in natural images can be reached only by the development of reliable quantitative evaluation methods. Here, we present a very careful and comprehensive analysis using three evaluation criteria related to redundancy reduction: In addition to the multi-information and the average log-loss, we compute complete rate–distortion curves for ICA in comparison with PCA. Without exception, we find that the advantage of the ICA filters is small. At the same time, we show that a simple spherically symmetric distribution with only two parameters can fit the data significantly better than the probabilistic model underlying ICA. This finding suggests that, although the amount of higher-order correlation in natural images can in fact be significant, the feature of orientation selectivity does not yield a large contribution to redundancy reduction within the linear filter bank models of V1 simple cells.     

A computational theory of visual receptive fields

A receptive field constitutes a region in the visual field where a visual cell or a visual operator responds to visual stimuli. This paper presents a theory for what types of receptive field profiles can be regarded as natural for an idealized vision system, given a set of structural requirements on the first stages of visual processing that reflect symmetry properties of the surrounding world. These symmetry properties include (i) covariance properties under scale changes, affine image deformations, and Galilean transformations of space–time as occur for real-world image data as well as specific requirements of (ii) temporal causality implying that the future cannot be accessed and (iii) a time-recursive updating mechanism of a limited temporal buffer of the past as is necessary for a genuine real-time system. Fundamental structural requirements are also imposed to ensure (iv) mutual consistency and a proper handling of internal representations at different spatial and temporal scales. It is shown how a set of families of idealized receptive field profiles can be derived by necessity regarding spatial, spatio-chromatic, and spatio-temporal receptive fields in terms of Gaussian kernels, Gaussian derivatives, or closely related operators. Such image filters have been successfully used as a basis for expressing a large number of visual operations in computer vision, regarding feature detection, feature classification, motion estimation, object recognition, spatio-temporal recognition, and shape estimation. Hence, the associated so-called scale-space theory constitutes a both theoretically well-founded and general framework for expressing visual operations. There are very close similarities between receptive field profiles predicted from this scale-space theory and receptive field profiles found by cell recordings in biological vision. Among the family of receptive field profiles derived by necessity from the assumptions, idealized models with very good qualitative agreement are obtained for (i) spatial on-center/off-surround and off-center/on-surround receptive fields in the fovea and the LGN, (ii) simple cells with spatial directional preference in V1, (iii) spatio-chromatic double-opponent neurons in V1, (iv) space–time separable spatio-temporal receptive fields in the LGN and V1, and (v) non-separable space–time tilted receptive fields in V1, all within the same unified theory. In addition, the paper presents a more general framework for relating and interpreting these receptive fields conceptually and possibly predicting new receptive field profiles as well as for pre-wiring covariance under scaling, affine, and Galilean transformations into the representations of visual stimuli. This paper describes the basic structure of the necessity results concerning receptive field profiles regarding the mathematical foundation of the theory and outlines how the proposed theory could be used in further studies and modelling of biological vision. It is also shown how receptive field responses can be interpreted physically, as the superposition of relative variations of surface structure and illumination variations, given a logarithmic brightness scale, and how receptive field measurements will be invariant under multiplicative illumination variations and exposure control mechanisms.     

A model of human pattern perception: association fields for adaptive spatial filters.

Visual neurons in the primary visual cortex 'look' at the retinal image through a four-dimensional array of spatial receptive fields (filter-elements): two spatial dimensions and, at each spatial location, two Fourier dimensions of spatial frequency and orientation. In general, visual objects activate filter-elements along each of these dimensions, suggesting a need for some kind of linking mechanism that determines whether two or more filter-elements are responding to the same or different contours or objects. In the spatial domain, a (spatial) association field between filter-elements, arranged to form first-order curves, has been inferred as a flexible method by which different parts of extended (luminance) contours become associated (Field et al., 1993). Linking has also been explored between filters selective for different regions in Fourier space (e.g. Georgeson and Meese, 1997). Perceived structure of stationary plaids suggests that spatial filtering is adaptive: synthetic filters can be created by the linear summation of basis-filters across orientation or spatial frequency in a stimulus-dependent way. For example, a plaid with a pair of sine-wave components at +/-45 deg looks like a blurred checkerboard; a structure that can be understood if features are derived after linear summation of spatial filters at different orientations. However, the addition of an oblique third-harmonic component causes the plaid to perceptually segment into overlapping oblique contours. This result can be understood if filters are summed across spatial frequency, but, in this case, treated independently across orientation. In the present paper, the architecture of an association field is proposed to permit linking and segmentation of filter-elements across spatial frequency and orientation. Three types of link are proposed: (1) A chain of constructive links around sites of common spatial frequency but different orientation, to promote binding of filters across orientation; (2) Constructive links between sites with common orientation but different spatial frequency, to promote binding of filters across spatial frequency; (3) Long-range links between sites of common spatial frequency but different orientation, whose activation and role are determined by activity in a higher spatial frequency band. A model employing the proposed network of links is consistent with at least six previously reported effects on the perception of briefly presented stationary plaids.