Idealized Computational Models for Auditory Receptive Fields

We present a theory by which idealized models of auditory receptive fields can be derived in a principled axiomatic manner, from a set of structural properties to (i) enable invariance of receptive field responses under natural sound transformations and (ii) ensure internal consistency between spectro-temporal receptive fields at different temporal and spectral scales. For defining a time-frequency transformation of a purely temporal sound signal, it is shown that the framework allows for a new way of deriving the Gabor and Gammatone filters as well as a novel family of generalized Gammatone filters, with additional degrees of freedom to obtain different trade-offs between the spectral selectivity and the temporal delay of time-causal temporal window functions. When applied to the definition of a second-layer of receptive fields from a spectrogram, it is shown that the framework leads to two canonical families of spectro-temporal receptive fields, in terms of spectro-temporal derivatives of either spectro-temporal Gaussian kernels for non-causal time or a cascade of time-causal first-order integrators over the temporal domain and a Gaussian filter over the logspectral domain. For each filter family, the spectro-temporal receptive fields can be either separable over the time-frequency domain or be adapted to local glissando transformations that represent variations in logarithmic frequencies over time. Within each domain of either non-causal or time-causal time, these receptive field families are derived by uniqueness from the assumptions. It is demonstrated how the presented framework allows for computation of basic auditory features for audio processing and that it leads to predictions about auditory receptive fields with good qualitative similarity to biological receptive fields measured in the inferior colliculus (ICC) and primary auditory cortex (A1) of mammals.     

Visual analysis in unspecialized receptive fields as an orthogonal series expansion

It was postulated on the basis of results obtained by investigating retinal receptive fields of cats that experimental data reflecting the pattern of sensitivity of the receptive field coincide with certain orthogonal polynomials. These special functions have certain properties which can be regarded as spatial characteristics of the receptive fields whose differential sensitivity they describe. Within the framework of the suggested hypothesis visual analysis in the retina from the mathematical point of view can be regarded as a process of expansion of the curve of spatial distribution of brightness within the receptive fields in an orthogonal series. Physiologically speaking the retinal receptive fields can be interpreted as filters distinguishing an elementary orthogonal harmonic from the signal. This method of analysis results in maximal economy of the coding of visual information in the retina. The concepts put forward provide a fresh explanation of a number of physiological facts and they provide precise quantitative evaluations of the response of the receptive field to an arbitrary stimulus.     

Emergence of spatiotemporal receptive fields and its application to motion detection

A model of motion sensitivity as observed in some cells of area V1 of the visual cortex is proposed. Motion sensitivity is achieved by a combination of different spatiotemporal receptive fields, in particular, spatial and temporal differentiators. The receptive fields emerge if a Hebbian learning rule is applied to the network. Similar to a Linsker model the network has a spatially convergent, linear feedforward structure. Additionally, however, delays omnipresent in the brain are incorporated in the model. The emerging spatiotemporal receptive fields are derived explicitly by extending the approach of MacKay and Miller. The response characteristic of the network is calculated in frequency space and shows that the network can be considered as a spacetime filter for motion in one direction. The emergence of different types of receptive field requires certain structural constraints regarding the spatial and temporal arborisation. These requirements can be derived from the theoretical analysis and might be compared with neuroanatomical data. In this way an explicit link between structure and function of the network is established.     

Evaluating Population Receptive Field Estimation Frameworks in Terms of Robustness and Reproducibility

Within vision research retinotopic mapping and the more general receptive field estimation approach constitute not only an active field of research in itself but also underlie a plethora of interesting applications. This necessitates not only good estimation of population receptive fields (pRFs) but also that these receptive fields are consistent across time rather than dynamically changing. It is therefore of interest to maximize the accuracy with which population receptive fields can be estimated in a functional magnetic resonance imaging (fMRI) setting. This, in turn, requires an adequate estimation framework providing the data for population receptive field mapping. More specifically, adequate decisions with regard to stimulus choice and mode of presentation need to be made. Additionally, it needs to be evaluated whether the stimulation protocol should entail mean luminance periods and whether it is advantageous to average the blood oxygenation level dependent (BOLD) signal across stimulus cycles or not. By systematically studying the effects of these decisions on pRF estimates in an empirical as well as simulation setting we come to the conclusion that a bar stimulus presented at random positions and interspersed with mean luminance periods is generally most favorable. Finally, using this optimal estimation framework we furthermore tested the assumption of temporal consistency of population receptive fields. We show that the estimation of pRFs from two temporally separated sessions leads to highly similar pRF parameters.     

A computational theory of visual receptive fields

A receptive field constitutes a region in the visual field where a visual cell or a visual operator responds to visual stimuli. This paper presents a theory for what types of receptive field profiles can be regarded as natural for an idealized vision system, given a set of structural requirements on the first stages of visual processing that reflect symmetry properties of the surrounding world. These symmetry properties include (i) covariance properties under scale changes, affine image deformations, and Galilean transformations of space–time as occur for real-world image data as well as specific requirements of (ii) temporal causality implying that the future cannot be accessed and (iii) a time-recursive updating mechanism of a limited temporal buffer of the past as is necessary for a genuine real-time system. Fundamental structural requirements are also imposed to ensure (iv) mutual consistency and a proper handling of internal representations at different spatial and temporal scales. It is shown how a set of families of idealized receptive field profiles can be derived by necessity regarding spatial, spatio-chromatic, and spatio-temporal receptive fields in terms of Gaussian kernels, Gaussian derivatives, or closely related operators. Such image filters have been successfully used as a basis for expressing a large number of visual operations in computer vision, regarding feature detection, feature classification, motion estimation, object recognition, spatio-temporal recognition, and shape estimation. Hence, the associated so-called scale-space theory constitutes a both theoretically well-founded and general framework for expressing visual operations. There are very close similarities between receptive field profiles predicted from this scale-space theory and receptive field profiles found by cell recordings in biological vision. Among the family of receptive field profiles derived by necessity from the assumptions, idealized models with very good qualitative agreement are obtained for (i) spatial on-center/off-surround and off-center/on-surround receptive fields in the fovea and the LGN, (ii) simple cells with spatial directional preference in V1, (iii) spatio-chromatic double-opponent neurons in V1, (iv) space–time separable spatio-temporal receptive fields in the LGN and V1, and (v) non-separable space–time tilted receptive fields in V1, all within the same unified theory. In addition, the paper presents a more general framework for relating and interpreting these receptive fields conceptually and possibly predicting new receptive field profiles as well as for pre-wiring covariance under scaling, affine, and Galilean transformations into the representations of visual stimuli. This paper describes the basic structure of the necessity results concerning receptive field profiles regarding the mathematical foundation of the theory and outlines how the proposed theory could be used in further studies and modelling of biological vision. It is also shown how receptive field responses can be interpreted physically, as the superposition of relative variations of surface structure and illumination variations, given a logarithmic brightness scale, and how receptive field measurements will be invariant under multiplicative illumination variations and exposure control mechanisms.     

A spatio-temporal model of ganglion cell receptive field in the cat retina

A spatio-temporal model of ganglion cell receptive fields is proposed on the basis of receptive field characteristics of cat retinal ganglion cells reported in our previous paper. The model consists of the linear and nonlinear mechanisms in the ganglion cell receptive field. The linear mechanism is assumed to be composed of antagonistic center and surround mechanisms. Then, by integrating these mechanisms we construct a spatio-temporal impulse response function of ganglion cell receptive field. Here we assume that spatio-temporal impulse response function may be factored into spatial and temporal terms. By Fouriertransforming the spatio-temporal impulse response function, we can obtain the spatio-temporal transfer function. Contrast sensitivity characteristics of X-and Y-cells in the cat retina may be explained by the transfer function.     

Invariance of visual operations at the level of receptive fields

The brain is able to maintain a stable perception although the visual stimuli vary substantially on the retina due to geometric transformations and lighting variations in the environment. This paper presents a theory for achieving basic invariance properties already at the level of receptive fields. Specifically, the presented framework comprises (i) local scaling transformations caused by objects of different size and at different distances to the observer, (ii) locally linearized image deformations caused by variations in the viewing direction in relation to the object, (iii) locally linearized relative motions between the object and the observer and (iv) local multiplicative intensity transformations caused by illumination variations. The receptive field model can be derived by necessity from symmetry properties of the environment and leads to predictions about receptive field profiles in good agreement with receptive field profiles measured by cell recordings in mammalian vision. Indeed, the receptive field profiles in the retina, LGN and V1 are close to ideal to what is motivated by the idealized requirements. By complementing receptive field measurements with selection mechanisms over the parameters in the receptive field families, it is shown how true invariance of receptive field responses can be obtained under scaling transformations, affine transformations and Galilean transformations. Thereby, the framework provides a mathematically well-founded and biologically plausible model for how basic invariance properties can be achieved already at the level of receptive fields and support invariant recognition of objects and events under variations in viewpoint, retinal size, object motion and illumination. The theory can explain the different shapes of receptive field profiles found in biological vision, which are tuned to different sizes and orientations in the image domain as well as to different image velocities in space-time, from a requirement that the visual system should be invariant to the natural types of image transformations that occur in its environment.     

A model study of the dynamic mechanisms of the double orientation tuning of the visual cortex neurons]

The dynamics of double orientation tuning of neurons in the visual cortex of cat was studied by means of computer simulation. It was possible to test the role of shape, relative localization and weight of the inhibitory and excitatory zones of a receptive field and the dynamics of these characteristics. It was shown that selective and acute double orientation tuning can be achieved only through changes in the weight of zones in the receptive fields with the end-stopping and lateral inhibitory zones whereas only the presence of the end-stopping inhibitory zone in the receptive field is sufficient during changes in zone localization and area.     

Experience-dependent asymmetric shape of hippocampal receptive fields

We propose a novel parameter, namely, the skewness, or asymmetry, of the shape of a receptive field to characterize two properties of hippocampal place fields. First, a majority of hippocampal receptive fields on linear tracks are negatively skewed, such that during a single pass the firing rate is low as the rat enters the field but high as it exits. Second, while the place fields are symmetric at the beginning of a session, they become highly asymmetric with experience. Further experiments suggest that these results are likely to arise due to synaptic plasticity during behavior. Using a purely feed forward neural network model, we show that following repeated directional activation, NMDA-dependent long-term potentiation/long-term depotentiation (LTP/LTD) could result in an experience-dependent asymmetrization of receptive fields.     

Structure of receptive fields of cat pulvinar neurons sensitive to photic stimulation

Receptive fields of 262 pulvinar neurons were studied. Receptive fields of 142 of these neurons were studied in detail with the aid of a stationary spot of light, flashing in different parts of the receptive field. Depending on responses to presentation of the stationary stimulus the neurons were divided into six groups. The first group included neurons with on—off responses to photic stimulation (44 of 142), the second group neurons with off responses only (42 of 142). In cells of the third group (19 of 142) an on response only was recorded in all structures of the receptive field tested. Neurons of the fourth group (eight of 142) had a receptive field of similar structure to that of the simple receptive fields of neurons in cortical area 17. The fifth group (10 of 142) included neurons with a receptive field of concentric structure, the sixth (19 of 142) consisted of neurons with receptive fields with multiple discharge centers. The structure of the receptive field of these neurons was mosaic, with an irregular distribution of exciting and "silent" zones. The mean response latency of the pulvinar neurons was 40–70 msec. Responses of neurons with shorter (20 msec) and longer (130–160 msec) latent periods also were recorded.L. A. Orbeli Institute of Physiology, Academy of Sciences of the Armenian SSR, Erevan. Translated from Neirofiziologiya, Vol. 11, No. 1, pp. 3–10, January–February, 1979.     

Are receptive fields of the visual cortex detectors or spatial frequency filters?

Besides its principal maximum, the spatial frequency characteristic curve of the complex visual cortical receptive field of curarized cats also has additional maxima and also negative regions, as predicted by the theory of piecewise Fourier analysis. Comparison of responses of the complex receptive field to sinusoidal gratings completely and incompletely contained in the field and comparison of responses to sinusoidal and square-wave gratings indicate that the receptive field, as a spatial frequency filter, has linear properties. The response of the complex receptive field rises with an increase in the number of periods of the sinusoidal grating. Several periods of optimal frequency match the complex field. Receptive fields tuned to a broad band of spatial frequencies were found in neuron columns. The results confirm the view that complex receptive fields are spatial frequency filters and not detectors.I. P. Pavlov Institute of Physiology, Academy of Sciences of the USSR, Leningrad. Translated from Neirofiziologiya, Vol. 11, No. 5, pp. 403–411, September–October, 1979.     

Small-field, binocular neurons in the superficial layers of the frog optic tectum

Binocular neurons with receptive fields about 5 degrees across were recorded just beneath the pia. Most of them responded to dark stimuli in the lower half of their receptive field and to light stimuli above. There was almost no vertical disparity between the left and right fields and the modal value of the horizontal disparity of the population of cells was 1.7 degrees. Because frogs do not verge their eyes it is possible to calculate at what distance the receptive fields through the two eyes are superimposed. This calculation suggests that the neurons are tuned to detect features in the external world about 50 cm away. This is too far for the neurons to be involved in the frog's everyday distance vision. It is more likely that they are concerned with assessing the vertical position of a horizontal surface.     

Local and global motion preferences in descending neurons of the fly

For a moving animal, optic flow is an important source of information about its ego-motion. In flies, the processing of optic flow is performed by motion sensitive tangential cells in the lobula plate. Amongst them, cells of the vertical system (VS cells) have receptive fields with similarities to optic flows generated during rotations around different body axes. Their output signals are further processed by pre-motor descending neurons. Here, we investigate the local motion preferences of two descending neurons called descending neurons of the ocellar and vertical system (DNOVS1 and DNOVS2). Using an LED arena subtending 240° × 95° of visual space, we mapped the receptive fields of DNOVS1 and DNOVS2 as well as those of their presynaptic elements, i.e. VS cells 1–10 and V2. The receptive field of DNOVS1 can be predicted in detail from the receptive fields of those VS cells that are most strongly coupled to the cell. The receptive field of DNOVS2 is a combination of V2 and VS cells receptive fields. Predicting the global motion preferences from the receptive field revealed a linear spatial integration in DNOVS1 and a superlinear spatial integration in DNOVS2. In addition, the superlinear integration of V2 output is necessary for DNOVS2 to differentiate between a roll rotation and a lift translation of the fly.     

Are V1 Simple Cells Optimized for Visual Occlusions? A Comparative Study

Simple cells in primary visual cortex were famously found to respond to low-level image components such as edges. Sparse coding and independent component analysis (ICA) emerged as the standard computational models for simple cell coding because they linked their receptive fields to the statistics of visual stimuli. However, a salient feature of image statistics, occlusions of image components, is not considered by these models. Here we ask if occlusions have an effect on the predicted shapes of simple cell receptive fields. We use a comparative approach to answer this question and investigate two models for simple cells: a standard linear model and an occlusive model. For both models we simultaneously estimate optimal receptive fields, sparsity and stimulus noise. The two models are identical except for their component superposition assumption. We find the image encoding and receptive fields predicted by the models to differ significantly. While both models predict many Gabor-like fields, the occlusive model predicts a much sparser encoding and high percentages of ‘globular’ receptive fields. This relatively new center-surround type of simple cell response is observed since reverse correlation is used in experimental studies. While high percentages of ‘globular’ fields can be obtained using specific choices of sparsity and overcompleteness in linear sparse coding, no or only low proportions are reported in the vast majority of studies on linear models (including all ICA models). Likewise, for the here investigated linear model and optimal sparsity, only low proportions of ‘globular’ fields are observed. In comparison, the occlusive model robustly infers high proportions and can match the experimentally observed high proportions of ‘globular’ fields well. Our computational study, therefore, suggests that ‘globular’ fields may be evidence for an optimal encoding of visual occlusions in primary visual cortex.     

Receptive fields of visual cortical neurons during changes in parameters of photic stimulation in cats

Spatial excitability contours in receptive fields of visual cortical neurons during changes in the physical and physiological parameters of photic stimulation were investigated in acute experiments on immobilized cats under conditions of dark, mesopic, and low photopic adaptation. With the change from dark to low mesopic adaptation the shape and size of the receptive fields detected by testing with flashes of constant intensity are unchanged, but with the transition to low photopic adaptation the receptive field becomes long and very narrow in 72% of cases, and the acuity of its orientational and directional tuning becomes much sharper. Against an unchanged background illumination, loss of brightness of the test light slit leads to narrowing of the measurable receptive field. Excitability contours of the receptive field estimated on the basis of absolute threshold of the cell response and level of intensity necessary to obtain the same number of spikes in the response become much narrower as the threshold criterion rises and during dark adaptation. Reactivity contours of the receptive field in response to stimulation of physiologically equal intensities (equal to the increase in threshold) under conditions of photopic adaptation also are much narrower than reactivity contours under conditions of dark adaptation. Evaluation of receptive fields with allowance for the possible contribution of effects of light scatter on the screen and in the ocular media showed that in most cases their size cannot be explained by these phenomena.Institute of Higher Nervous Activity and Neurophysiology, Academy of Sciences of the USSR, Moscow. Translated from Neirofiziologiya, Vol. 12, No. 2, pp. 115–123, March–April, 1980.     

Perisaccadic Remapping and Rescaling of Visual Responses in Macaque Superior Colliculus

Visual neurons have spatial receptive fields that encode the positions of objects relative to the fovea. Because foveate animals execute frequent saccadic eye movements, this position information is constantly changing, even though the visual world is generally stationary. Interestingly, visual receptive fields in many brain regions have been found to exhibit changes in strength, size, or position around the time of each saccade, and these changes have often been suggested to be involved in the maintenance of perceptual stability. Crucial to the circuitry underlying perisaccadic changes in visual receptive fields is the superior colliculus (SC), a brainstem structure responsible for integrating visual and oculomotor signals. In this work we have studied the time-course of receptive field changes in the SC. We find that the distribution of the latencies of SC responses to stimuli placed outside the fixation receptive field is bimodal: The first mode is comprised of early responses that are temporally locked to the onset of the visual probe stimulus and stronger for probes placed closer to the classical receptive field. We suggest that such responses are therefore consistent with a perisaccadic rescaling, or enhancement, of weak visual responses within a fixed spatial receptive field. The second mode is more similar to the remapping that has been reported in the cortex, as responses are time-locked to saccade onset and stronger for stimuli placed in the postsaccadic receptive field location. We suggest that these two temporal phases of spatial updating may represent different sources of input to the SC.     

Pattern-dependent response modulations in motion-sensitive visual interneurons--a model study

Even if a stimulus pattern moves at a constant velocity across the receptive field of motion-sensitive neurons, such as lobula plate tangential cells (LPTCs) of flies, the response amplitude modulates over time. The amplitude of these response modulations is related to local pattern properties of the moving retinal image. On the one hand, pattern-dependent response modulations have previously been interpreted as 'pattern-noise', because they deteriorate the neuron's ability to provide unambiguous velocity information. On the other hand, these modulations might also provide the system with valuable information about the textural properties of the environment. We analyzed the influence of the size and shape of receptive fields by simulations of four versions of LPTC models consisting of arrays of elementary motion detectors of the correlation type (EMDs). These models have previously been suggested to account for many aspects of LPTC response properties. Pattern-dependent response modulations decrease with an increasing number of EMDs included in the receptive field of the LPTC models, since spatial changes within the visual field are smoothed out by the summation of spatially displaced EMD responses. This effect depends on the shape of the receptive field, being the more pronounced--for a given total size--the more elongated the receptive field is along the direction of motion. Large elongated receptive fields improve the quality of velocity signals. However, if motion signals need to be localized the velocity coding is only poor but the signal provides--potentially useful--local pattern information. These modelling results suggest that motion vision by correlation type movement detectors is subject to uncertainty: you cannot obtain both an unambiguous and a localized velocity signal from the output of a single cell. Hence, the size and shape of receptive fields of motion sensitive neurons should be matched to their potential computational task.     

Types of receptive fields of the lateral geniculate body and their functional model

Functional differences between the type I and II receptive fields of the lateral geniculate body were studied in the cat. Some properties of these fields were shown to coincide with properties of "phasic" (Y type) and "tonic" (X type) of receptive fields. The type I fields have a limited range for transmission of information about the intensity of illumination; the type II fields, on the other hand, have a normal dynamic range of 2 log units. Using the number of spikes in groups as a measure of nervous activity, a neurophysiological scale of brightness corresponding to the psychological scale can be constructed on the basis of responses of the type II receptive field. It is postulated that type I receptive fields serve to transmit information on the shape of the image (spatial and temporal contrasts) and the type II fields transmit information on intensity of illumination. Investigation of the dynamic functional model showed that the type of receptive field is determined by the depth of inhibition through the interneuron. The depth of inhibition is much greater for type I than for type II.     

Zero-crossing detectors in primary visual cortex?

David Marr and others have hypothesized that the visual system processes complex scene information in stages, the first of which involves the detection of light intensity edges or zero-crossings (Marr, 1982). Ideal zero-crossing detector mechanisms have been described and modeled in terms of their possible physiological implementation (Marr and Hildreth, 1980; Poggio, 1983). We now present evidence of visual cortical receptive fields which resemble in spatial organizational terms the requirements of zero-crossing analysis. The linear and nonlinear summation within and between the receptive field subunits are described and compared with predicted processes. The relative subunit sizes and separations are analyzed in these terms. Our findings support the notion that receptive fields may correspond with zero-crossing filters rather than zero-crossing detector gates.