Spike-based synaptic plasticity and the emergence of direction selective simple cells: simulation results

Direction selectivity (DS) of simple cells in the primary visual cortex was recently suggested to arise from short-term synaptic depression in thalamocortical afferents (Chance F, Nelson S, Abbott L (1998), J. Neuroscience 18(12): 4785–4799). In the model, two groups of afferents with spatially displaced receptive fields project through either depressing and non-depressing synapses onto the V1 cell. The degree of synaptic depression determines the temporal phase advance of the response to drifting gratings. We show that the spatial displacement and the appropriate degree of synaptic depression required for DS can develop within an unbiased input scenario by means of temporally asymmetric spike-timing dependent plasticity (STDP) which modifies both the synaptic strength and the degree of synaptic depression. Moving stimuli of random velocities and directions break any initial receptive field symmetry and produce DS. Frequency tuning curves and subthreshold membrane potentials akin to those measured for non-directional simple cells are thereby changed into those measured for directional cells. If STDP is such that down-regulation dominates up-regulation the overall synaptic strength adapts in a self-organizing way such that eventually the postsynaptic response for the non-preferred direction becomes subthreshold. To prevent unlearning of the acquired DS by randomly changing stimulus directions an additional learning threshold is necessary. To further protect the development of the simple cell properties against noise in the stimulus, asynchronous and irregular synaptic inputs are required.     

A reaction-diffusion model to capture disparity selectivity in primary visual cortex

Decades of experimental studies are available on disparity selective cells in visual cortex of macaque and cat. Recently, local disparity map for iso-orientation sites for near-vertical edge preference is reported in area 18 of cat visual cortex. No experiment is yet reported on complete disparity map in V1. Disparity map for layer IV in V1 can provide insight into how disparity selective complex cell receptive field is organized from simple cell subunits. Though substantial amounts of experimental data on disparity selective cells is available, no model on receptive field development of such cells or disparity map development exists in literature. We model disparity selectivity in layer IV of cat V1 using a reaction-diffusion two-eye paradigm. In this model, the wiring between LGN and cortical layer IV is determined by resource an LGN cell has for supporting connections to cortical cells and competition for target space in layer IV. While competing for target space, the same type of LGN cells, irrespective of whether it belongs to left-eye-specific or right-eye-specific LGN layer, cooperate with each other while trying to push off the other type. Our model captures realistic 2D disparity selective simple cell receptive fields, their response properties and disparity map along with orientation and ocular dominance maps. There is lack of correlation between ocular dominance and disparity selectivity at the cell population level. At the map level, disparity selectivity topography is not random but weakly clustered for similar preferred disparities. This is similar to the experimental result reported for macaque. The details of weakly clustered disparity selectivity map in V1 indicate two types of complex cell receptive field organization.     

Sparse Coding Models Can Exhibit Decreasing Sparseness while Learning Sparse Codes for Natural Images

The sparse coding hypothesis has enjoyed much success in predicting response properties of simple cells in primary visual cortex (V1) based solely on the statistics of natural scenes. In typical sparse coding models, model neuron activities and receptive fields are optimized to accurately represent input stimuli using the least amount of neural activity. As these networks develop to represent a given class of stimulus, the receptive fields are refined so that they capture the most important stimulus features. Intuitively, this is expected to result in sparser network activity over time. Recent experiments, however, show that stimulus-evoked activity in ferret V1 becomes less sparse during development, presenting an apparent challenge to the sparse coding hypothesis. Here we demonstrate that some sparse coding models, such as those employing homeostatic mechanisms on neural firing rates, can exhibit decreasing sparseness during learning, while still achieving good agreement with mature V1 receptive field shapes and a reasonably sparse mature network state. We conclude that observed developmental trends do not rule out sparseness as a principle of neural coding per se: a mature network can perform sparse coding even if sparseness decreases somewhat during development. To make comparisons between model and physiological receptive fields, we introduce a new nonparametric method for comparing receptive field shapes using image registration techniques.     

Predictive Feedback Can Account for Biphasic Responses in the Lateral Geniculate Nucleus

Biphasic neural response properties, where the optimal stimulus for driving a neural response changes from one stimulus pattern to the opposite stimulus pattern over short periods of time, have been described in several visual areas, including lateral geniculate nucleus (LGN), primary visual cortex (V1), and middle temporal area (MT). We describe a hierarchical model of predictive coding and simulations that capture these temporal variations in neuronal response properties. We focus on the LGN-V1 circuit and find that after training on natural images the model exhibits the brain's LGN-V1 connectivity structure, in which the structure of V1 receptive fields is linked to the spatial alignment and properties of center-surround cells in the LGN. In addition, the spatio-temporal response profile of LGN model neurons is biphasic in structure, resembling the biphasic response structure of neurons in cat LGN. Moreover, the model displays a specific pattern of influence of feedback, where LGN receptive fields that are aligned over a simple cell receptive field zone of the same polarity decrease their responses while neurons of opposite polarity increase their responses with feedback. This phase-reversed pattern of influence was recently observed in neurophysiology. These results corroborate the idea that predictive feedback is a general coding strategy in the brain.     

Receptive fields of disparity-tuned simple cells in macaque V1

Binocular simple cells in primary visual cortex (V1) are the first cells along the mammalian visual pathway to receive input from both eyes. Two models of how binocular simple cells could extract disparity information have been put forward. The phase-shift model proposes that the receptive fields in the two eyes have different subunit organizations, while the position-shift model proposes that they have different overall locations. In five fixating macaque monkeys, we recorded from 30 disparity-tuned simple cells that showed selectivity to the disparity in a random dot stereogram. High-resolution maps of the left and right eye receptive fields indicated that both phase and position shifts were common. Single cells usually showed a combination of the two, and the optimum disparity was best correlated with the sum of receptive field phase and position shift.     

A cooperation and competition based simple cell receptive field model and study of feed-forward linear and nonlinear contributions to orientation selectivity

We present a model for development of orientation selectivity in layer IV simple cells. Receptive field (RF) development in the model, is determined by diffusive cooperation and resource limited competition guided axonal growth and retraction in geniculocortical pathway. The simulated cortical RFs resemble experimental RFs. The receptive field model is incorporated in a three-layer visual pathway model consisting of retina, LGN and cortex. We have studied the effect of activity dependent synaptic scaling on orientation tuning of cortical cells. The mean value of hwhh (half width at half the height of maximum response) in simulated cortical cells is 58° when we consider only the linear excitatory contribution from LGN. We observe a mean improvement of 22.8° in tuning response due to the non-linear spiking mechanisms that include effects of threshold voltage and synaptic scaling factor.     

A multi-stage model for fundamental functional properties in primary visual cortex

Many neurons in mammalian primary visual cortex have properties such as sharp tuning for contour orientation, strong selectivity for motion direction, and insensitivity to stimulus polarity, that are not shared with their sub-cortical counterparts. Successful models have been developed for a number of these properties but in one case, direction selectivity, there is no consensus about underlying mechanisms. We here define a model that accounts for many of the empirical observations concerning direction selectivity. The model describes a single column of cat primary visual cortex and comprises a series of processing stages. Each neuron in the first cortical stage receives input from a small number of on-centre and off-centre relay cells in the lateral geniculate nucleus. Consistent with recent physiological evidence, the off-centre inputs to cortex precede the on-centre inputs by a small (～4 ms) interval, and it is this difference that confers direction selectivity on model neurons. We show that the resulting model successfully matches the following empirical data: the proportion of cells that are direction selective; tilted spatiotemporal receptive fields; phase advance in the response to a stationary contrast-reversing grating stepped across the receptive field. The model also accounts for several other fundamental properties. Receptive fields have elongated subregions, orientation selectivity is strong, and the distribution of orientation tuning bandwidth across neurons is similar to that seen in the laboratory. Finally, neurons in the first stage have properties corresponding to simple cells, and more complex-like cells emerge in later stages. The results therefore show that a simple feed-forward model can account for a number of the fundamental properties of primary visual cortex.     

A Feedback Model of Attention Explains the Diverse Effects of Attention on Neural Firing Rates and Receptive Field Structure

Visual attention has many effects on neural responses, producing complex changes in firing rates, as well as modifying the structure and size of receptive fields, both in topological and feature space. Several existing models of attention suggest that these effects arise from selective modulation of neural inputs. However, anatomical and physiological observations suggest that attentional modulation targets higher levels of the visual system (such as V4 or MT) rather than input areas (such as V1). Here we propose a simple mechanism that explains how a top-down attentional modulation, falling on higher visual areas, can produce the observed effects of attention on neural responses. Our model requires only the existence of modulatory feedback connections between areas, and short-range lateral inhibition within each area. Feedback connections redistribute the top-down modulation to lower areas, which in turn alters the inputs of other higher-area cells, including those that did not receive the initial modulation. This produces firing rate modulations and receptive field shifts. Simultaneously, short-range lateral inhibition between neighboring cells produce competitive effects that are automatically scaled to receptive field size in any given area. Our model reproduces the observed attentional effects on response rates (response gain, input gain, biased competition automatically scaled to receptive field size) and receptive field structure (shifts and resizing of receptive fields both spatially and in complex feature space), without modifying model parameters. Our model also makes the novel prediction that attentional effects on response curves should shift from response gain to contrast gain as the spatial focus of attention drifts away from the studied cell.     

An intracellular study of space and time representation in primary visual cortical receptive fields

In contrast with previous knowledge based on extracellular recordings, the recent development of intracellular techniques in vivo (sharp electrode or ‘blind patch’) ideally allows experimenters to analyze and dissect the contribution of feedforward and lateral connectivity in the functional expression of a synaptic ‘integration field’. We will present recent data which demonstrate that the visual receptive field of cortical neurons described at the level of subthreshold synaptic events extends over much larger regions of the visual field than previously thought, and that the capacity of cells to amplify subthreshold responses depends on the immediate past history of their membrane potential. Our data suggest that visual cortical receptive fields should not be considered as a fixed entity but more as a dynamic field of integration and association. Two types of dynamics can be argued for: 1) the spatial structure of the minimal discharge field (defined by suprathreshold activation of the cell) can be profoundly reorganized at least during development and most probably during selective phases of learning under the control of activity-dependent mechanisms. Adaptive changes in visual responses are thought to reflect long-lasting potentiation and/or depression of synaptic efficacies conveying ON- and OFF-center information; and 2) during sensory processing, reconfiguration of synaptic weights may be achieved on a much faster time-scale and linke to non-linear properties of the postsynaptic membrane as well as that of recruited networks. Association of information available in the central part of the receptive field (RF) and of input coming from the reputedly ‘unresponsive’ regions surrounding it, or arising simultaneously from different parts of the visual field, might be suppressive in certain cases and capable of boosting hidden responses in other cases, depending on the global stimulus configuration.     

Computational Modeling of Orientation Tuning Dynamics in Monkey Primary Visual Cortex

In the primate visual pathway, orientation tuning of neurons is first observed in the primary visual cortex. The LGN cells that comprise the thalamic input to V1 are not orientation tuned, but some V1 neurons are quite selective. Two main classes of theoretical models have been offered to explain orientation selectivity: feedforward models, in which inputs from spatially aligned LGN cells are summed together by one cortical neuron; and feedback models, in which an initial weak orientation bias due to convergent LGN input is sharpened and amplified by intracortical feedback. Recent data on the dynamics of orientation tuning, obtained by a cross-correlation technique, may help to distinguish between these classes of models. To test this possibility, we simulated the measurement of orientation tuning dynamics on various receptive field models, including a simple Hubel-Wiesel type feedforward model: a linear spatiotemporal filter followed by an integrate-and-fire spike generator. The computational study reveals that simple feedforward models may account for some aspects of the experimental data but fail to explain many salient features of orientation tuning dynamics in V1 cells. A simple feedback model of interacting cells is also considered. This model is successful in explaining the appearance of Mexican-hat orientation profiles, but other features of the data continue to be unexplained.     

A computational theory of visual receptive fields

A receptive field constitutes a region in the visual field where a visual cell or a visual operator responds to visual stimuli. This paper presents a theory for what types of receptive field profiles can be regarded as natural for an idealized vision system, given a set of structural requirements on the first stages of visual processing that reflect symmetry properties of the surrounding world. These symmetry properties include (i) covariance properties under scale changes, affine image deformations, and Galilean transformations of space–time as occur for real-world image data as well as specific requirements of (ii) temporal causality implying that the future cannot be accessed and (iii) a time-recursive updating mechanism of a limited temporal buffer of the past as is necessary for a genuine real-time system. Fundamental structural requirements are also imposed to ensure (iv) mutual consistency and a proper handling of internal representations at different spatial and temporal scales. It is shown how a set of families of idealized receptive field profiles can be derived by necessity regarding spatial, spatio-chromatic, and spatio-temporal receptive fields in terms of Gaussian kernels, Gaussian derivatives, or closely related operators. Such image filters have been successfully used as a basis for expressing a large number of visual operations in computer vision, regarding feature detection, feature classification, motion estimation, object recognition, spatio-temporal recognition, and shape estimation. Hence, the associated so-called scale-space theory constitutes a both theoretically well-founded and general framework for expressing visual operations. There are very close similarities between receptive field profiles predicted from this scale-space theory and receptive field profiles found by cell recordings in biological vision. Among the family of receptive field profiles derived by necessity from the assumptions, idealized models with very good qualitative agreement are obtained for (i) spatial on-center/off-surround and off-center/on-surround receptive fields in the fovea and the LGN, (ii) simple cells with spatial directional preference in V1, (iii) spatio-chromatic double-opponent neurons in V1, (iv) space–time separable spatio-temporal receptive fields in the LGN and V1, and (v) non-separable space–time tilted receptive fields in V1, all within the same unified theory. In addition, the paper presents a more general framework for relating and interpreting these receptive fields conceptually and possibly predicting new receptive field profiles as well as for pre-wiring covariance under scaling, affine, and Galilean transformations into the representations of visual stimuli. This paper describes the basic structure of the necessity results concerning receptive field profiles regarding the mathematical foundation of the theory and outlines how the proposed theory could be used in further studies and modelling of biological vision. It is also shown how receptive field responses can be interpreted physically, as the superposition of relative variations of surface structure and illumination variations, given a logarithmic brightness scale, and how receptive field measurements will be invariant under multiplicative illumination variations and exposure control mechanisms.     

Independent Component Analysis of Temporal Sequences Subject to Constraints by Lateral Geniculate Nucleus Inputs Yields All the Three Major Cell Types of the Primary Visual Cortex

Information maximization has long been suggested as the underlying coding strategy of the primary visual cortex (V1). Grouping image sequences into blocks has been shown by others to improve agreement between experiments and theory. We have studied the effect of temporal convolution on the formation of spatiotemporal filters—that is, the analogues of receptive fields—since this temporal feature is characteristic to the response function of lagged and nonlagged cells of the lateral geniculate nucleus. Concatenated input sequences were used to learn the linear transformation that maximizes the information transfer. Learning was accomplished by means of principal component analysis and independent component analysis. Properties of the emerging spatiotemporal filters closely resemble the three major types of V1 cells: simple cells with separable receptive field, simple cells with nonseparable receptive field, and complex cells.     

The role of retinal bipolar cell in early vision: an implication with analogue networks and regularization theory

A linear analogue network model is proposed to describe the neuronal circuit of the outer retina consisting of cones, horizontal cells, and bipolar cells. The model reflects previous physiological findings on the spatial response properties of these neurons to dim illumination and is expressed by physiological mechanisms, i.e., membrane conductances, gap-junctional conductances, and strengths of chemical synaptic interactions. Using the model, we characterized the spatial filtering properties of the bipolar cell receptive field with the standard regularization theory, in which the early vision problems are attributed to minimization of a cost function. The cost function accompanying the present characterization is derived from the linear analogue network model, and one can gain intuitive insights on how physiological mechanisms contribute to the spatial filtering properties of the bipolar cell receptive field. We also elucidated a quantitative relation between the Laplacian of Gaussian operator and the bipolar cell receptive field. From the computational point of view, the dopaminergic modulation of the gap-junctional conductance between horizontal cells is inferred to be a suitable neural adaptation mechanism for transition between photopic and mesopic vision. Received: 20 December 1996 / Accepted in revised form: 16 May 1997     

Motion detection, noise reduction, texture suppression, and contour enhancement by spatiotemporal Gabor filters with surround inhibition

We study the orientation and speed tuning properties of spatiotemporal three-dimensional (3D) Gabor and motion energy filters as models of time-dependent receptive fields of simple and complex cells in the primary visual cortex (V1). We augment the motion energy operator with surround suppression to model the inhibitory effect of stimuli outside the classical receptive field. We show that spatiotemporal integration and surround suppression lead to substantial noise reduction. We propose an effective and straightforward motion detection computation that uses the population code of a set of motion energy filters tuned to different velocities. We also show that surround inhibition leads to suppression of texture and thus improves the visibility of object contours and facilitates figure/ground segregation and the detection and recognition of objects.     

Computational models of visual neurons specialised in the detection of periodic and aperiodic oriented visual stimuli: bar and grating cells

Computational models of periodic- and aperiodic-pattern selective cells, also called grating and bar cells, respectively, are proposed. Grating cells are found in areas V1 and V2 of the visual cortex of monkeys and respond strongly to bar gratings of a given orientation and periodicity but very weakly or not at all to single bars. This non-linear behaviour, which is quite different from the spatial frequency filtering behaviour exhibited by the other types of orientation-selective neurons such as the simple cells, is incorporated in the proposed computational model by using an AND-type non-linearity to combine the responses of simple cells with symmetric receptive field profiles and opposite polarities. The functional behaviour of bar cells, which are found in the same areas of the visual cortex as grating cells, is less well explored and documented in the literature. In general, these cells respond to single bars and their responses decrease when further bars are added to form a periodic pattern. These properties of bar cells are implemented in a computational model in which the responses of bar cells are computed as thresholded differences of the responses of corresponding complex (or simple) cells and grating cells. Bar and grating cells seem to play complementary roles in resolving the ambiguity with which the responses of simple and complex cells represent oriented visual stimuli, in that bar cells are selective only for form information as present in contours and grating cells only respond to oriented texture information. The proposed model is capable of explaining the results of neurophysiological experiments as well as the psychophysical observation that the perception of texture and the perception of form are complementary processes. Received: 4 June 1996 / Accepted in revised form: 7 October 1996     

Crossover inhibition in the retina: circuitry that compensates for nonlinear rectifying synaptic transmission

In the mammalian retina, complementary ON and OFF visual streams are formed at the bipolar cell dendrites, then carried to amacrine and ganglion cells via nonlinear excitatory synapses from bipolar cells. Bipolar, amacrine and ganglion cells also receive a nonlinear inhibitory input from amacrine cells. The most common form of such inhibition crosses over from the opposite visual stream: Amacrine cells carry ON inhibition to the OFF cells and carry OFF inhibition to the ON cells (”crossover inhibition”). Although these synapses are predominantly nonlinear, linear signal processing is required for computing many properties of the visual world such as average intensity across a receptive field. Linear signaling is also necessary for maintaining the distinction between brightness and contrast. It has long been known that a subset of retinal outputs provide exactly this sort of linear representation of the world; we show here that rectifying (nonlinear) synaptic currents, when combined thorough crossover inhibition can generate this linear signaling. Using simple mathematical models we show that for a large set of cases, repeated rounds of synaptic rectification without crossover inhibition can destroy information carried by those synapses. A similar circuit motif is employed in the electronics industry to compensate for transistor nonlinearities in analog circuits.