Simple model for encoding natural images by retinal ganglion cells with nonlinear spatial integration

A central goal in sensory neuroscience is to understand the neuronal signal processing involved in the encoding of natural stimuli. A critical step towards this goal is the development of successful computational encoding models. For ganglion cells in the vertebrate retina, the development of satisfactory models for responses to natural visual scenes is an ongoing challenge. Standard models typically apply linear integration of visual stimuli over space, yet many ganglion cells are known to show nonlinear spatial integration, in particular when stimulated with contrast-reversing gratings. We here study the influence of spatial nonlinearities in the encoding of natural images by ganglion cells, using multielectrode-array recordings from isolated salamander and mouse retinas. We assess how responses to natural images depend on first- and second-order statistics of spatial patterns inside the receptive field. This leads us to a simple extension of current standard ganglion cell models. We show that taking not only the weighted average of light intensity inside the receptive field into account but also its variance over space can partly account for nonlinear integration and substantially improve response predictions of responses to novel images. For salamander ganglion cells, we find that response predictions for cell classes with large receptive fields profit most from including spatial contrast information. Finally, we demonstrate how this model framework can be used to assess the spatial scale of nonlinear integration. Our results underscore that nonlinear spatial stimulus integration translates to stimulation with natural images. Furthermore, the introduced model framework provides a simple, yet powerful extension of standard models and may serve as a benchmark for the development of more detailed models of the nonlinear structure of receptive fields.     

GABAergic neurotransmission and retinal ganglion cell function

Ganglion cells are the output retinal neurons that convey visual information to the brain. There are ~20 different types of ganglion cells, each encoding a specific aspect of the visual scene as spatial and temporal contrast, orientation, direction of movement, presence of looming stimuli; etc. Ganglion cell functioning depends on the intrinsic properties of ganglion cell’s membrane as well as on the excitatory and inhibitory inputs that these cells receive from other retinal neurons. GABA is one of the most abundant inhibitory neurotransmitters in the retina. How it modulates the activity of different types of ganglion cells and what is its significance in extracting the basic features from visual scene are questions with fundamental importance in visual neuroscience. The present review summarizes current data concerning the types of membrane receptors that mediate GABA action in proximal retina; the effects of GABA and its antagonists on the ganglion cell light-evoked postsynaptic potentials and spike discharges; the action of GABAergic agents on centre-surround organization of the receptive fields and feature related ganglion cell activity. Special emphasis is put on the GABA action regarding the ON–OFF and sustained–transient ganglion cell dichotomy in both nonmammalian and mammalian retina.     

Simultaneous Recording of Input and Output of Lateral Geniculate Neurones

TO understand the way in which the cat dorsal lateral geniculate nucleus (LGN) processes visual information it would be useful to know the number and type of retinal inputs to individual LGN neurones. Using electrical stimulation of the optic nerve Bishop et al.¹concluded that an impulse in a single optic nerve fibre is sufficient to excite a single LGN neurone. From the appearance of excitatory postsynaptic potentials (EPSPs) recorded essentially intracellularly, Creutzfeldt suggested that LGN neurones are driven by perhaps one² or a few³ retinal ganglion cells. Hubel and Wiesel⁴ proposed models of convergence of several retinal inputs on single LGN neurones based on analyses of receptive fields. Guillery⁵ produced anatomical evidence that some types of LGN neurones receive inputs from several different retinal fibres. Now we report direct observations which were made by recording simultaneously from single LGN neurones and from individual retinal ganglion cells which provided excitatory input to them. We shall not consider inhibitory influences, which are currently under study.     

Closed-loop measurements of iso-response stimuli reveal dynamic nonlinear stimulus integration in the retina

Neurons often integrate information from multiple parallel signaling streams. How a neuron combines these inputs largely determines its computational role in signal processing. Experimental assessment of neuronal signal integration, however, is often confounded by cell-intrinsic nonlinear processes that arise after signal integration has taken place. To overcome this problem and determine how ganglion cells in the salamander retina integrate visual contrast over space, we used automated online analysis of recorded spike trains and closed-loop control of the visual stimuli to identify different stimulus patterns that give the same neuronal response. These iso-response stimuli revealed a threshold-quadratic transformation as a fundamental nonlinearity within the receptive field center. Moreover, for a subset of ganglion cells, the method revealed an additional dynamic nonlinearity that renders these cells particularly sensitive to spatially homogeneous stimuli. This function is shown to arise from a local inhibition-mediated dynamic gain control mechanism.     

Initial processing of visual information within the retina and the LGN

The initial stage of information processing by the visual system reduces the information contained in the continuous image on the retina into a discrete set of responses which are carried from the lateral geniculate nucleus (LGN) to the visual cortex.-1. The optimal sampling of the light intensity distribution in the visual environment is achieved only if each channel in the visual pathways carries undistorted information corresponding to an image element. The visual system approaches as closely as possible the scheme of optimal spatial sampling, retaining the full information on the low spatial frequency content of the object light intensity. The ideal receptive field of a sustained LGN cell is then of the form J ₁ (Kr)/Kr.-2. The experimentally determined receptive fields of sustained LGN cells (and to some extent retinal ganglion cells as well) in cat closely resemble the functional form J ₁ (Kr)/Kr. The centre-surround organization of the receptive fields is therefore understood as a scheme which leads to a maximal information flow through the visual pathways.-3. The optimal sampling scheme cannot be realized by the retina alone, because of restrictions on the size of neural networks. It is therefore constructed in two stages, ending at the LGN level. A recombination of ganglion cell signals into optimal receptive fields is a major role of the LGN.     

Receptive field microstructure and dendritic geometry of retinal ganglion cells

We studied the fine spatial structure of the receptive fields of retinal ganglion cells and its relationship to the dendritic geometry of these cells. Cells from which recordings had been made were microinjected with Lucifer yellow, so that responses generated at precise locations within the receptive field center could be directly compared with that cell's dendritic structure. While many cells with small receptive fields had domeshaped sensitivity profiles, the majority of large receptive fields were composed of multiple regions of high sensitivity. The density of dendritic branches at any one location did not predict the regions of high sensitivity. Instead, the interactions between a ganglion cell's dendritic tree and the local mosaic of bipolar cell axons seem to define the fine structure of the receptive field center.     

A spatio-temporal model of ganglion cell receptive field in the cat retina

A spatio-temporal model of ganglion cell receptive fields is proposed on the basis of receptive field characteristics of cat retinal ganglion cells reported in our previous paper. The model consists of the linear and nonlinear mechanisms in the ganglion cell receptive field. The linear mechanism is assumed to be composed of antagonistic center and surround mechanisms. Then, by integrating these mechanisms we construct a spatio-temporal impulse response function of ganglion cell receptive field. Here we assume that spatio-temporal impulse response function may be factored into spatial and temporal terms. By Fouriertransforming the spatio-temporal impulse response function, we can obtain the spatio-temporal transfer function. Contrast sensitivity characteristics of X-and Y-cells in the cat retina may be explained by the transfer function.     

Crossover inhibition in the retina: circuitry that compensates for nonlinear rectifying synaptic transmission

In the mammalian retina, complementary ON and OFF visual streams are formed at the bipolar cell dendrites, then carried to amacrine and ganglion cells via nonlinear excitatory synapses from bipolar cells. Bipolar, amacrine and ganglion cells also receive a nonlinear inhibitory input from amacrine cells. The most common form of such inhibition crosses over from the opposite visual stream: Amacrine cells carry ON inhibition to the OFF cells and carry OFF inhibition to the ON cells (”crossover inhibition”). Although these synapses are predominantly nonlinear, linear signal processing is required for computing many properties of the visual world such as average intensity across a receptive field. Linear signaling is also necessary for maintaining the distinction between brightness and contrast. It has long been known that a subset of retinal outputs provide exactly this sort of linear representation of the world; we show here that rectifying (nonlinear) synaptic currents, when combined thorough crossover inhibition can generate this linear signaling. Using simple mathematical models we show that for a large set of cases, repeated rounds of synaptic rectification without crossover inhibition can destroy information carried by those synapses. A similar circuit motif is employed in the electronics industry to compensate for transistor nonlinearities in analog circuits.     

Can the data of Campbell and Robson be explained without assuming fourier analysis?

Certain experiments on the detection of low-contrast gratings, occasionally cited as evidence of Fourier analysis within the visual system, are interpreted without the assumption of Fourier analysis. Theoretical curves are obtained and compared with the published experimental points, showing mostly satisfactory agreement. The computations utilize Gaussian receptive fields (on-center and off-center) for the retinal ganglion cells, spatial summation, center-surround antagonism, quasilinear response at low contrasts (X-cells), and the assumption that the first significant convergence is primarily between cells of like response type and like receptive field geometry.     

Receptive field mechanisms of cat X and Y retinal ganglion cells

We investigated receptive field properties of cat retinal ganglion cells with visual stimuli which were sinusoidal spatial gratings amplitude modulated in time by a sum of sinusoids. Neural responses were analyzed into the Fourier components at the input frequencies and the components at sum and difference frequencies. The first-order frequency response of X cells had a marked spatial phase and spatial frequency dependence which could be explained in terms of linear interactions between center and surround mechanisms in the receptive field. The second-order frequency response of X cells was much smaller than the first-order frequency response at all spatial frequencies. The spatial phase and spatial frequency dependence of the first-order frequency response in Y cells in some ways resembled that of X cells. However, the Y first-order response declined to zero at a much lower spatial frequency than in X cells. Furthermore, the second-order frequency response was larger in Y cells; the second-order frequency components became the dominant part of the response for patterns of high spatial frequency. This implies that the receptive field center and surround mechanisms are physiologically quite different in Y cells from those in X cells, and that the Y cells also receive excitatory drive from an additional nonlinear receptive field mechanism.     

Differential targeting of optical neuromodulators to ganglion cell soma and dendrites allows dynamic control of center-surround antagonism

Retinal degenerative diseases cause photoreceptor loss and often result in remodeling and deafferentation of the inner retina. Fortunately, ganglion cell morphology appears to remain intact long after photoreceptors and distal retinal circuitry have degenerated. We have introduced the optical neuromodulators channelrhodopsin-2 (ChR2) and halorhodopsin (NpHR) differentially into the soma and dendrites of ganglion cells to recreate antagonistic center-surround receptive field interactions. We then reestablished the physiological receptive field dimensions of primate parafoveal ganglion cells by convolving Gaussian-blurred versions of the visual scene at the appropriate wavelength for each neuromodulator with the Gaussians inherent in the soma and dendrites. These Gaussian-modified ganglion cells responded with physiologically relevant antagonistic receptive field components and encoded edges with parafoveal resolution. This approach bypasses the degenerated areas of the distal retina and could provide a first step in restoring sight to individuals suffering from retinal disease.     

Temporal pattern discrimination within the receptive field of cat retinal ganglion cells

ON-center and OFF-center receptive fields of cat retinal ganglion cells can be divided into two categories: sensitive (type N) and insensitive (type L) to three statistical temporal visual stimuli with different second order statistics but identical first order statistics (Tsukada et al. 1982). The temporal pattern sensitivity of type N response is closely related to the nonlinear stage of Y cells depending on the interaction between center and surround mechanism. The temporal pattern sensitivity of type N responses has a spatial profile within the receptive field; it is highly sensitive in the center region of the receptive field and less sensitive toward the field periphery. The temporal pattern sensitivity in the center region of the receptive field to statistical properties (irregular or regular) of a surrounding flash annulus shows modulation like a switching element: when the surrounding area is stimulated by a more regular flash stimulus with normal distribution of inter-stimulus intervals the system is sensitive (switching on) to the temporal pattern, while a change to an irregular one with an exponential distribution makes it insensitive (switching off) to the temporal pattern.     

视网膜神经节细胞感受野的一种新模型 I.含大周边的感受野模型

感受野是视觉系统信息处理的基本结构和功能单元。Ｘ、Ｙ细胞是两类主要的视网膜神经节细胞。生理实验发现,在经典感受野之外还存在一个大范围的在周边去抑制区。文中采用周边去抑制区对经典外周的去抑制非线性使用方式,建立一个二维的与实验结果联系紧密的Ｘ、Ｙ细胞统一的复合感受野模型。该模型不仅能模拟Ｘ细胞的ｎｕｌｌ－ｔｅｓｔ反应和Ｙ细胞的ｏｎ－ｏｆｆ反应,还模拟了Ｙ细胞在低空频刺激时的信频反应、圆面积空间的倍频     

A possible explanation of the low-level brightness–contrast illusions in the light of an extended classical receptive field model of retinal ganglion cells

The low-level brightness–contrast illusions constitute a special class within visual illusions. Speculations exist that these illusions may be processed through the filtering action of the retinal ganglion cells without necessitating much intervention from higher order processes of visual perception. Concept of the classical receptive field of the ganglion cell, derived from early physiological studies, prompted the idea that a Difference of Gaussian (DoG) model might explain the low-level illusions. In spite of its many successes, the DoG model fails to explain some of these illusions. It has been shown in this paper that it is possible to simulate those illusions with a model that takes into cognizance the role of the extended classical receptive field     

A unified neural network model of spatiotemporal processing in X and Y retinal ganglion cells

This article makes use of a push-pull shunting network, which was introduced in the companion article, to model certain properties of X and Y retinal ganglion cells. Input to the push-pull network is preprocessed by a nonlinear mechanism for temporal adaptation, which is ascribed here to photoreceptor dynamics. The complete circuit is used to show that a simple change in receptive field morphology within a single model equation can change the network's response characteristics to closely resemble those of either X or Y cells. Specifically, an increase in width of the receptive field center mechanism is sufficient to account for generation of on-off (Y-like) instead of null (X-like) responses to modulated gratings. In agreement with experimental data, the Y cell on-off response is independent of spatial phase. Also, the model accurately predicts that on-off responses can be observed in X cells for particular stimulus configurations. Taken together, the results show how the retina combines individually inadequate modules to efficiently handle the tasks required for accurate spatial and temporal visual information processing. The model is also able to clarify a number of controversial experimental findings on the nature of spatiotemporal visual processing in the retina.     

Modeling convergent ON and OFF pathways in the early visual system

For understanding the computation and function of single neurons in sensory systems, one needs to investigate how sensory stimuli are related to a neuron’s response and which biological mechanisms underlie this relationship. Mathematical models of the stimulus–response relationship have proved very useful in approaching these issues in a systematic, quantitative way. A starting point for many such analyses has been provided by phenomenological “linear–nonlinear” (LN) models, which comprise a linear filter followed by a static nonlinear transformation. The linear filter is often associated with the neuron’s receptive field. However, the structure of the receptive field is generally a result of inputs from many presynaptic neurons, which may form parallel signal processing pathways. In the retina, for example, certain ganglion cells receive excitatory inputs from ON-type as well as OFF-type bipolar cells. Recent experiments have shown that the convergence of these pathways leads to intriguing response characteristics that cannot be captured by a single linear filter. One approach to adjust the LN model to the biological circuit structure is to use multiple parallel filters that capture ON and OFF bipolar inputs. Here, we review these new developments in modeling neuronal responses in the early visual system and provide details about one particular technique for obtaining the required sets of parallel filters from experimental data.     

Retinal ganglion cells can rapidly change polarity from Off to On

Retinal ganglion cells are commonly classified as On-center or Off-center depending on whether they are excited predominantly by brightening or dimming within the receptive field. Here we report that many ganglion cells in the salamander retina can switch from one response type to the other, depending on stimulus events far from the receptive field. Specifically, a shift of the peripheral image--as produced by a rapid eye movement--causes a brief transition in visual sensitivity from Off-type to On-type for approximately 100 ms. We show that these ganglion cells receive inputs from both On and Off bipolar cells, and the Off inputs are normally dominant. The peripheral shift strongly modulates the strength of these two inputs in opposite directions, facilitating the On pathway and suppressing the Off pathway. Furthermore, we identify certain wide-field amacrine cells that contribute to this modulation. Depolarizing such an amacrine cell affects nearby ganglion cells in the same way as the peripheral image shift, facilitating the On inputs and suppressing the Off inputs. This study illustrates how inhibitory interneurons can rapidly gate the flow of information within a circuit, dramatically altering the behavior of the principal neurons in the course of a computation.     

Sensitivity of a sensory process to short time delays: A study in pattern induced flicker colors (PIFCs)

Pattern induced flicker colors (PIFCs) were generated by means of a modified version of Benham's top, the stimulus pattern of which could be varied continuously during stimulation by the human subjects. The sensitivity of the color sensation to small phase shifts between the periodic stimuli on neighboring retinal areas was recorded under several conditions of stimulus parameters. A mathematical model was developed to describe the influence of the stimulus parameters on the recorded sensory effect. Concerning the underlying neurophysiological processes, a hypothesis is advanced according to which the phase sensitive lateral interaction within the retina changes the spatial excitation distribution within color coding receptive fields of the retinal ganglion cells. The resulting ganglion cell excitation is supposed to generate PIFCs.     

Development of spatiotemporal receptive fields of simple cells: I. Model formulation

A model for the development of spatiotemporal receptive fields of simple cells in the visual cortex is proposed. The model is based on the 1990 hypothesis of Saul and Humphrey that the convergence of four types of input onto a cortical cell, viz. non-lagged ON and OFF inputs and lagged ON and OFF inputs, underlies the spatial and temporal structure of the receptive fields. It therefore explains both orientation and direction selectivity of simple cells. The response properties of the four types of input are described by the product of linear spatial and temporal response functions. Extending the 1994 model of one of the authors (K.D. Miller), we describe the development of spatiotemporal receptive fields as a Hebbian learning process taking into account not only spatial but also temporal correlations between the different inputs. We derive the correlation functions that drive the development both for the period before and after eye-opening and demonstrate how the joint development of orientation and direction selectivity can be understood in the framework of correlation-based learning. Our investigation is split into two parts that are presented in two papers. In the first, the model for the response properties and for the development of direction-selective receptive fields is presented. In the second paper we present simulation results that are compared with experimental data, and also provide a first analysis of our model. Received: 18 June 1997 / Accepted: 16 September 1997