Features and functions of nonlinear spatial integration by retinal ganglion cells

Ganglion cells in the vertebrate retina integrate visual information over their receptive fields. They do so by pooling presynaptic excitatory inputs from typically many bipolar cells, which themselves collect inputs from several photoreceptors. In addition, inhibitory interactions mediated by horizontal cells and amacrine cells modulate the structure of the receptive field. In many models, this spatial integration is assumed to occur in a linear fashion. Yet, it has long been known that spatial integration by retinal ganglion cells also incurs nonlinear phenomena. Moreover, several recent examples have shown that nonlinear spatial integration is tightly connected to specific visual functions performed by different types of retinal ganglion cells. This work discusses these advances in understanding the role of nonlinear spatial integration and reviews recent efforts to quantitatively study the nature and mechanisms underlying spatial nonlinearities. These new insights point towards a critical role of nonlinearities within ganglion cell receptive fields for capturing responses of the cells to natural and behaviorally relevant visual stimuli. In the long run, nonlinear phenomena of spatial integration may also prove important for implementing the actual neural code of retinal neurons when designing visual prostheses for the eye.     

Simple model for encoding natural images by retinal ganglion cells with nonlinear spatial integration

A central goal in sensory neuroscience is to understand the neuronal signal processing involved in the encoding of natural stimuli. A critical step towards this goal is the development of successful computational encoding models. For ganglion cells in the vertebrate retina, the development of satisfactory models for responses to natural visual scenes is an ongoing challenge. Standard models typically apply linear integration of visual stimuli over space, yet many ganglion cells are known to show nonlinear spatial integration, in particular when stimulated with contrast-reversing gratings. We here study the influence of spatial nonlinearities in the encoding of natural images by ganglion cells, using multielectrode-array recordings from isolated salamander and mouse retinas. We assess how responses to natural images depend on first- and second-order statistics of spatial patterns inside the receptive field. This leads us to a simple extension of current standard ganglion cell models. We show that taking not only the weighted average of light intensity inside the receptive field into account but also its variance over space can partly account for nonlinear integration and substantially improve response predictions of responses to novel images. For salamander ganglion cells, we find that response predictions for cell classes with large receptive fields profit most from including spatial contrast information. Finally, we demonstrate how this model framework can be used to assess the spatial scale of nonlinear integration. Our results underscore that nonlinear spatial stimulus integration translates to stimulation with natural images. Furthermore, the introduced model framework provides a simple, yet powerful extension of standard models and may serve as a benchmark for the development of more detailed models of the nonlinear structure of receptive fields.     

Weighted cue integration in the rodent head direction system

How the brain combines information from different sensory modalities and of differing reliability is an important and still-unanswered question. Using the head direction (HD) system as a model, we explored the resolution of conflicts between landmarks and background cues. Sensory cue integration models predict averaging of the two cues, whereas attractor models predict capture of the signal by the dominant cue. We found that a visual landmark mostly captured the HD signal at low conflicts: however, there was an increasing propensity for the cells to integrate the cues thereafter. A large conflict presented to naive rats resulted in greater visual cue capture (less integration) than in experienced rats, revealing an effect of experience. We propose that weighted cue integration in HD cells arises from dynamic plasticity of the feed-forward inputs to the network, causing within-trial spatial redistribution of the visual inputs onto the ring. This suggests that an attractor network can implement decision processes about cue reliability using simple architecture and learning rules, thus providing a potential neural substrate for weighted cue integration.     

Modeling convergent ON and OFF pathways in the early visual system

For understanding the computation and function of single neurons in sensory systems, one needs to investigate how sensory stimuli are related to a neuron’s response and which biological mechanisms underlie this relationship. Mathematical models of the stimulus–response relationship have proved very useful in approaching these issues in a systematic, quantitative way. A starting point for many such analyses has been provided by phenomenological “linear–nonlinear” (LN) models, which comprise a linear filter followed by a static nonlinear transformation. The linear filter is often associated with the neuron’s receptive field. However, the structure of the receptive field is generally a result of inputs from many presynaptic neurons, which may form parallel signal processing pathways. In the retina, for example, certain ganglion cells receive excitatory inputs from ON-type as well as OFF-type bipolar cells. Recent experiments have shown that the convergence of these pathways leads to intriguing response characteristics that cannot be captured by a single linear filter. One approach to adjust the LN model to the biological circuit structure is to use multiple parallel filters that capture ON and OFF bipolar inputs. Here, we review these new developments in modeling neuronal responses in the early visual system and provide details about one particular technique for obtaining the required sets of parallel filters from experimental data.     

Transformation of Stimulus Correlations by the Retina

Redundancies and correlations in the responses of sensory neurons may seem to waste neural resources, but they can also carry cues about structured stimuli and may help the brain to correct for response errors. To investigate the effect of stimulus structure on redundancy in retina, we measured simultaneous responses from populations of retinal ganglion cells presented with natural and artificial stimuli that varied greatly in correlation structure; these stimuli and recordings are publicly available online. Responding to spatio-temporally structured stimuli such as natural movies, pairs of ganglion cells were modestly more correlated than in response to white noise checkerboards, but they were much less correlated than predicted by a non-adapting functional model of retinal response. Meanwhile, responding to stimuli with purely spatial correlations, pairs of ganglion cells showed increased correlations consistent with a static, non-adapting receptive field and nonlinearity. We found that in response to spatio-temporally correlated stimuli, ganglion cells had faster temporal kernels and tended to have stronger surrounds. These properties of individual cells, along with gain changes that opposed changes in effective contrast at the ganglion cell input, largely explained the pattern of pairwise correlations across stimuli where receptive field measurements were possible.     

Contrast gain control in the lower vertebrate retinas [published erratum appears in J Gen Physiol 1995 Aug;106(2):following 388]

Control of contrast sensitivity was studied in two kinds of retina, that of the channel catfish and that of the kissing gourami. The former preparation is dominantly monochromatic and the latter is bichromatic. Various stimuli were used, namely a large field of light, a spot- annulus configuration and two overlapping stimuli of red and green. Recordings were made from horizontal, amacrine, and ganglion cells and the results were analyzed by means of Wiener's theory, in which the kernels are the contrast (incremental) sensitivity. Modulation responses from horizontal cells are linear, in that the waveform and amplitude of the first-order kernels are independent of the depth of modulation. In the N (sustained) amacrine and ganglion cells, contrast sensitivity was low for a large modulation input and was high for a small modulation input, providing an example of contrast gain control. In most of the cells, the contrast gain control did not affect the dynamics of the response because the waveform of the first-order kernels remained unchanged when the contrast sensitivity increased more than fivefold. The signature of the second-order kernels also remained unchanged over a wide range of modulation. The increase in the contrast sensitivity for the second-order component, as defined by the amplitude of the kernels, was much larger than for the first-order component. This observation suggests that the contrast gain control proceeded the generation of the second-order nonlinearity. An analysis of a cascade of the Wiener type shows that the control of contrast sensitivity in the proximal retinal cells could be modeled by assuming the presence of a simple (static) saturation nonlinearity. Such a nonlinearity must exist somewhere between the horizontal cells and the amacrine cells. The functional implications of the contrast gain control are as follows: (a) neurons in the proximal retina exhibit greater sensitivity to input of lower contrast; (b) saturation of a neuronal response can be prevented because of the lower sensitivity for an input with large contrast, and (c) over a large range of modulation depths, the amplitude of the response remains approximately constant.     

Visual pattern discrimination by population retinal ganglion cells’ activities during natural movie stimulation

In the visual system, neurons often fire in synchrony, and it is believed that synchronous activities of group neurons are more efficient than single cell response in transmitting neural signals to down-stream neurons. However, whether dynamic natural stimuli are encoded by dynamic spatiotemporal firing patterns of synchronous group neurons still needs to be investigated. In this paper we recorded the activities of population ganglion cells in bullfrog retina in response to time-varying natural images (natural scene movie) using multi-electrode arrays. In response to some different brief section pairs of the movie, synchronous groups of retinal ganglion cells (RGCs) fired with similar but different spike events. We attempted to discriminate the movie sections based on temporal firing patterns of single cells and spatiotemporal firing patterns of the synchronous groups of RGCs characterized by a measurement of subsequence distribution discrepancy. The discrimination performance was assessed by a classification method based on Support Vector Machines. Our results show that different movie sections of the natural movie elicited reliable dynamic spatiotemporal activity patterns of the synchronous RGCs, which are more efficient in discriminating different movie sections than the temporal patterns of the single cells’ spike events. These results suggest that, during natural vision, the down-stream neurons may decode the visual information from the dynamic spatiotemporal patterns of the synchronous group of RGCs’ activities.     

Long-Term Stability of Visual Pattern Selective Responses of Monkey Temporal Lobe Neurons

Many neurons in primate inferotemporal (IT) cortex respond selectively to complex, often meaningful, stimuli such as faces and objects. An important unanswered question is whether such response selectivity, which is thought to arise from experience-dependent plasticity, is maintained from day to day, or whether the roles of individual cells are continually reassigned based on the diet of natural vision. We addressed this question using microwire electrodes that were chronically implanted in the temporal lobe of two monkeys, often allowing us to monitor activity of individual neurons across days. We found that neurons maintained their selectivity in both response magnitude and patterns of spike timing across a large set of visual images throughout periods of stable signal isolation from the same cell that sometimes exceeded two weeks. These results indicate that stimulus-selectivity of responses in IT is stable across days and weeks of visual experience.     

VEGF activates divergent intracellular signaling components to regulate retinal progenitor cell proliferation and neuronal differentiation

During vertebrate neurogenesis, multiple extracellular signals influence progenitor cell fate choices. The process by which uncommitted progenitor cells interpret and integrate signals is not well understood. We demonstrate here that in the avascular chicken retina, vascular endothelial growth factor (VEGF) secreted by postmitotic neurons acts through the FLK1 receptor present on progenitor cells to influence cell proliferation and commitment. Augmenting VEGF signals increases progenitor cell proliferation and decreases retinal ganglion cell genesis. Conversely, absorbing endogenous VEGF ligand or disrupting FLK1 activity attenuates cell proliferation and enhances retinal ganglion cell production. In addition, we provide evidence that VEGF signals transmitted by the FLK1 receptor activate divergent intracellular signaling components, which regulate different responses of progenitor cells. VEGF-induced proliferation is influenced by the MEK-ERK pathway, as well as by the basic helix-loop-helix factor HES1. By contrast, VEGF-dependent ganglion cell suppression does not require MEK-ERK activation, but instead relies on VEGF-stimulated HES1 activity, which is independent of NOTCH signaling. Moreover, elevated HES1 expression promotes progenitor cell proliferation and prevents overproduction of retinal ganglion cells owing to the loss of VEGF or sonic hedgehog (SHH), another signal that suppresses ganglion cell development. Based on previous and current findings, we propose that HES1 serves as a convergent signaling node within early retinal progenitor cells to integrate various cell-extrinsic cues, including VEGF and SHH, in order to control cell proliferation and neuronal specification.     

Firing coincidences between neighboring retinal ganglion cells: inside information or epiphenomenon?

Retinal ganglion cells often fire impulses in synchrony; is this synchronization an irrelevant by-product of processing shared inputs, or does it encode information? We examined the rate of occurrence of coincident impulses from pairs of ganglion cells responding to stimuli that varied along several dimensions. We find that coincidences convey little if any additional information about simple static stimuli beyond what could be determined from the firing rates of the two cells considered separately. In fact, at least one of the separate cells generally provided a better information channel than the coincidence rate, implying that under these conditions ganglion cells do not employ a strategy of encoding by coincidences.     

The vestibular system implements a linear-nonlinear transformation in order to encode self-motion

Although it is well established that the neural code representing the world changes at each stage of a sensory pathway, the transformations that mediate these changes are not well understood. Here we show that self-motion (i.e. vestibular) sensory information encoded by VIIIth nerve afferents is integrated nonlinearly by post-synaptic central vestibular neurons. This response nonlinearity was characterized by a strong (～50%) attenuation in neuronal sensitivity to low frequency stimuli when presented concurrently with high frequency stimuli. Using computational methods, we further demonstrate that a static boosting nonlinearity in the input-output relationship of central vestibular neurons accounts for this unexpected result. Specifically, when low and high frequency stimuli are presented concurrently, this boosting nonlinearity causes an intensity-dependent bias in the output firing rate, thereby attenuating neuronal sensitivities. We suggest that nonlinear integration of afferent input extends the coding range of central vestibular neurons and enables them to better extract the high frequency features of self-motion when embedded with low frequency motion during natural movements. These findings challenge the traditional notion that the vestibular system uses a linear rate code to transmit information and have important consequences for understanding how the representation of sensory information changes across sensory pathways.     

How signaling proteins integrate multiple inputs: a comparison of N-WASP and Cdk2

Signal transduction proteins that can integrate multiple upstream signals play a critical role in the complex regulatory circuits that control cellular behavior. The two signaling node proteins cyclin-dependent kinase 2 and the actin regulator neuronal Wiskott-Aldrich syndrome protein have qualitatively similar signaling properties. Recent studies, however, reveal that these proteins utilize distinct mechanisms of signal integration, leading to subtle but important quantitative differences in behavior.     

Computing Complex Visual Features with Retinal Spike Times

Neurons in sensory systems can represent information not only by their firing rate, but also by the precise timing of individual spikes. For example, certain retinal ganglion cells, first identified in the salamander, encode the spatial structure of a new image by their first-spike latencies. Here we explore how this temporal code can be used by downstream neural circuits for computing complex features of the image that are not available from the signals of individual ganglion cells. To this end, we feed the experimentally observed spike trains from a population of retinal ganglion cells to an integrate-and-fire model of post-synaptic integration. The synaptic weights of this integration are tuned according to the recently introduced tempotron learning rule. We find that this model neuron can perform complex visual detection tasks in a single synaptic stage that would require multiple stages for neurons operating instead on neural spike counts. Furthermore, the model computes rapidly, using only a single spike per afferent, and can signal its decision in turn by just a single spike. Extending these analyses to large ensembles of simulated retinal signals, we show that the model can detect the orientation of a visual pattern independent of its phase, an operation thought to be one of the primitives in early visual processing. We analyze how these computations work and compare the performance of this model to other schemes for reading out spike-timing information. These results demonstrate that the retina formats spatial information into temporal spike sequences in a way that favors computation in the time domain. Moreover, complex image analysis can be achieved already by a simple integrate-and-fire model neuron, emphasizing the power and plausibility of rapid neural computing with spike times.     

GABAergic neurotransmission and retinal ganglion cell function

Ganglion cells are the output retinal neurons that convey visual information to the brain. There are ~20 different types of ganglion cells, each encoding a specific aspect of the visual scene as spatial and temporal contrast, orientation, direction of movement, presence of looming stimuli; etc. Ganglion cell functioning depends on the intrinsic properties of ganglion cell’s membrane as well as on the excitatory and inhibitory inputs that these cells receive from other retinal neurons. GABA is one of the most abundant inhibitory neurotransmitters in the retina. How it modulates the activity of different types of ganglion cells and what is its significance in extracting the basic features from visual scene are questions with fundamental importance in visual neuroscience. The present review summarizes current data concerning the types of membrane receptors that mediate GABA action in proximal retina; the effects of GABA and its antagonists on the ganglion cell light-evoked postsynaptic potentials and spike discharges; the action of GABAergic agents on centre-surround organization of the receptive fields and feature related ganglion cell activity. Special emphasis is put on the GABA action regarding the ON–OFF and sustained–transient ganglion cell dichotomy in both nonmammalian and mammalian retina.     

Involvement of Right STS in Audio-Visual Integration for Affective Speech Demonstrated Using MEG

Speech and emotion perception are dynamic processes in which it may be optimal to integrate synchronous signals emitted from different sources. Studies of audio-visual (AV) perception of neutrally expressed speech demonstrate supra-additive (i.e., where AV>[unimodal auditory+unimodal visual]) responses in left STS to crossmodal speech stimuli. However, emotions are often conveyed simultaneously with speech; through the voice in the form of speech prosody and through the face in the form of facial expression. Previous studies of AV nonverbal emotion integration showed a role for right (rather than left) STS. The current study therefore examined whether the integration of facial and prosodic signals of emotional speech is associated with supra-additive responses in left (cf. results for speech integration) or right (due to emotional content) STS. As emotional displays are sometimes difficult to interpret, we also examined whether supra-additive responses were affected by emotional incongruence (i.e., ambiguity). Using magnetoencephalography, we continuously recorded eighteen participants as they viewed and heard AV congruent emotional and AV incongruent emotional speech stimuli. Significant supra-additive responses were observed in right STS within the first 250 ms for emotionally incongruent and emotionally congruent AV speech stimuli, which further underscores the role of right STS in processing crossmodal emotive signals.     

Crizotinib-Induced Abnormal Signal Processing in the Retina

Molecular target therapy for cancer is characterized by unique adverse effects that are not usually observed with cytotoxic chemotherapy. For example, the anaplastic lymphoma kinase (ALK)-tyrosine kinase inhibitor crizotinib causes characteristic visual disturbances, whereas such effects are rare when another ALK-tyrosine kinase inhibitor, alectinib, is used. To elucidate the mechanism responsible for these visual disturbances, the responses to light exhibited by retinal ganglion cells treated with these agents were evaluated using a C57BL6 mouse ex vivo model. Both crizotinib and alectinib changed the firing rate of ON and OFF type retinal ganglion cells. However, the ratio of alectinib-affected cells (15.7%) was significantly lower than that of crizotinib-affected cells (38.6%). Furthermore, these drugs changed the response properties to light stimuli of retinal ganglion cells in some of the affected cells, i.e., OFF cells responded to both ON and OFF stimuli, etc. Finally, the expressions of ALK (a target receptor of both crizotinib and alectinib) and of MET and ROS1 (additional target receptors of crizotinib) were observed at the mRNA level in the retina. Our findings suggest that these drugs might target retinal ganglion cells and that the potency of the drug actions on the light responses of retinal ganglion cells might be responsible for the difference in the frequencies of visual disturbances observed between patients treated with crizotinib and those treated with alectinib. The present experimental system might be useful for screening new molecular target agents prior to their use in clinical trials.     

Computational investigations into the origins of short-term biochemical memory in T cell activation

Recent studies have reported that T cells can integrate signals between interrupted encounters with Antigen Presenting Cells (APCs) in such a way that the process of signal integration exhibits a form of memory. Here, we carry out a computational study using a simple mathematical model of T cell activation to investigate the ramifications of interrupted T cell-APC contacts on signal integration. We consider several mechanisms of how signal integration at these time scales may be achieved and conclude that feedback control of immediate early gene products (IEGs) appears to be a highly plausible mechanism that allows for effective signal integration and cytokine production from multiple exposures to APCs. Analysis of these computer simulations provides an experimental roadmap involving several testable predictions.     

First-occupancy model for signal transfer in the brain

In rats and rabbits, optic nerve fibers are asymmetrically decussated. Evoked bioelectric signal/noise power ratio as a function of decussation ratio afferent to the lateral geniculate and thence to visual cortex conforms approximately to what one would expect if the visual system functioned as a parallel coherent detector. Brain mechanisms which can quantitatively account for these findings have not been previously proposed. A theoretical model is proposed to explain observed bioelectric input-output relationships in asymmetrically decussated visual systems and to suggest a general mechanism for signal transfer in the mammalian brain. The model is stochastic and is based upon earlier work concerning coherent neuronal activity. A specific quantitative implication of the model is that, on the average, three ganglion cells in the retina will drive each principal cell in the lateral geniculate. This prediction of the model is verified by recent experimental findings.     

Melanopsin-Derived Visual Responses under Light Adapted Conditions in the Mouse dLGN

A direct projection from melanopsin-expressing intrinsically photosensitive retinal ganglion cells (ipRGCs) reaches the primary visual thalamus (dorsal lateral geniculate nucleus; dLGN). The significance of this melanopsin input to the visual system is only recently being investigated. One unresolved question is the degree to which neurons in the dLGN could use melanopsin to track dynamic changes in light intensity under light adapted conditions. Here we set out to address this question. We were able to present full field steps visible only to melanopsin by switching between rod-isoluminant ‘yellow’ and ‘blue’ lights in a mouse lacking cone function (Cnga3^-/-). In the retina these stimuli elicited melanopsin-like responses from a subset of ganglion cells. When presented to anaesthetised mice, we found that ~25-30% of visually responsive neurones in the contralateral dLGN responded to these melanopsin-isolating steps with small increases in firing rate. Such responses could be elicited even with fairly modest increases in effective irradiance (32% Michelson contrast for melanopsin). These melanopsin-driven responses were apparent at bright backgrounds (corresponding to twilight-daylight conditions), but their threshold irradiance was strongly dependent upon prior light exposure when stimuli were superimposed on a spectrally neutral ramping background light. While both onset and offset latencies were long for melanopsin-derived responses compared to those evoked by rods, there was great variability in these parameters with some cells responding to melanopsin steps in <1 s. These data indicate that a subset of dLGN units can employ melanopsin signals to detect modest changes in irradiance under photopic conditions.