Optimal Stimulus Coding by Neural Populations Using Rate Codes

We create a framework based on Fisher information for determining the most effective population coding scheme for representing a continuous-valued stimulus attribute over its entire range. Using this scheme, we derive optimal single- and multi-neuron rate codes for homogeneous populations using several statistical models frequently used to describe neural data. We show that each neuron's discharge rate should increase quadratically with the stimulus and that statistically independent neural outputs provides optimal coding. Only cooperative populations can achieve this condition in an informationally effective way.     

The Sign Rule and Beyond: Boundary Effects,Flexibility, and Noise Correlations in Neural Population Codes

Over repeat presentations of the same stimulus, sensory neurons show variable responses. This “noise” is typically correlated between pairs of cells, and a question with rich history in neuroscience is how these noise correlations impact the population''s ability to encode the stimulus. Here, we consider a very general setting for population coding, investigating how information varies as a function of noise correlations, with all other aspects of the problem – neural tuning curves, etc. – held fixed. This work yields unifying insights into the role of noise correlations. These are summarized in the form of theorems, and illustrated with numerical examples involving neurons with diverse tuning curves. Our main contributions are as follows. (1) We generalize previous results to prove a sign rule (SR) — if noise correlations between pairs of neurons have opposite signs vs. their signal correlations, then coding performance will improve compared to the independent case. This holds for three different metrics of coding performance, and for arbitrary tuning curves and levels of heterogeneity. This generality is true for our other results as well. (2) As also pointed out in the literature, the SR does not provide a necessary condition for good coding. We show that a diverse set of correlation structures can improve coding. Many of these violate the SR, as do experimentally observed correlations. There is structure to this diversity: we prove that the optimal correlation structures must lie on boundaries of the possible set of noise correlations. (3) We provide a novel set of necessary and sufficient conditions, under which the coding performance (in the presence of noise) will be as good as it would be if there were no noise present at all.     

Olfactory coding in the insect brain: molecular receptive ranges, spatial and temporal coding

Odor information is coded in the insect brain in a sequence of steps, ranging from the receptor cells, via the neural network in the antennal lobe, to higher order brain centers, among which the mushroom bodies and the lateral horn are the most prominent. Across all of these processing steps, coding logic is combinatorial, in the sense that information is represented as patterns of activity across a population of neurons, rather than in individual neurons. Because different neurons are located in different places, such a coding logic is often termed spatial, and can be visualized with optical imaging techniques. We employ in vivo calcium imaging in order to record odor‐evoked activity patterns in olfactory receptor neurons, different populations of local neurons in the antennal lobes, projection neurons linking antennal lobes to the mushroom bodies, and the intrinsic cells of the mushroom bodies themselves, the Kenyon cells. These studies confirm the combinatorial nature of coding at all of these stages. However, the transmission of odor‐evoked activity patterns from projection neuron dendrites via their axon terminals onto Kenyon cells is accompanied by a progressive sparsening of the population code. Activity patterns also show characteristic temporal properties. While a part of the temporal response properties reflect the physical sequence of odor filaments, another part is generated by local neuron networks. In honeybees, γ‐aminobutyric acid (GABA)‐ergic and histaminergic neurons both contribute inhibitory networks to the antennal lobe. Interestingly, temporal properties differ markedly in different brain areas. In particular, in the antennal lobe odor‐evoked activity develops over slow time courses, while responses in Kenyon cells are phasic and transient. The termination of an odor stimulus is reflected by a decrease in activity within most glomeruli of the antennal lobe and an off‐response in some glomeruli, while in the mushroom bodies about half of the odor‐activated Kenyon cells also exhibit off‐responses.     

A Population Density Approach That Facilitates Large-Scale Modeling of Neural Networks: Analysis and an Application to Orientation Tuning

We explore a computationally efficient method of simulating realistic networks of neurons introduced by Knight, Manin, and Sirovich (1996) in which integrate-and-fire neurons are grouped into large populations of similar neurons. For each population, we form a probability density that represents the distribution of neurons over all possible states. The populations are coupled via stochastic synapses in which the conductance of a neuron is modulated according to the firing rates of its presynaptic populations. The evolution equation for each of these probability densities is a partial differential-integral equation, which we solve numerically. Results obtained for several example networks are tested against conventional computations for groups of individual neurons.We apply this approach to modeling orientation tuning in the visual cortex. Our population density model is based on the recurrent feedback model of a hypercolumn in cat visual cortex of Somers et al. (1995). We simulate the response to oriented flashed bars. As in the Somers model, a weak orientation bias provided by feed-forward lateral geniculate input is transformed by intracortical circuitry into sharper orientation tuning that is independent of stimulus contrast.The population density approach appears to be a viable method for simulating large neural networks. Its computational efficiency overcomes some of the restrictions imposed by computation time in individual neuron simulations, allowing one to build more complex networks and to explore parameter space more easily. The method produces smooth rate functions with one pass of the stimulus and does not require signal averaging. At the same time, this model captures the dynamics of single-neuron activity that are missed in simple firing-rate models.     

Neural population codes

In many regions of the brain, information is represented by patterns of activity occurring over populations of neurons. Understanding the encoding of information in neural population activity is important both for grasping the fundamental computations underlying brain function, and for interpreting signals that may be useful for the control of prosthetic devices. We concentrate on the representation of information in neurons with Poisson spike statistics, in which information is contained in the average spike firing rate. We analyze the properties of population codes in terms of the tuning functions that describe individual neuron behavior. The discussion centers on three computational questions: first, what information is encoded in a population; second, how does the brain compute using populations; and third, when is a population optimal? To answer these questions, we discuss several methods for decoding population activity in an experimental setting. We also discuss how computation can be performed within the brain in networks of interconnected populations. Finally, we examine questions of optimal design of population codes that may help to explain their particular form and the set of variables that are best represented. We show that for population codes based on neurons that have a Poisson distribution of spike probabilities, the behavior and computational properties of the code can be understood in terms of the tuning properties of individual cells.     

Information-Theoretic Analysis of Neural Coding

We describe an approach to analyzing single- and multiunit (ensemble) discharge patterns based on information-theoretic distance measures and on empirical theories derived from work in universal signal processing. In this approach, we quantify the difference between response patterns, whether time-varying or not, using information-theoretic distance measures. We apply these techniques to single- and multiple-unit processing of sound amplitude and sound location. These examples illustrate that neurons can simultaneously represent at least two kinds of information with different levels of fidelity. The fidelity can persist through a transient and a subsequent steady-state response, indicating that it is possible for an evolving neural code to represent information with constant fidelity.     

<Emphasis Type="BoldItalic">In vivo</Emphasis> conditions influence the coding of stimulus features by bursts of action potentials

The functional role of burst firing (i.e. the firing of packets of action potentials followed by quiescence) in sensory processing is still under debate. Should bursts be considered as unitary events that signal the presence of a particular feature in the sensory environment or is information about stimulus attributes contained within their temporal structure? We compared the coding of stimulus attributes by bursts in vivo and in vitro of electrosensory pyramidal neurons in weakly electric fish by computing correlations between burst and stimulus attributes. Our results show that, while these correlations were strong in magnitude and significant in vitro, they were actually much weaker in magnitude if at all significant in vivo. We used a mathematical model of pyramidal neuron activity in vivo and showed that such a model could reproduce the correlations seen in vitro, thereby suggesting that differences in burst coding were not due to differences in bursting seen in vivo and in vitro. We next tested whether variability in the baseline (i.e. without stimulation) activity of ELL pyramidal neurons could account for these differences. To do so, we injected noise into our model whose intensity was calibrated to mimic baseline activity variability as quantified by the coefficient of variation. We found that this noise caused significant decreases in the magnitude of correlations between burst and stimulus attributes and could account for differences between in vitro and in vivo conditions. We then tested this prediction experimentally by directly injecting noise in vitro through the recording electrode. Our results show that this caused a lowering in magnitude of the correlations between burst and stimulus attributes in vitro and gave rise to values that were quantitatively similar to those seen under in vivo conditions. While it is expected that noise in the form of baseline activity variability will lower correlations between burst and stimulus attributes, our results show that such variability can account for differences seen in vivo. Thus, the high variability seen under in vivo conditions has profound consequences on the coding of information by bursts in ELL pyramidal neurons. In particular, our results support the viewpoint that bursts serve as a detector of particular stimulus features but do not carry detailed information about such features in their structure.     

Neural coding of categories: information efficiency and optimal population codes

This paper deals with the analytical study of coding a discrete set of categories by a large assembly of neurons. We consider population coding schemes, which can also be seen as instances of exemplar models proposed in the literature to account for phenomena in the psychophysics of categorization. We quantify the coding efficiency by the mutual information between the set of categories and the neural code, and we characterize the properties of the most efficient codes, considering different regimes corresponding essentially to different signal-to-noise ratio. One main outcome is to find that, in a high signal-to-noise ratio limit, the Fisher information at the population level should be the greatest between categories, which is achieved by having many cells with the stimulus-discriminating parts (steepest slope) of their tuning curves placed in the transition regions between categories in stimulus space. We show that these properties are in good agreement with both psychophysical data and with the neurophysiology of the inferotemporal cortex in the monkey, a cortex area known to be specifically involved in classification tasks.     

Neural network model of the primary visual cortex: From functional architecture to lateral connectivity and back

The role of intrinsic cortical dynamics is a debatable issue. A recent optical imaging study (Kenet et al., 2003) found that activity patterns similar to orientation maps (OMs), emerge in the primary visual cortex (V1) even in the absence of sensory input, suggesting an intrinsic mechanism of OM activation. To better understand these results and shed light on the intrinsic V1 processing, we suggest a neural network model in which OMs are encoded by the intrinsic lateral connections. The proposed connectivity pattern depends on the preferred orientation and, unlike previous models, on the degree of orientation selectivity of the interconnected neurons. We prove that the network has a ring attractor composed of an approximated version of the OMs. Consequently, OMs emerge spontaneously when the network is presented with an unstructured noisy input. Simulations show that the model can be applied to experimental data and generate realistic OMs. We study a variation of the model with spatially restricted connections, and show that it gives rise to states composed of several OMs. We hypothesize that these states can represent local properties of the visual scene. Action Editor: Jonathan D. Victor     

Frequency-invariant representation of interaural time differences in mammals

Interaural time differences (ITDs) are the major cue for localizing low-frequency sounds. The activity of neuronal populations in the brainstem encodes ITDs with an exquisite temporal acuity of about 10 μs. The response of single neurons, however, also changes with other stimulus properties like the spectral composition of sound. The influence of stimulus frequency is very different across neurons and thus it is unclear how ITDs are encoded independently of stimulus frequency by populations of neurons. Here we fitted a statistical model to single-cell rate responses of the dorsal nucleus of the lateral lemniscus. The model was used to evaluate the impact of single-cell response characteristics on the frequency-invariant mutual information between rate response and ITD. We found a rough correspondence between the measured cell characteristics and those predicted by computing mutual information. Furthermore, we studied two readout mechanisms, a linear classifier and a two-channel rate difference decoder. The latter turned out to be better suited to decode the population patterns obtained from the fitted model.     

Dynamical coding of sensory information with competitive networks.

Based on experiments with the locust olfactory system, we demonstrate that model sensory neural networks with lateral inhibition can generate stimulus specific identity-temporal patterns in the form of stimulus-dependent switching among small and dynamically changing neural ensembles (each ensemble being a group of synchronized projection neurons). Networks produce this switching mode of dynamical activity when lateral inhibitory connections are strongly non-symmetric. Such coding uses 'winner-less competitive' (WLC) dynamics. In contrast to the well known winner-take-all competitive (WTA) networks and Hopfield nets, winner-less competition represents sensory information dynamically. Such dynamics are reproducible, robust against intrinsic noise and sensitive to changes in the sensory input. We demonstrate the validity of sensory coding with WLC networks using two different formulations of the dynamics, namely the average and spiking dynamics of projection neurons (PN).     

Degree Correlations Optimize Neuronal Network Sensitivity to Sub-Threshold Stimuli

Information processing in the brain crucially depends on the topology of the neuronal connections. We investigate how the topology influences the response of a population of leaky integrate-and-fire neurons to a stimulus. We devise a method to calculate firing rates from a self-consistent system of equations taking into account the degree distribution and degree correlations in the network. We show that assortative degree correlations strongly improve the sensitivity for weak stimuli and propose that such networks possess an advantage in signal processing. We moreover find that there exists an optimum in assortativity at an intermediate level leading to a maximum in input/output mutual information.     

Functional implications of temporal structure in primate cortical information processing

Processing of information in the cerebral cortex of primates is characterized by distributed representations and processing in neuronal assemblies rather than by detector neurons, cardinal cells or command neurons. Responses of individual neurons in sensory cortical areas contain limited and ambiguous information on common features of the natural environment which is disambiguated by comparison with the responses of other, related neurons. Distributed representations are also capable to represent the enormous complexity and variability of the natural environment by the large number of possible combinations of neurons that can engage in the representation of a stimulus or other content. A critical problem of distributed representation and processing is the superposition of several assemblies activated at the same time since interpretation and processing of a population code requires that the responses related to a single representation can be identified and distinguished from other, related activity. A possible mechanism which tags related responses is the synchronization of neuronal responses of the same assembly with a precision in the millisecond range. This mechanism also supports the separate processing of distributed activity and dynamic assembly formation. Experimental evidence from electrophysiological investigations of non-human primates and human subjects shows that synchronous activity can be found in visual, auditory and motor areas of the cortex. Simultaneous recordings of neurons in the visual cortex indicate that individual neurons synchronize their activity with each other, if they respond to the same stimulus but not if they are part of different assemblies representing different contents. Furthermore, evidence for synchronous activity related to perception, expectation, memory, and attention has been observed.     

Different Stimuli,Different Spatial Codes: A Visual Map and an Auditory Rate Code for Oculomotor Space in the Primate Superior Colliculus

Maps are a mainstay of visual, somatosensory, and motor coding in many species. However, auditory maps of space have not been reported in the primate brain. Instead, recent studies have suggested that sound location may be encoded via broadly responsive neurons whose firing rates vary roughly proportionately with sound azimuth. Within frontal space, maps and such rate codes involve different response patterns at the level of individual neurons. Maps consist of neurons exhibiting circumscribed receptive fields, whereas rate codes involve open-ended response patterns that peak in the periphery. This coding format discrepancy therefore poses a potential problem for brain regions responsible for representing both visual and auditory information. Here, we investigated the coding of auditory space in the primate superior colliculus(SC), a structure known to contain visual and oculomotor maps for guiding saccades. We report that, for visual stimuli, neurons showed circumscribed receptive fields consistent with a map, but for auditory stimuli, they had open-ended response patterns consistent with a rate or level-of-activity code for location. The discrepant response patterns were not segregated into different neural populations but occurred in the same neurons. We show that a read-out algorithm in which the site and level of SC activity both contribute to the computation of stimulus location is successful at evaluating the discrepant visual and auditory codes, and can account for subtle but systematic differences in the accuracy of auditory compared to visual saccades. This suggests that a given population of neurons can use different codes to support appropriate multimodal behavior.     

Neural coding properties based on spike timing and pattern correlation of retinal ganglion cells

Correlation between spike trains or neurons sometimes indicates certain neural coding rules in the visual system. In this paper, the relationship between spike timing correlation and pattern correlation is discussed, and their ability to represent stimulus features is compared to examine their coding strategies not only in individual neurons but also in population. Two kinds of stimuli, natural movies and checkerboard, are used to arouse firing activities in chicken retinal ganglion cells. The spike timing correlation and pattern correlation are calculated by cross-correlation function and Lempel–Ziv distance respectively. According to the correlation values, it is demonstrated that spike trains with similar spike patterns are not necessarily concerted in firing time. Moreover, spike pattern correlation values between individual neurons’ responses reflect the difference of natural movies and checkerboard; neurons cooperate with each other with higher pattern correlation values which represent spatiotemporal correlations during response to natural movies. Spike timing does not reflect stimulus features as obvious as spike patterns, caused by their particular coding properties or physiological foundation. As a result, separating the pattern correlation out of traditional timing correlation concept uncover additional insight in neural coding.     

Neural Coding with Graded Membrane Potential Changes and Spikes

The neural encoding of sensory stimuli is usually investigated for spike responses, although many neurons are known to convey information by graded membrane potential changes. We compare by model simulations how well different dynamical stimuli can be discriminated on the basis of spiking or graded responses. Although a continuously varying membrane potential contains more information than binary spike trains, we find situations where different stimuli can be better discriminated on the basis of spike responses than on the basis of graded responses. Spikes can be superior to graded membrane potential fluctuations if spikes sharpen the temporal structure of neuronal responses by amplifying fast transients of the membrane potential. Such fast membrane potential changes can be induced deterministically by the stimulus or can be due to membrane potential noise that is influenced in its statistical properties by the stimulus. The graded response mode is superior for discrimination between stimuli on a fine time scale.     

Differential parallel processing of olfactory information in the honeybee, Apis mellifera L.

Two distinct neuronal pathways connect the first olfactory neuropil, the antennal lobe, with higher integration areas, such as the mushroom bodies, via antennal lobe projection neurons. Intracellular recordings were used to address the question whether neuroanatomical features affect odor-coding properties. We found that neurons in the median antennocerebral tract code odors by latency differences or specific inhibitory phases in combination with excitatory phases, have a more specific activity profile for different odors and convey the information with a delay. The neurons of the lateral antennocerebral tract code odors by spike rate differences, have a broader activity profile for different odors, and convey the information quickly. Thus, rather preliminary information about the olfactory stimulus first reaches the mushroom bodies and the lateral horn via neurons of the lateral antennocerebral tract and subsequently odor information becomes more specified by activities of neurons of the median antennocerebral tract. We conclude that this neuroanatomical feature is not related to the distinction between different odors, but rather reflects a dual coding of the same odor stimuli by two different neuronal strategies focusing different properties of the same stimulus.     

Neural coding of gustatory information.

The nervous system encodes information relating chemical stimuli to taste perception, beginning with transduction mechanisms at the receptor and ending in the representation of stimulus attributes by the activity of neurons in the brain. Recent studies have rekindled the long-standing debate about whether taste information is coded by the pattern of activity across afferent neurons or by specifically tuned 'labeled lines'. Taste neurons are broadly tuned to stimuli representing different qualities and are also responsive to stimulus intensity and often to touch and temperature. Their responsiveness is also modulated by a number of physiological factors. In addition to representing stimulus quality and intensity, activity in taste neurons must code information about the hedonic value of gustatory stimuli. These considerations suggest that individual gustatory neurons contribute to the coding of more than one stimulus parameter, making the response of any one cell meaningful only in the context of the activity of its neighbors.     

Stimulus-dependent Maximum Entropy Models of Neural Population Codes

Neural populations encode information about their stimulus in a collective fashion, by joint activity patterns of spiking and silence. A full account of this mapping from stimulus to neural activity is given by the conditional probability distribution over neural codewords given the sensory input. For large populations, direct sampling of these distributions is impossible, and so we must rely on constructing appropriate models. We show here that in a population of 100 retinal ganglion cells in the salamander retina responding to temporal white-noise stimuli, dependencies between cells play an important encoding role. We introduce the stimulus-dependent maximum entropy (SDME) model—a minimal extension of the canonical linear-nonlinear model of a single neuron, to a pairwise-coupled neural population. We find that the SDME model gives a more accurate account of single cell responses and in particular significantly outperforms uncoupled models in reproducing the distributions of population codewords emitted in response to a stimulus. We show how the SDME model, in conjunction with static maximum entropy models of population vocabulary, can be used to estimate information-theoretic quantities like average surprise and information transmission in a neural population.     

Neural ensembles in navigation: From single cells to population codes

The brain can represent behaviorally relevant information through the firing of individual neurons as well as the coordinated firing of ensembles of neurons. Neurons in the hippocampus and associated cortical regions participate in a variety of types of ensembles to support navigation. These ensemble types include single cell codes, population codes, time-compressed sequences, behavioral sequences, and engrams. We present the physiological basis and behavioral relevance of ensemble firing. We discuss how these traditional definitions of ensembles can constrain or expand potential analyses due to the underlying assumptions and abstractions made. We highlight how coding can change at the ensemble level while underlying single cell codes remain intact. Finally, we present how ensemble definitions could be broadened to better understand the full complexity of the brain.