Information processing with population codes

Information is encoded in the brain by populations or clusters of cells, rather than by single cells. This encoding strategy is known as population coding. Here we review the standard use of population codes for encoding and decoding information, and consider how population codes can be used to support neural computations such as noise removal and nonlinear mapping. More radical ideas about how population codes may directly represent information about stimulus uncertainty are also discussed.     

Real time computation: zooming in on population codes

Information processing in nervous systems intricately combines computation at the neuronal and network levels. Many computations may be envisioned as sequences of signal processing steps along some pathway. How can information encoded by single cells be mapped onto network population codes, and how do different modules or layers in the computation synchronize their communication and computation? These fundamental questions are particularly severe when dealing with real time streams of inputs. Here we study this problem within the context of a minimal signal perception task. In particular, we encode neuronal information by externally applying a space- and time-localized stimulus to individual neurons within a network. We show that a pulse-coupled recurrent neural network can successfully handle this task in real time, and obeys three key requirements: (i) stimulus dependence, (ii) initial-conditions independence, and (iii) accessibility by a readout mechanism. In particular, we suggest that the network's overall level of activity can be used as a temporal cue for a robust readout mechanism. Within this framework, the network can rapidly map a local stimulus onto a population code that can then be reliably read out during some narrow but well defined window of time.     

Spike-based population coding and working memory

Compelling behavioral evidence suggests that humans can make optimal decisions despite the uncertainty inherent in perceptual or motor tasks. A key question in neuroscience is how populations of spiking neurons can implement such probabilistic computations. In this article, we develop a comprehensive framework for optimal, spike-based sensory integration and working memory in a dynamic environment. We propose that probability distributions are inferred spike-per-spike in recurrently connected networks of integrate-and-fire neurons. As a result, these networks can combine sensory cues optimally, track the state of a time-varying stimulus and memorize accumulated evidence over periods much longer than the time constant of single neurons. Importantly, we propose that population responses and persistent working memory states represent entire probability distributions and not only single stimulus values. These memories are reflected by sustained, asynchronous patterns of activity which make relevant information available to downstream neurons within their short time window of integration. Model neurons act as predictive encoders, only firing spikes which account for new information that has not yet been signaled. Thus, spike times signal deterministically a prediction error, contrary to rate codes in which spike times are considered to be random samples of an underlying firing rate. As a consequence of this coding scheme, a multitude of spike patterns can reliably encode the same information. This results in weakly correlated, Poisson-like spike trains that are sensitive to initial conditions but robust to even high levels of external neural noise. This spike train variability reproduces the one observed in cortical sensory spike trains, but cannot be equated to noise. On the contrary, it is a consequence of optimal spike-based inference. In contrast, we show that rate-based models perform poorly when implemented with stochastically spiking neurons.     

Variable synaptic strengths controls the firing rate distribution in feedforward neural networks

Heterogeneity of firing rate statistics is known to have severe consequences on neural coding. Recent experimental recordings in weakly electric fish indicate that the distribution-width of superficial pyramidal cell firing rates (trial- and time-averaged) in the electrosensory lateral line lobe (ELL) depends on the stimulus, and also that network inputs can mediate changes in the firing rate distribution across the population. We previously developed theoretical methods to understand how two attributes (synaptic and intrinsic heterogeneity) interact and alter the firing rate distribution in a population of integrate-and-fire neurons with random recurrent coupling. Inspired by our experimental data, we extend these theoretical results to a delayed feedforward spiking network that qualitatively capture the changes of firing rate heterogeneity observed in in-vivo recordings. We demonstrate how heterogeneous neural attributes alter firing rate heterogeneity, accounting for the effect with various sensory stimuli. The model predicts how the strength of the effective network connectivity is related to intrinsic heterogeneity in such delayed feedforward networks: the strength of the feedforward input is positively correlated with excitability (threshold value for spiking) when firing rate heterogeneity is low and is negatively correlated with excitability with high firing rate heterogeneity. We also show how our theory can be used to predict effective neural architecture. We demonstrate that neural attributes do not interact in a simple manner but rather in a complex stimulus-dependent fashion to control neural heterogeneity and discuss how it can ultimately shape population codes.     

Neural dynamics as sampling: a model for stochastic computation in recurrent networks of spiking neurons

The organization of computations in networks of spiking neurons in the brain is still largely unknown, in particular in view of the inherently stochastic features of their firing activity and the experimentally observed trial-to-trial variability of neural systems in the brain. In principle there exists a powerful computational framework for stochastic computations, probabilistic inference by sampling, which can explain a large number of macroscopic experimental data in neuroscience and cognitive science. But it has turned out to be surprisingly difficult to create a link between these abstract models for stochastic computations and more detailed models of the dynamics of networks of spiking neurons. Here we create such a link and show that under some conditions the stochastic firing activity of networks of spiking neurons can be interpreted as probabilistic inference via Markov chain Monte Carlo (MCMC) sampling. Since common methods for MCMC sampling in distributed systems, such as Gibbs sampling, are inconsistent with the dynamics of spiking neurons, we introduce a different approach based on non-reversible Markov chains that is able to reflect inherent temporal processes of spiking neuronal activity through a suitable choice of random variables. We propose a neural network model and show by a rigorous theoretical analysis that its neural activity implements MCMC sampling of a given distribution, both for the case of discrete and continuous time. This provides a step towards closing the gap between abstract functional models of cortical computation and more detailed models of networks of spiking neurons.     

Information theoretic approaches to understanding circuit function

The analysis of stimulus/response patterns using information theoretic approaches requires the full probability distribution of stimuli and response. Recent progress in using information-based tools to understand circuit function has advanced understanding of neural coding at the single cell and population level. In advances over traditional reverse correlation approaches, the determination of receptive fields using information as a metric has allowed novel insights into stimulus representation and transformation. The application of maximum entropy methods to population codes has opened a rich exploration of the internal structure of these codes, revealing stimulus-driven functional connectivity. We speculate about the prospects and limitations of information as a general tool for dissecting neural circuits and relating their structure and function.     

Optimal Stimulus Coding by Neural Populations Using Rate Codes

We create a framework based on Fisher information for determining the most effective population coding scheme for representing a continuous-valued stimulus attribute over its entire range. Using this scheme, we derive optimal single- and multi-neuron rate codes for homogeneous populations using several statistical models frequently used to describe neural data. We show that each neuron's discharge rate should increase quadratically with the stimulus and that statistically independent neural outputs provides optimal coding. Only cooperative populations can achieve this condition in an informationally effective way.     

Reading spike timing without a clock: intrinsic decoding of spike trains

The precise timing of action potentials of sensory neurons relative to the time of stimulus presentation carries substantial sensory information that is lost or degraded when these responses are summed over longer time windows. However, it is unclear whether and how downstream networks can access information in precise time-varying neural responses. Here, we review approaches to test the hypothesis that the activity of neural populations provides the temporal reference frames needed to decode temporal spike patterns. These approaches are based on comparing the single-trial stimulus discriminability obtained from neural codes defined with respect to network-intrinsic reference frames to the discriminability obtained from codes defined relative to the experimenter''s computer clock. Application of this formalism to auditory, visual and somatosensory data shows that information carried by millisecond-scale spike times can be decoded robustly even with little or no independent external knowledge of stimulus time. In cortex, key components of such intrinsic temporal reference frames include dedicated neural populations that signal stimulus onset with reliable and precise latencies, and low-frequency oscillations that can serve as reference for partitioning extended neuronal responses into informative spike patterns.     

Optimal nonlinear cue integration for sound localization

Integration of multiple sensory cues can improve performance in detection and estimation tasks. There is an open theoretical question of the conditions under which linear or nonlinear cue combination is Bayes-optimal. We demonstrate that a neural population decoded by a population vector requires nonlinear cue combination to approximate Bayesian inference. Specifically, if cues are conditionally independent, multiplicative cue combination is optimal for the population vector. The model was tested on neural and behavioral responses in the barn owl’s sound localization system where space-specific neurons owe their selectivity to multiplicative tuning to sound localization cues interaural phase (IPD) and level (ILD) differences. We found that IPD and ILD cues are approximately conditionally independent. As a result, the multiplicative combination selectivity to IPD and ILD of midbrain space-specific neurons permits a population vector to perform Bayesian cue combination. We further show that this model describes the owl’s localization behavior in azimuth and elevation. This work provides theoretical justification and experimental evidence supporting the optimality of nonlinear cue combination.     

Probabilistic inference in general graphical models through sampling in stochastic networks of spiking neurons

An important open problem of computational neuroscience is the generic organization of computations in networks of neurons in the brain. We show here through rigorous theoretical analysis that inherent stochastic features of spiking neurons, in combination with simple nonlinear computational operations in specific network motifs and dendritic arbors, enable networks of spiking neurons to carry out probabilistic inference through sampling in general graphical models. In particular, it enables them to carry out probabilistic inference in Bayesian networks with converging arrows ("explaining away") and with undirected loops, that occur in many real-world tasks. Ubiquitous stochastic features of networks of spiking neurons, such as trial-to-trial variability and spontaneous activity, are necessary ingredients of the underlying computational organization. We demonstrate through computer simulations that this approach can be scaled up to neural emulations of probabilistic inference in fairly large graphical models, yielding some of the most complex computations that have been carried out so far in networks of spiking neurons.     

Dynamics on the manifold: Identifying computational dynamical activity from neural population recordings

The question of how the collective activity of neural populations gives rise to complex behaviour is fundamental to neuroscience. At the core of this question lie considerations about how neural circuits can perform computations that enable sensory perception, decision making, and motor control. It is thought that such computations are implemented through the dynamical evolution of distributed activity in recurrent circuits. Thus, identifying dynamical structure in neural population activity is a key challenge towards a better understanding of neural computation. At the same time, interpreting this structure in light of the computation of interest is essential for linking the time-varying activity patterns of the neural population to ongoing computational processes. Here, we review methods that aim to quantify structure in neural population recordings through a dynamical system defined in a low-dimensional latent variable space. We discuss advantages and limitations of different modelling approaches and address future challenges for the field.     

Optimal computation with attractor networks

We investigate the ability of multi-dimensional attractor networks to perform reliable computations with noisy population codes. We show that such networks can perform computations as reliably as possible--meaning they can reach the Cramér-Rao bound--so long as the noise is small enough. "Small enough" depends on the properties of the noise, especially its correlational structure. For many correlational structures, noise in the range of what is observed in the cortex is sufficiently small that biologically plausible networks can compute optimally. We demonstrate that this result applies to computations that involve cues of varying reliability, such as the position of an object on the retina in bright versus dim light.     

To Burst or Not to Burst?

It is well known that some neurons tend to fire packets of action potentials followed by periods of quiescence (bursts) while others within the same stage of sensory processing fire in a tonic manner. However, the respective computational advantages of bursting and tonic neurons for encoding time varying signals largely remain a mystery. Weakly electric fish use cutaneous electroreceptors to convey information about sensory stimuli and it has been shown that some electroreceptors exhibit bursting dynamics while others do not. In this study, we compare the neural coding capabilities of tonically firing and bursting electroreceptor model neurons using information theoretic measures. We find that both bursting and tonically firing model neurons efficiently transmit information about the stimulus. However, the decoding mechanisms that must be used for each differ greatly: a non-linear decoder would be required to extract all the available information transmitted by the bursting model neuron whereas a linear one might suffice for the tonically firing model neuron. Further investigations using stimulus reconstruction techniques reveal that, unlike the tonically firing model neuron, the bursting model neuron does not encode the detailed time course of the stimulus. A novel measure of feature detection reveals that the bursting neuron signals certain stimulus features. Finally, we show that feature extraction and stimulus estimation are mutually exclusive computations occurring in bursting and tonically firing model neurons, respectively. Our results therefore suggest that stimulus estimation and feature extraction might be parallel computations in certain sensory systems rather than being sequential as has been previously proposed.     

Human preferences for symmetry: subjective experience, cognitive conflict and cortical brain activity

This study examines the links between human perceptions, cognitive biases and neural processing of symmetrical stimuli. While preferences for symmetry have largely been examined in the context of disorders such as obsessive-compulsive disorder and autism spectrum disorders, we examine various these phenomena in non-clinical subjects and suggest that such preferences are distributed throughout the typical population as part of our cognitive and neural architecture. In Experiment 1, 82 young adults reported on the frequency of their obsessive-compulsive spectrum behaviors. Subjects also performed an emotional Stroop or variant of an Implicit Association Task (the OC-CIT) developed to assess cognitive biases for symmetry. Data not only reveal that subjects evidence a cognitive conflict when asked to match images of positive affect with asymmetrical stimuli, and disgust with symmetry, but also that their slowed reaction times when asked to do so were predicted by reports of OC behavior, particularly checking behavior. In Experiment 2, 26 participants were administered an oddball Event-Related Potential task specifically designed to assess sensitivity to symmetry as well as the OC-CIT. These data revealed that reaction times on the OC-CIT were strongly predicted by frontal electrode sites indicating faster processing of an asymmetrical stimulus (unparallel lines) relative to a symmetrical stimulus (parallel lines). The results point to an overall cognitive bias linking disgust with asymmetry and suggest that such cognitive biases are reflected in neural responses to symmetrical/asymmetrical stimuli.     

Encoding and processing biologically relevant temporal information in electrosensory systems

Wave-type weakly electric fish are specialists in time-domain processing: behaviors in these animals are often tightly correlated with the temporal structure of electrosensory signals. Behavioral responses in these fish can be dependent on differences in the temporal structure of electrosensory signals alone. This feature has facilitated the study of temporal codes and processing in central nervous system circuits of these animals. The temporal encoding and mechanisms used to transform temporal codes in the brain have been identified and characterized in several species, including South American gymnotid species and in the African mormyrid genus Gymnarchus. These distantly related groups use similar strategies for neural computations of information on the order of microseconds, milliseconds, and seconds. Here, we describe a suite of mechanisms for behaviorally relevant computations of temporal information that have been elucidated in these systems. These results show the critical role that behavioral experiments continue to have in the study of the neural control of behavior and its evolution.     

Modulation of the dynamical state in cortical network models

Cortical neural responses can be modulated by various factors, such as stimulus inputs and the behavior state of the animal. Understanding the circuit mechanisms underlying modulations of network dynamics is important to understand the flexibility of circuit computations. Identifying the dynamical state of a network is an important first step to predict network responses to external stimulus and top-down modulatory inputs. Models in stable or unstable dynamical regimes require different analytic tools to estimate the network responses to inputs and the structure of neural variability. In this article, I review recent cortical models of state-dependent responses and their predictions about the underlying modulatory mechanisms.     

Neural population codes

In many regions of the brain, information is represented by patterns of activity occurring over populations of neurons. Understanding the encoding of information in neural population activity is important both for grasping the fundamental computations underlying brain function, and for interpreting signals that may be useful for the control of prosthetic devices. We concentrate on the representation of information in neurons with Poisson spike statistics, in which information is contained in the average spike firing rate. We analyze the properties of population codes in terms of the tuning functions that describe individual neuron behavior. The discussion centers on three computational questions: first, what information is encoded in a population; second, how does the brain compute using populations; and third, when is a population optimal? To answer these questions, we discuss several methods for decoding population activity in an experimental setting. We also discuss how computation can be performed within the brain in networks of interconnected populations. Finally, we examine questions of optimal design of population codes that may help to explain their particular form and the set of variables that are best represented. We show that for population codes based on neurons that have a Poisson distribution of spike probabilities, the behavior and computational properties of the code can be understood in terms of the tuning properties of individual cells.     

Stimulus and network dynamics collide in a ratiometric model of the antennal lobe macroglomerular complex

Time is considered to be an important encoding dimension in olfaction, as neural populations generate odour-specific spatiotemporal responses to constant stimuli. However, during pheromone mediated anemotactic search insects must discriminate specific ratios of blend components from rapidly time varying input. The dynamics intrinsic to olfactory processing and those of naturalistic stimuli can therefore potentially collide, thereby confounding ratiometric information. In this paper we use a computational model of the macroglomerular complex of the insect antennal lobe to study the impact on ratiometric information of this potential collision between network and stimulus dynamics. We show that the model exhibits two different dynamical regimes depending upon the connectivity pattern between inhibitory interneurons (that we refer to as fixed point attractor and limit cycle attractor), which both generate ratio-specific trajectories in the projection neuron output population that are reminiscent of temporal patterning and periodic hyperpolarisation observed in olfactory antennal lobe neurons. We compare the performance of the two corresponding population codes for reporting ratiometric blend information to higher centres of the insect brain. Our key finding is that whilst the dynamically rich limit cycle attractor spatiotemporal code is faster and more efficient in transmitting blend information under certain conditions it is also more prone to interference between network and stimulus dynamics, thus degrading ratiometric information under naturalistic input conditions. Our results suggest that rich intrinsically generated network dynamics can provide a powerful means of encoding multidimensional stimuli with high accuracy and efficiency, but only when isolated from stimulus dynamics. This interference between temporal dynamics of the stimulus and temporal patterns of neural activity constitutes a real challenge that must be successfully solved by the nervous system when faced with naturalistic input.