Decoding cortical neuronal signals: Network models,information estimation and spatial tuning

We have studied the encoding of spatial pattern information by complex cells in the primary visual cortex of awake monkeys. Three models for the conditional probabilities of different stimuli, given the neuronal response, were fit and compared using cross-validation. For our data, a feed-forward neural network proved to be the best of these models.The information carried by a cell about a stimulus set can be calculated from the estimated conditional probabilities. We performed a spatial spectroscopy of the encoding, examining how the transmitted information varies with both the average coarseness of the stimulus set and the coarseness differences within it. We find that each neuron encodes information about many features at multiple scales. Our data do not appear to allow a characterization of these variations in terms of the detection of simple single features such as oriented bars.     

Correlation-distortion based identification of Linear-Nonlinear-Poisson models

Linear-Nonlinear-Poisson (LNP) models are a popular and powerful tool for describing encoding (stimulus-response) transformations by single sensory as well as motor neurons. Recently, there has been rising interest in the second- and higher-order correlation structure of neural spike trains, and how it may be related to specific encoding relationships. The distortion of signal correlations as they are transformed through particular LNP models is predictable and in some cases analytically tractable and invertible. Here, we propose that LNP encoding models can potentially be identified strictly from the correlation transformations they induce, and develop a computational method for identifying minimum-phase single-neuron temporal kernels under white and colored random Gaussian excitation. Unlike reverse-correlation or maximum-likelihood, correlation-distortion based identification does not require the simultaneous observation of stimulus-response pairs—only their respective second order statistics. Although in principle filter kernels are not necessarily minimum-phase, and only their spectral amplitude can be uniquely determined from output correlations, we show that in practice this method provides excellent estimates of kernels from a range of parametric models of neural systems. We conclude by discussing how this approach could potentially enable neural models to be estimated from a much wider variety of experimental conditions and systems, and its limitations.     

High-content behavioral analysis of Caenorhabditis elegans in precise spatiotemporal chemical environments

To quantitatively understand chemosensory behaviors, it is desirable to present many animals with repeatable, well-defined chemical stimuli. To that end, we describe a microfluidic system to analyze Caenorhabditis elegans behavior in defined temporal and spatial stimulus patterns. A 2 cm × 2 cm structured arena allowed C. elegans to perform crawling locomotion in a controlled liquid environment. We characterized behavioral responses to attractive odors with three stimulus patterns: temporal pulses, spatial stripes and a linear concentration gradient, all delivered in the fluid phase to eliminate variability associated with air-fluid transitions. Different stimulus configurations preferentially revealed turning dynamics in a biased random walk, directed orientation into an odor stripe and speed regulation by odor. We identified both expected and unexpected responses in wild-type worms and sensory mutants by quantifying dozens of behavioral parameters. The devices are inexpensive, easy to fabricate, reusable and suitable for delivering any liquid-borne stimulus.     

Learning cortical topography from spatiotemporal stimuli

Stimulus representation is a functional interpretation of early sensory cortices. Early sensory cortices are subject to stimulus-induced modifications. Common models for stimulus-induced learning within topographic representations are based on the stimuli's spatial structure and probability distribution. Furthermore, we argue that average temporal stimulus distances reflect the stimuli's relatedness. As topographic representations reflect the stimuli's relatedness, the temporal structure of incoming stimuli is important for the learning in cortical maps. Motivated by recent neurobiological findings, we present an approach of cortical self-organization that additionally takes temporal stimulus aspects into account. The proposed model transforms average interstimulus intervals into representational distances. Thereby, neural topography is related to stimulus dynamics. This offers a new time-based interpretation of cortical maps. Our approach is based on a wave-like spread of cortical activity. Interactions between dynamics and feedforward activations lead to shifts of neural activity. The psychophysical saltation phenomenon may represent an analogue to the shifts proposed here. With regard to cortical plasticity, we offer an explanation for neurobiological findings that other models cannot explain. Moreover, we predict cortical reorganizations under new experimental, spatiotemporal conditions. With regard to psychophysics, we relate the saltation phenomenon to dynamics and interaction in early sensory cortices and predict further effects in the perception of spatiotemporal stimuli. Received: 17 March 1999 / Accepted in revised form: 10 August 1999     

An efficient coding hypothesis links sparsity and selectivity of neural responses

To what extent are sensory responses in the brain compatible with first-order principles? The efficient coding hypothesis projects that neurons use as few spikes as possible to faithfully represent natural stimuli. However, many sparsely firing neurons in higher brain areas seem to violate this hypothesis in that they respond more to familiar stimuli than to nonfamiliar stimuli. We reconcile this discrepancy by showing that efficient sensory responses give rise to stimulus selectivity that depends on the stimulus-independent firing threshold and the balance between excitatory and inhibitory inputs. We construct a cost function that enforces minimal firing rates in model neurons by linearly punishing suprathreshold synaptic currents. By contrast, subthreshold currents are punished quadratically, which allows us to optimally reconstruct sensory inputs from elicited responses. We train synaptic currents on many renditions of a particular bird''s own song (BOS) and few renditions of conspecific birds'' songs (CONs). During training, model neurons develop a response selectivity with complex dependence on the firing threshold. At low thresholds, they fire densely and prefer CON and the reverse BOS (REV) over BOS. However, at high thresholds or when hyperpolarized, they fire sparsely and prefer BOS over REV and over CON. Based on this selectivity reversal, our model suggests that preference for a highly familiar stimulus corresponds to a high-threshold or strong-inhibition regime of an efficient coding strategy. Our findings apply to songbird mirror neurons, and in general, they suggest that the brain may be endowed with simple mechanisms to rapidly change selectivity of neural responses to focus sensory processing on either familiar or nonfamiliar stimuli. In summary, we find support for the efficient coding hypothesis and provide new insights into the interplay between the sparsity and selectivity of neural responses.     

Information-processing demands in electrosensory and mechanosensory lateral line systems.

The electrosensory and mechanosensory lateral line systems of fish exhibit many common features in their structural and functional organization, both at the sensory periphery as well as in central processing pathways. These two sensory systems also appear to play similar roles in many behavioral tasks such as prey capture, orientation with respect to external environmental cues, navigation in low-light conditions, and mediation of interactions with nearby animals. In this paper, we briefly review key morphological, physiological, and behavioral aspects of these two closely related sensory systems. We present arguments that the information processing demands associated with spatial processing are likely to be quite similar, due largely to the spatial organization of both systems and the predominantly dipolar nature of many electrosensory and mechanosensory stimulus fields. Demands associated with temporal processing may be quite different, however, due primarily to differences in the physical bases of electrosensory and mechanosensory stimuli (e.g. speed of transmission). With a better sense of the information processing requirements, we turn our attention to an analysis of the functional organization of the associated first-order sensory nuclei in the hindbrain, including the medial octavolateral nucleus (MON), dorsal octavolateral nucleus (DON), and electrosensory lateral line lobe (ELL). One common feature of these systems is a set of neural mechanisms for improving signal-to-noise ratios, including mechanisms for adaptive suppression of reafferent signals. This comparative analysis provides new insights into how the nervous system extracts biologically significant information from dipolar stimulus fields in order to solve a variety of behaviorally relevant problems faced by aquatic animals.     

Symbols as Self-emergent Entities in an Optimization Process of Feature Extraction and Predictions

In the mammalian cortex the early sensory processing can be characterized as feature extraction resulting in local and analogue low-level representations. As a direct consequence, these map directly to the environment, but interpretation under natural conditions is ambiguous. In contrast, high-level representations for cognitive processing, e.g. language, require symbolic representations characterized by expression and syntax. The representations are binary, structured and disambiguated. However, do these fundamental functional distinctions translate into a fundamental distinction of the respective brain areas and their anatomical and physiological properties? Here we argue that the distinction between early sensory processing and higher cognitive functions may not be based on structural differences of cortical areas; instead similar learning principles acting on input signals with different statistics give rise to the observed variations of function. Firstly, we give an account of present research describing neuronal properties at early stages of sensory systems as a consequence of an optimization process over the set of natural stimuli. Secondly, addressing a stage following early visual processing we suggest to extend the unsupervised learning scheme by including predictive processes. These contain the widely used objective of temporal coherence as a special case and are a powerful approach to resolve ambiguities. Furthermore, in combination with a prior on the bandwidth of information exchange between units it leads to a condensation of information. Thirdly, as a crucial step, not only are predictive units optimized, but the selectivity of the feature extractors are adapted to allow optimal predictability. Thus, over and beyond making useful predictions, we propose that the predictability of a stimulus be in itself a selection criterion for further processing. In a hierarchical system the combined optimization process leads to entities that represent condensed pieces of knowledge and that are not analogue anymore. Instead, these entities work as arguments in a framework of transformations that realize predictions. Thus, the criteria of predictability and condensation in an optimization of sensory representations relate directly to the two defining properties of symbols of expression and syntax. In this paper, we sketch an unsupervised learning process that gradually transforms analogue local representations into discrete binary representations by means of four hypotheses. We propose that in this optimization process acting in a hierarchical system, entities emerge at, higher levels that fulfil the criteria defining symbols, instantiating qualitatively different representations at similarly structured low and high levels.     

The role of inhibition and adaptation in sensory information processing

Some common features of neural transformations along sensory pathways are discussed. The emphasis is on spatial mapping in the visual system, but close parallels exist in temporal visual mapping as well as other sensory systems. The role played by lateral inhibition in sequential transformations is investigated by direct computation and by mathematical analysis.     

Simple model for encoding natural images by retinal ganglion cells with nonlinear spatial integration

A central goal in sensory neuroscience is to understand the neuronal signal processing involved in the encoding of natural stimuli. A critical step towards this goal is the development of successful computational encoding models. For ganglion cells in the vertebrate retina, the development of satisfactory models for responses to natural visual scenes is an ongoing challenge. Standard models typically apply linear integration of visual stimuli over space, yet many ganglion cells are known to show nonlinear spatial integration, in particular when stimulated with contrast-reversing gratings. We here study the influence of spatial nonlinearities in the encoding of natural images by ganglion cells, using multielectrode-array recordings from isolated salamander and mouse retinas. We assess how responses to natural images depend on first- and second-order statistics of spatial patterns inside the receptive field. This leads us to a simple extension of current standard ganglion cell models. We show that taking not only the weighted average of light intensity inside the receptive field into account but also its variance over space can partly account for nonlinear integration and substantially improve response predictions of responses to novel images. For salamander ganglion cells, we find that response predictions for cell classes with large receptive fields profit most from including spatial contrast information. Finally, we demonstrate how this model framework can be used to assess the spatial scale of nonlinear integration. Our results underscore that nonlinear spatial stimulus integration translates to stimulation with natural images. Furthermore, the introduced model framework provides a simple, yet powerful extension of standard models and may serve as a benchmark for the development of more detailed models of the nonlinear structure of receptive fields.     

How to cope with negative estimates of components of variance in ecological field studies

Ecologists often need to estimate components of spatial or temporal variation. The most widely used method in ecology uses the observed and expected mean squares in an analysis of variance. A more general approach, which can be used for balanced and unbalanced designs, is based on residual maximal likelihood (REML). This method is less well known by ecologists and requires specialist software. If the design is balanced, the two methods are equivalent, except for one important respect: estimates from analysis of variance can be negative whereas REML estimates cannot. The purpose of this note is to point out a simple modification to the analyses of variance which yields the same estimates as REML for many of the designs commonly used in ecological studies. This modification has been available in the mathematical literature for over 30 years, but appears not to be well known amongst ecologists. It is useful in many cases of balanced analytical designs.     

Covariate measurement error adjustment for matched case-control studies

We propose a conditional scores procedure for obtaining bias-corrected estimates of log odds ratios from matched case-control data in which one or more covariates are subject to measurement error. The approach involves conditioning on sufficient statistics for the unobservable true covariates that are treated as fixed unknown parameters. For the case of Gaussian nondifferential measurement error, we derive a set of unbiased score equations that can then be solved to estimate the log odds ratio parameters of interest. The procedure successfully removes the bias in naive estimates, and standard error estimates are obtained by resampling methods. We present an example of the procedure applied to data from a matched case-control study of prostate cancer and serum hormone levels, and we compare its performance to that of regression calibration procedures.     

Spatial sensing of stimulus gradients can be superior to temporal sensing for free-swimming bacteria.

Predictions of the minimal size an organism must have to swim along stimulus gradients were used to compare the relative advantages of sensory systems employing spatial (simultaneous) and temporal (sequential) gradient detection mechanisms for small free-swimming bacteria, leading to the following conclusions: 1) there are environmental conditions where spatial detection mechanisms can function for smaller organisms than can temporal mechanisms, 2) temporal mechanisms are superior (have a smaller size limit) for the difficult conditions of low concentration and shallow gradients, but 3) observed bacterial chemotaxis occurs mostly under conditions where spatial mechanisms have a smaller size limit, and 4) relevant conditions in the natural environment favor temporal mechanisms in some cases and spatial mechanisms in others. Thus, sensory ecology considerations do not preclude free-swimming bacteria from employing spatial detection mechanisms, as has been thought, and microbiologists should be on the lookout for them. If spatial mechanisms do not occur, the explanation should be sought elsewhere.     

Rapid adaptation and efficient coding

Rapid adaptation is a prominent feature of biological neuronal systems. From a functional perspective the adaptation of neuronal properties, namely the input-output relation of sensory neurons, is usually interpreted as an adaptation of the sensory system to changing environments as characterized by their stimulus statistics. Here we argue that this interpretation is only applicable as long as the adaptation processes are slower than the time-scale at which the stimulus statistics change. We present a definition of optimality of a neuronal code which still captures the idea of efficient coding, but which can also explain rapid adaptation without referring to an adaptation to different sensory environments. Finally, we apply our new idea to a simple model of an orientation hypercolumn in the primary visual cortex and predict that the interactions between orientation columns should adapt at the time-scale of a single stimulus presentation.     

Recalibration of the Multisensory Temporal Window of Integration Results from Changing Task Demands

The notion of the temporal window of integration, when applied in a multisensory context, refers to the breadth of the interval across which the brain perceives two stimuli from different sensory modalities as synchronous. It maintains a unitary perception of multisensory events despite physical and biophysical timing differences between the senses. The boundaries of the window can be influenced by attention and past sensory experience. Here we examined whether task demands could also influence the multisensory temporal window of integration. We varied the stimulus onset asynchrony between simple, short-lasting auditory and visual stimuli while participants performed two tasks in separate blocks: a temporal order judgment task that required the discrimination of subtle auditory-visual asynchronies, and a reaction time task to the first incoming stimulus irrespective of its sensory modality. We defined the temporal window of integration as the range of stimulus onset asynchronies where performance was below 75% in the temporal order judgment task, as well as the range of stimulus onset asynchronies where responses showed multisensory facilitation (race model violation) in the reaction time task. In 5 of 11 participants, we observed audio-visual stimulus onset asynchronies where reaction time was significantly accelerated (indicating successful integration in this task) while performance was accurate in the temporal order judgment task (indicating successful segregation in that task). This dissociation suggests that in some participants, the boundaries of the temporal window of integration can adaptively recalibrate in order to optimize performance according to specific task demands.     

Reduction of information redundancy in the ascending auditory pathway

Information processing by a sensory system is reflected in the changes in stimulus representation along its successive processing stages. We measured information content and stimulus-induced redundancy in the neural responses to a set of natural sounds in three successive stations of the auditory pathway-inferior colliculus (IC), auditory thalamus (MGB), and primary auditory cortex (A1). Information about stimulus identity was somewhat reduced in single A1 and MGB neurons relative to single IC neurons, when information is measured using spike counts, latency, or temporal spiking patterns. However, most of this difference was due to differences in firing rates. On the other hand, IC neurons were substantially more redundant than A1 and MGB neurons. IC redundancy was largely related to frequency selectivity. Redundancy reduction may be a generic organization principle of neural systems, allowing for easier readout of the identity of complex stimuli in A1 relative to IC.     

Neural correlates of knowledge: stable representation of stimulus associations across variations in behavioral performance

Behavioral responses to a sensory stimulus are often guided by associative memories. These associations remain intact even when other factors determine behavior. The substrates of associative memory should therefore be identifiable by neuronal responses that are independent of behavioral choices. We tested this hypothesis using a paired-associates task in which monkeys learned arbitrary associations between pairs of visual stimuli. We examined the activity of neurons in inferior temporal cortex as the animals prepared to choose a remembered stimulus from a visual display. The activity of some neurons (22%) depended on the monkey's behavioral choice; but for a novel class of neurons (54%), activity reflected the stimulus that the monkey was instructed to choose, regardless of the behavioral response. These neurons appear to represent memorized stimulus associations that are stable across variations in behavioral performance. In addition, many neurons (74%) were modulated by the spatial arrangement of the stimuli in the display.     

Spatial and temporal analysis of disease occurrence for detection of clustering

Data on disease occurrence often consist of the number of cases recorded in a set of regions during each of several time periods. In this paper a method of analysis of such data is proposed which allows one to distinguish and test for spatial clustering, temporal clustering, and space-time clustering of cases. Departures of case numbers from expectation are partitioned into region and time period main effects and region-by-time interactions. Spatial and temporal relationships among regions and time periods are incorporated by pooling estimates over neighborhoods. In addition to providing tests for clustering, the analysis identifies the contributions to clustering made by individual neighborhoods. Data on Creutzfeldt-Jakob disease in France are used to illustrate the approach.     

Neural signatures of cell assembly organization

Cortical neurons show irregular but structured spike trains. This has been interpreted as evidence for 'temporal coding', whereby stimuli are represented by precise spike-timing patterns. Here, we suggest an alternative interpretation based on the older concept of the cell assembly. The dynamic evolution of assembly sequences, which are steered but not deterministically controlled by sensory input, is the proposed substrate of psychological processes beyond simple stimulus-response associations. Accordingly, spike trains show a temporal structure that is stimulus-dependent and more variable than would be predicted by strict sensory control. We propose four signatures of assembly organization that can be experimentally tested. We argue that many observations that have been interpreted as evidence for temporal coding might instead reflect an underlying assembly structure.