Cortical sensitivity to visual features in natural scenes

A central hypothesis concerning sensory processing is that the neuronal circuits are specifically adapted to represent natural stimuli efficiently. Here we show a novel effect in cortical coding of natural images. Using spike-triggered average or spike-triggered covariance analyses, we first identified the visual features selectively represented by each cortical neuron from its responses to natural images. We then measured the neuronal sensitivity to these features when they were present in either natural images or random stimuli. We found that in the responses of complex cells, but not of simple cells, the sensitivity was markedly higher for natural images than for random stimuli. Such elevated sensitivity leads to increased detectability of the visual features and thus an improved cortical representation of natural scenes. Interestingly, this effect is due not to the spatial power spectra of natural images, but to their phase regularities. These results point to a distinct visual-coding strategy that is mediated by contextual modulation of cortical responses tuned to the spatial-phase structure of natural scenes.     

A universal model of esthetic perception based on the sensory coding of natural stimuli

Philosophers have pointed out that there is a close relation between the esthetics of art and the beauty of natural scenes. Supporting this similarity at the experimental level, we have recently shown that visual art and natural scenes share fractal-like, scale-invariant statistical properties. Moreover, evidence from neurophysiological experiments shows that the visual system uses an efficient (sparse) code to process optimally the statistical properties of natural stimuli. In the present work, a hypothetical model of esthetic perception is described that combines both lines of evidence. Specifically, it is proposed that an artist creates a work of art so that it induces a specific resonant state in the visual system. This resonant state is thought to be based on the adaptation of the visual system to natural scenes. The proposed model is universal and predicts that all human beings share the same general concept of esthetic judgment. The model implies that esthetic perception, like the coding of natural stimuli, depends on stimulus form rather than content, depends on higher-order statistics of the stimuli, and is non-intuitive to cognitive introspection. The model accommodates the central tenet of neuroesthetic theory that esthetic perception reflects fundamental functional properties of the nervous system.     

Block structure and stability of the genetic code

It is known that different codons may be unified into larger groups related to the hierarchical structure, approximate hidden symmetries, and evolutionary origin of the universal genetic code. Using a simplified evolutionary motivated two-letter version of genetic code, the general principles of the most stable coding are discussed. By the complete enumeration in such a reduced code it is strictly proved that the maximum stability with respect to point mutations and shifts in the reading frame needs the fixation of the middle letters within codons in groups with different physico-chemical properties, thus, explaining a key feature of the universal genetic code. The translational stability of the genetic code is studied by the mapping of code onto de Bruijn graph providing both the compact visual representation of mutual relationships between different codons as well as between codons and protein coding DNA sequence and a powerful tool for the investigation of stability of protein coding. Then, the results are extended to four-letter codes. As is shown, the universal genetic code obeys mainly the principles of optimal coding. These results demonstrate the hierarchical character of optimization of universal genetic code with strictly optimal coding being evolved at the earliest stages of molecular evolution. Finally, the universal genetic code is compared with the other natural variants of genetic codes.     

Neural coding of natural stimuli: information at sub-millisecond resolution

Sensory information about the outside world is encoded by neurons in sequences of discrete, identical pulses termed action potentials or spikes. There is persistent controversy about the extent to which the precise timing of these spikes is relevant to the function of the brain. We revisit this issue, using the motion-sensitive neurons of the fly visual system as a test case. Our experimental methods allow us to deliver more nearly natural visual stimuli, comparable to those which flies encounter in free, acrobatic flight. New mathematical methods allow us to draw more reliable conclusions about the information content of neural responses even when the set of possible responses is very large. We find that significant amounts of visual information are represented by details of the spike train at millisecond and sub-millisecond precision, even though the sensory input has a correlation time of ~55 ms; different patterns of spike timing represent distinct motion trajectories, and the absolute timing of spikes points to particular features of these trajectories with high precision. Finally, the efficiency of our entropy estimator makes it possible to uncover features of neural coding relevant for natural visual stimuli: first, the system's information transmission rate varies with natural fluctuations in light intensity, resulting from varying cloud cover, such that marginal increases in information rate thus occur even when the individual photoreceptors are counting on the order of one million photons per second. Secondly, we see that the system exploits the relatively slow dynamics of the stimulus to remove coding redundancy and so generate a more efficient neural code.     

自然图像效率编码

效率编码理论认为经过漫长历史进化,大脑感知系统有效地适应了自然环境.自然图像统计规律计算建模对视觉信息处理机理的理解大有裨益.本文简要回顾近期视觉系统对自然图像效率编码的最新进展.     

A three-layer model of natural image statistics

An important property of visual systems is to be simultaneously both selective to specific patterns found in the sensory input and invariant to possible variations. Selectivity and invariance (tolerance) are opposing requirements. It has been suggested that they could be joined by iterating a sequence of elementary selectivity and tolerance computations. It is, however, unknown what should be selected or tolerated at each level of the hierarchy. We approach this issue by learning the computations from natural images. We propose and estimate a probabilistic model of natural images that consists of three processing layers. Two natural image data sets are considered: image patches, and complete visual scenes downsampled to the size of small patches. For both data sets, we find that in the first two layers, simple and complex cell-like computations are performed. In the third layer, we mainly find selectivity to longer contours; for patch data, we further find some selectivity to texture, while for the downsampled complete scenes, some selectivity to curvature is observed.     

Processing of complex stimuli and natural scenes in the visual cortex

A major part of vision research builds on the assumption that processing of visual stimuli can be understood on the basis of knowledge about the processing of simplified, artificial stimuli. Recent experimental advances, however, show that a combination of responses to simplified stimuli does not adequately describe responses to natural visual scenes. The systems performance exceeds the performance predicted from understanding its basic constituents. This highlights the fact that the visual system is specifically adapted to the properties of its everyday input and can only fully be understood when probed with naturalistic stimuli.     

What we see is how we are: new paradigms in visual research

As most sensory modalities, the visual system needs to deal with very fast changes in the environment. Instead of processing all sensory stimuli, the brain is able to construct a perceptual experience by combining selected sensory input with an ongoing internal activity. Thus, the study of visual perception needs to be approached by examining not only the physical properties of stimuli, but also the brain's ongoing dynamical states onto which these perturbations are imposed. At least three different models account for this internal dynamics. One model is based on cardinal cells where the activity of few cells by itself constitutes the neuronal correlate of perception, while a second model is based on a population coding that states that the neuronal correlate of perception requires distributed activity throughout many areas of the brain. A third proposition, known as the temporal correlation hypothesis states that the distributed neuronal populations that correlate with perception, are also defined by synchronization of the activity on a millisecond time scale. This would serve to encode contextual information by defining relations between the features of visual objects. If temporal properties of neural activity are important to establish the neural mechanisms of perception, then the study of appropriate dynamical stimuli should be instrumental to determine how these systems operate. The use of natural stimuli and natural behaviors such as free viewing, which features fast changes of internal brain states as seen by motor markers, is proposed as a new experimental paradigm to study visual perception.     

Slow Feature Analysis on Retinal Waves Leads to V1 Complex Cells

The developing visual system of many mammalian species is partially structured and organized even before the onset of vision. Spontaneous neural activity, which spreads in waves across the retina, has been suggested to play a major role in these prenatal structuring processes. Recently, it has been shown that when employing an efficient coding strategy, such as sparse coding, these retinal activity patterns lead to basis functions that resemble optimal stimuli of simple cells in primary visual cortex (V1). Here we present the results of applying a coding strategy that optimizes for temporal slowness, namely Slow Feature Analysis (SFA), to a biologically plausible model of retinal waves. Previously, SFA has been successfully applied to model parts of the visual system, most notably in reproducing a rich set of complex-cell features by training SFA with quasi-natural image sequences. In the present work, we obtain SFA units that share a number of properties with cortical complex-cells by training on simulated retinal waves. The emergence of two distinct properties of the SFA units (phase invariance and orientation tuning) is thoroughly investigated via control experiments and mathematical analysis of the input-output functions found by SFA. The results support the idea that retinal waves share relevant temporal and spatial properties with natural visual input. Hence, retinal waves seem suitable training stimuli to learn invariances and thereby shape the developing early visual system such that it is best prepared for coding input from the natural world.     

Testing the efficiency of sensory coding with optimal stimulus ensembles

According to Barlow's seminal "efficient coding hypothesis," the coding strategy of sensory neurons should be matched to the statistics of stimuli that occur in an animal's natural habitat. Using an automatic search technique, we here test this hypothesis and identify stimulus ensembles that sensory neurons are optimized for. Focusing on grasshopper auditory receptor neurons, we find that their optimal stimulus ensembles differ from the natural environment, but largely overlap with a behaviorally important sub-ensemble of the natural sounds. This indicates that the receptors are optimized for peak rather than average performance. More generally, our results suggest that the coding strategies of sensory neurons are heavily influenced by differences in behavioral relevance among natural stimuli.     

Odour concentration affects odour identity in honeybees

The fact that most types of sensory stimuli occur naturally over a large range of intensities is a challenge to early sensory processing. Sensory mechanisms appear to be optimized to extract perceptually significant stimulus fluctuations that can be analysed in a manner largely independent of the absolute stimulus intensity. This general principle may not, however, extend to olfaction; many studies have suggested that olfactory stimuli are not perceptually invariant with respect to odour intensity. For many animals, absolute odour intensity may be a feature in itself, such that it forms a part of odour identity and thus plays an important role in discrimination alongside other odour properties such as the molecular identity of the odorant. The experiments with honeybees reported here show a departure from odour-concentration invariance and are consistent with a lower-concentration regime in which odour concentration contributes to overall odour identity and a higher-concentration regime in which it may not. We argue that this could be a natural consequence of odour coding and suggest how an 'intensity feature' might be useful to the honeybee in natural odour detection and discrimination.     

A Simple Model of Optimal Population Coding for Sensory Systems

A fundamental task of a sensory system is to infer information about the environment. It has long been suggested that an important goal of the first stage of this process is to encode the raw sensory signal efficiently by reducing its redundancy in the neural representation. Some redundancy, however, would be expected because it can provide robustness to noise inherent in the system. Encoding the raw sensory signal itself is also problematic, because it contains distortion and noise. The optimal solution would be constrained further by limited biological resources. Here, we analyze a simple theoretical model that incorporates these key aspects of sensory coding, and apply it to conditions in the retina. The model specifies the optimal way to incorporate redundancy in a population of noisy neurons, while also optimally compensating for sensory distortion and noise. Importantly, it allows an arbitrary input-to-output cell ratio between sensory units (photoreceptors) and encoding units (retinal ganglion cells), providing predictions of retinal codes at different eccentricities. Compared to earlier models based on redundancy reduction, the proposed model conveys more information about the original signal. Interestingly, redundancy reduction can be near-optimal when the number of encoding units is limited, such as in the peripheral retina. We show that there exist multiple, equally-optimal solutions whose receptive field structure and organization vary significantly. Among these, the one which maximizes the spatial locality of the computation, but not the sparsity of either synaptic weights or neural responses, is consistent with known basic properties of retinal receptive fields. The model further predicts that receptive field structure changes less with light adaptation at higher input-to-output cell ratios, such as in the periphery.     

Coding of natural scenes in primary visual cortex

Natural scene coding in ferret visual cortex was investigated using a new technique for multi-site recording of neuronal activity from the cortical surface. Surface recordings accurately reflected radially aligned layer 2/3 activity. At individual sites, evoked activity to natural scenes was weakly correlated with the local image contrast structure falling within the cells' classical receptive field. However, a population code, derived from activity integrated across cortical sites having retinotopically overlapping receptive fields, correlated strongly with the local image contrast structure. Cell responses demonstrated high lifetime sparseness, population sparseness, and high dispersal values, implying efficient neural coding in terms of information processing. These results indicate that while cells at an individual cortical site do not provide a reliable estimate of the local contrast structure in natural scenes, cell activity integrated across distributed cortical sites is closely related to this structure in the form of a sparse and dispersed code.     

Redundancy in the population code of the retina

We have explored the manner in which the population of retinal ganglion cells collectively represent the visual world. Ganglion cells in the salamander were recorded simultaneously with a multielectrode array during stimulation with both artificial and natural visual stimuli, and the mutual information that single cells and pairs of cells conveyed about the stimulus was estimated. We found significant redundancy between cells spaced as far as 500 mum apart. When we used standard methods for defining functional types, only ON-type and OFF-type cells emerged as truly independent information channels. Although the average redundancy between nearby cell pairs was moderate, each ganglion cell shared information with many neighbors, so that visual information was represented approximately 10-fold within the ganglion cell population. This high degree of retinal redundancy suggests that design principles beyond coding efficiency may be important at the population level.     

Contrast discrimination,non-uniform patterns and change blindness.

Change blindness--our inability to detect large changes in natural scenes when saccades, blinks and other transients interrupt visual input--seems to contradict psychophysical evidence for our exquisite sensitivity to contrast changes. Can the type of effects described as ''change blindness'' be observed with simple, multi-element stimuli, amenable to psychophysical analysis? Such stimuli, composed of five mixed contrast elements, elicited a striking increase in contrast increment thresholds compared to those for an isolated element. Cue presentation prior to the stimulus substantially reduced thresholds, as for change blindness with natural scenes. On one hand, explanations for change blindness based on abstract and sketchy representations in short-term visual memory seem inappropriate for this low-level image property of contrast where there is ample evidence for exquisite performance on memory tasks. On the other hand, the highly increased thresholds for mixed contrast elements, and the decreased thresholds when a cue is present, argue against any simple early attentional or sensory explanation for change blindness. Thus, psychophysical results for very simple patterns cannot straightforwardly predict results even for the slightly more complicated patterns studied here.     

Retinal coding of visual scenes -- repetitive and redundant too?

Visual information reaches the brain by way of a fine cable, the optic nerve. The million or so axons in the optic nerve represent an information bottleneck in the visual pathway-where the fewest number of neurons convey the visual scene. It has long been thought that to make the most of the optic nerve's limited capacity the retina may encode visual information in an optimally efficient manner. In this issue of Neuron, Puchalla et al. report a test of this hypothesis using multielectrode recordings from retinal ganglion cells stimulated with movies of natural scenes. The authors find substantial redundancy in the retinal code and estimate that there is an approximately 10-fold overrepresentation of visual information.     

Summation of perceptual cues in natural visual scenes

Natural visual scenes are rich in information, and any neural system analysing them must piece together the many messages from large arrays of diverse feature detectors. It is known how threshold detection of compound visual stimuli (sinusoidal gratings) is determined by their components' thresholds. We investigate whether similar combination rules apply to the perception of the complex and suprathreshold visual elements in naturalistic visual images. Observers gave magnitude estimations (ratings) of the perceived differences between pairs of images made from photographs of natural scenes. Images in some pairs differed along one stimulus dimension such as object colour, location, size or blur. But, for other image pairs, there were composite differences along two dimensions (e.g. both colour and object-location might change). We examined whether the ratings for such composite pairs could be predicted from the two ratings for the respective pairs in which only one stimulus dimension had changed. We found a pooling relationship similar to that proposed for simple stimuli: Minkowski summation with exponent 2.84 yielded the best predictive power (r=0.96), an exponent similar to that generally reported for compound grating detection. This suggests that theories based on detecting simple stimuli can encompass visual processing of complex, suprathreshold stimuli.     

Sensory coding and contrast invariance emerge from the control of plastic inhibition over emergent selectivity

Visual stimuli are represented by a highly efficient code in the primary visual cortex, but the development of this code is still unclear. Two distinct factors control coding efficiency: Representational efficiency, which is determined by neuronal tuning diversity, and metabolic efficiency, which is influenced by neuronal gain. How these determinants of coding efficiency are shaped during development, supported by excitatory and inhibitory plasticity, is only partially understood. We investigate a fully plastic spiking network of the primary visual cortex, building on phenomenological plasticity rules. Our results suggest that inhibitory plasticity is key to the emergence of tuning diversity and accurate input encoding. We show that inhibitory feedback (random and specific) increases the metabolic efficiency by implementing a gain control mechanism. Interestingly, this led to the spontaneous emergence of contrast-invariant tuning curves. Our findings highlight that (1) interneuron plasticity is key to the development of tuning diversity and (2) that efficient sensory representations are an emergent property of the resulting network.     

Functional mechanisms shaping lateral geniculate responses to artificial and natural stimuli

Functional models of the early visual system should predict responses not only to simple artificial stimuli but also to sequences of complex natural scenes. An ideal testbed for such models is the lateral geniculate nucleus (LGN). Mechanisms shaping LGN responses include the linear receptive field and two fast adaptation processes, sensitive to luminance and contrast. We propose a compact functional model for these mechanisms that operates on sequences of arbitrary images. With the same parameters that fit the firing rate responses to simple stimuli, it predicts the bulk of the firing rate responses to complex stimuli, including natural scenes. Further improvements could result by adding a spiking mechanism, possibly one capable of bursts, but not by adding mechanisms of slow adaptation. We conclude that up to the LGN the responses to natural scenes can be largely explained through insights gained with simple artificial stimuli.