Integrated information increases with fitness in the evolution of animats

One of the hallmarks of biological organisms is their ability to integrate disparate information sources to optimize their behavior in complex environments. How this capability can be quantified and related to the functional complexity of an organism remains a challenging problem, in particular since organismal functional complexity is not well-defined. We present here several candidate measures that quantify information and integration, and study their dependence on fitness as an artificial agent ("animat") evolves over thousands of generations to solve a navigation task in a simple, simulated environment. We compare the ability of these measures to predict high fitness with more conventional information-theoretic processing measures. As the animat adapts by increasing its "fit" to the world, information integration and processing increase commensurately along the evolutionary line of descent. We suggest that the correlation of fitness with information integration and with processing measures implies that high fitness requires both information processing as well as integration, but that information integration may be a better measure when the task requires memory. A correlation of measures of information integration (but also information processing) and fitness strongly suggests that these measures reflect the functional complexity of the animat, and that such measures can be used to quantify functional complexity even in the absence of fitness data.     

Information and discriminability as measures of reliability of sensory coding

Response variability is a fundamental issue in neural coding because it limits all information processing. The reliability of neuronal coding is quantified by various approaches in different studies. In most cases it is largely unclear to what extent the conclusions depend on the applied reliability measure, making a comparison across studies almost impossible. We demonstrate that different reliability measures can lead to very different conclusions even if applied to the same set of data: in particular, we applied information theoretical measures (Shannon information capacity and Kullback-Leibler divergence) as well as a discrimination measure derived from signal-detection theory to the responses of blowfly photoreceptors which represent a well established model system for sensory information processing. We stimulated the photoreceptors with white noise modulated light intensity fluctuations of different contrasts. Surprisingly, the signal-detection approach leads to a safe discrimination of the photoreceptor response even when the response signal-to-noise ratio (SNR) is well below unity whereas Shannon information capacity and also Kullback-Leibler divergence indicate a very low performance. Applying different measures, can, therefore, lead to very different interpretations concerning the system's coding performance. As a consequence of the lower sensitivity compared to the signal-detection approach, the information theoretical measures overestimate internal noise sources and underestimate the importance of photon shot noise. We stress that none of the used measures and, most likely no other measure alone, allows for an unbiased estimation of a neuron's coding properties. Therefore the applied measure needs to be selected with respect to the scientific question and the analyzed neuron's functional context.     

Quantifying and tracing information cascades in swarms

We propose a novel, information-theoretic, characterisation of cascades within the spatiotemporal dynamics of swarms, explicitly measuring the extent of collective communications. This is complemented by dynamic tracing of collective memory, as another element of distributed computation, which represents capacity for swarm coherence. The approach deals with both global and local information dynamics, ultimately discovering diverse ways in which an individual's spatial position is related to its information processing role. It also allows us to contrast cascades that propagate conflicting information with waves of coordinated motion. Most importantly, our simulation experiments provide the first direct information-theoretic evidence (verified in a simulation setting) for the long-held conjecture that the information cascades occur in waves rippling through the swarm. Our experiments also exemplify how features of swarm dynamics, such as cascades' wavefronts, can be filtered and predicted. We observed that maximal information transfer tends to follow the stage with maximal collective memory, and principles like this may be generalised in wider biological and social contexts.     

On the validity of Shannon-information calculations for molecular biological sequences

The usefulness of information-theoretic measures of the Shannon-Weaver type, when applied to molecular biological systems such as DNA or protein sequences, has been critically evaluated. It is shown that entropy can be re-expressed in dimensionless terms, thereby making it commensurate with information. Further, we have identified processes in which entropy S and information H change in opposite directions. These processes of opposing signs for delta S and delta H demonstrate that while the Second Law of Thermodynamics mandates that entropy always increases, it places no such restrictions on changes in information. Additionally, we have developed equations permitting information calculations, incorporating conditional occurrence probabilities, on DNA and protein sequences. When the results of such calculations are compared for sequences of various general types, there are no informational content patterns. We conclude that information-theoretic calculations of the present level of sophistication do not provide any useful insights into molecular biological sequences.     

Positional Information,Positional Error,and Readout Precision in Morphogenesis: A Mathematical Framework

The concept of positional information is central to our understanding of how cells determine their location in a multicellular structure and thereby their developmental fates. Nevertheless, positional information has neither been defined mathematically nor quantified in a principled way. Here we provide an information-theoretic definition in the context of developmental gene expression patterns and examine the features of expression patterns that affect positional information quantitatively. We connect positional information with the concept of positional error and develop tools to directly measure information and error from experimental data. We illustrate our framework for the case of gap gene expression patterns in the early Drosophila embryo and show how information that is distributed among only four genes is sufficient to determine developmental fates with nearly single-cell resolution. Our approach can be generalized to a variety of different model systems; procedures and examples are discussed in detail.     

Neural Population Structures and Consequences for Neural Coding

Researchers studying neural coding have speculated that populations of neurons would more effectively represent the stimulus if the neurons "cooperated:" by interacting through lateral connections, the neurons would process and represent information better than if they functioned independently. We apply our new theory of information processing to determine the fidelity limits of simple population structures to encode stimulus features. We focus on noncooperative populations, which have no lateral connections. We show that they always exhibit positively correlated responses and that as population size increases, they perfectly represent the information conveyed by their inputs regardless of the individual neuron's coding scheme. Cooperative populations, which do have lateral connections, can, depending on the nature of the connections, perform better or worse than their noncooperative counterparts. We further show that common notions of synergy fail to capture the level of cooperation and to reflect the information processing properties of populations.     

High-resolution multiple-unit EEG in cat auditory cortex reveals large spatio-temporal stochastic interactions

It has been argued that information processing in the cortex is optimised with regard to certain information theoretic principles. We have, for instance, recently shown that spike-timing dependent plasticity can improve an information-theoretic measure called spatio-temporal stochastic interaction which captures how strongly a set of neurons cooperates in space and time. Systems with high stochastic interaction reveal Poisson spike trains but nonetheless occupy only a strongly reduced area in their global phase space, they reveal repetiting but complex global activation patterns, and they can be interpreted as computational systems operating on selected sets of collective patterns or "global states" in a rule-like manner. In the present work we investigate stochastic interaction in high-resolution EEG-data from cat auditory cortex. Using Kohonen maps to reduce the high-dimensional dynamics of the system, we are able to detect repetiting system states and estimate the stochastic interaction in the data, which turns out to be fairly high. This suggests an organised cooperation in the underlying neural networks which cause the data and may reflect generic intrinsic computational capabilities of the cortex.     

Probing real sensory worlds of receivers with unsupervised clustering

The task of an organism to extract information about the external environment from sensory signals is based entirely on the analysis of ongoing afferent spike activity provided by the sense organs. We investigate the processing of auditory stimuli by an acoustic interneuron of insects. In contrast to most previous work we do this by using stimuli and neurophysiological recordings directly in the nocturnal tropical rainforest, where the insect communicates. Different from typical recordings in sound proof laboratories, strong environmental noise from multiple sound sources interferes with the perception of acoustic signals in these realistic scenarios. We apply a recently developed unsupervised machine learning algorithm based on probabilistic inference to find frequently occurring firing patterns in the response of the acoustic interneuron. We can thus ask how much information the central nervous system of the receiver can extract from bursts without ever being told which type and which variants of bursts are characteristic for particular stimuli. Our results show that the reliability of burst coding in the time domain is so high that identical stimuli lead to extremely similar spike pattern responses, even for different preparations on different dates, and even if one of the preparations is recorded outdoors and the other one in the sound proof lab. Simultaneous recordings in two preparations exposed to the same acoustic environment reveal that characteristics of burst patterns are largely preserved among individuals of the same species. Our study shows that burst coding can provide a reliable mechanism for acoustic insects to classify and discriminate signals under very noisy real-world conditions. This gives new insights into the neural mechanisms potentially used by bushcrickets to discriminate conspecific songs from sounds of predators in similar carrier frequency bands.     

Embedding optimization reveals long-lasting history dependence in neural spiking activity

Information processing can leave distinct footprints on the statistics of neural spiking. For example, efficient coding minimizes the statistical dependencies on the spiking history, while temporal integration of information may require the maintenance of information over different timescales. To investigate these footprints, we developed a novel approach to quantify history dependence within the spiking of a single neuron, using the mutual information between the entire past and current spiking. This measure captures how much past information is necessary to predict current spiking. In contrast, classical time-lagged measures of temporal dependence like the autocorrelation capture how long—potentially redundant—past information can still be read out. Strikingly, we find for model neurons that our method disentangles the strength and timescale of history dependence, whereas the two are mixed in classical approaches. When applying the method to experimental data, which are necessarily of limited size, a reliable estimation of mutual information is only possible for a coarse temporal binning of past spiking, a so-called past embedding. To still account for the vastly different spiking statistics and potentially long history dependence of living neurons, we developed an embedding-optimization approach that does not only vary the number and size, but also an exponential stretching of past bins. For extra-cellular spike recordings, we found that the strength and timescale of history dependence indeed can vary independently across experimental preparations. While hippocampus indicated strong and long history dependence, in visual cortex it was weak and short, while in vitro the history dependence was strong but short. This work enables an information-theoretic characterization of history dependence in recorded spike trains, which captures a footprint of information processing that is beyond time-lagged measures of temporal dependence. To facilitate the application of the method, we provide practical guidelines and a toolbox.     

An information theoretic approach for analyzing temporal patterns of gene expression

MOTIVATION: Arrays allow measurements of the expression levels of thousands of mRNAs to be made simultaneously. The resulting data sets are information rich but require extensive mining to enhance their usefulness. Information theoretic methods are capable of assessing similarities and dissimilarities between data distributions and may be suited to the analysis of gene expression experiments. The purpose of this study was to investigate information theoretic data mining approaches to discover temporal patterns of gene expression from array-derived gene expression data. RESULTS: The Kullback-Leibler divergence, an information-theoretic distance that measures the relative dissimilarity between two data distribution profiles, was used in conjunction with an unsupervised self-organizing map algorithm. Two published, array-derived gene expression data sets were analyzed. The patterns obtained with the KL clustering method were found to be superior to those obtained with the hierarchical clustering algorithm using the Pearson correlation distance measure. The biological significance of the results was also examined. AVAILABILITY: Software code is available by request from the authors. All programs were written in ANSI C and Matlab (Mathworks Inc., Natick, MA).     

Understanding auditory distance estimation by humpback whales: a computational approach

Ranging, the ability to judge the distance to a sound source, depends on the presence of predictable patterns of attenuation. We measured long-range sound propagation in coastal waters to assess whether humpback whales might use frequency degradation cues to range singing whales. Two types of neural networks, a multi-layer and a single-layer perceptron, were trained to classify recorded sounds by distance traveled based on their frequency content. The multi-layer network successfully classified received sounds, demonstrating that the distorting effects of underwater propagation on frequency content provide sufficient cues to estimate source distance. Normalizing received sounds with respect to ambient noise levels increased the accuracy of distance estimates by single-layer perceptrons, indicating that familiarity with background noise can potentially improve a listening whale's ability to range. To assess whether frequency patterns predictive of source distance were likely to be perceived by whales, recordings were pre-processed using a computational model of the humpback whale's peripheral auditory system. Although signals processed with this model contained less information than the original recordings, neural networks trained with these physiologically based representations estimated source distance more accurately, suggesting that listening whales should be able to range singers using distance-dependent changes in frequency content.     

Measuring information transfer in the spike generator of crayfish sustaining fibers

We present a method based on information-theoretic distances for measuring the information transfer efficiency of voltage to impulse encoders. In response to light pulses, we simultaneously recorded the EPSP and spiking output of crayfish sustaining fibers. To measure the distance between analog EPSP responses, we developed a membrane noise model that accurately captures stimulus-induced nonstationarities. By comparing the EPSP and spike responses, we found encoding efficiencies on the order of 10^–4, with interesting dynamics occurring during initial transients. A simple analog to point-process converter predicted the small information transfer efficiencies and dynamic properties we measured.     

听觉中枢神经元对声信号的识别和处理

人类听觉的基本特性和机制与其他哺乳动物相似,因此,利用动物所作的听觉研究和获得的结果,有助于认识人类自身的听觉.围绕听觉中枢神经元对不同模式的声信号的识别和处理,简要综述了这方面的研究.声信号和声模式识别在听觉中枢对声信号的感受和加工中具有重要意义.听神经元作为声模式识别的结构和功能基础,对不同的声刺激模式产生不同反应,甚至是在同一声刺激模式下,改变其中的某个声参数,神经元的反应也会发生相应改变,而其反应的特性和机制均需要更多研究来解答.另外,声信号作为声信息的载体,不同的声信息寓于不同的声参数和声特征之中,研究发现,听觉中枢神经元存在相应的声信息甄别和选择的神经基础,能对动态变化的声频率、幅度和时程等进行反应和编码,并且,在不同种类动物上获得的研究结果极为相似,表明听觉中枢对不同声信号和声刺激模式的识别、分析和加工,具有共同性和普遍性.     

Listening for males and bats: spectral processing in the hearing organ of <Emphasis Type="Italic">Neoconocephalus bivocatus</Emphasis> (Orthoptera: Tettigoniidae)

Tettigoniids use hearing for mate finding and the avoidance of predators (mainly bats). Using intracellular recordings, we studied the response properties of auditory receptor cells of Neoconocephalus bivocatus to different sound frequencies, with a special focus on the frequency ranges representative of male calls and bat cries. We found several response properties that may represent adaptations for hearing in both contexts. Receptor cells with characteristic frequencies close to the dominant frequency of the communication signal were more broadly tuned, thus extending their range of high sensitivity. This increases the number of cells responding to the dominant frequency of the male call at low signal amplitudes, which should improve long distance call localization. Many cells tuned to audio frequencies had intermediate thresholds for ultrasound. As a consequence, a large number of receptors should be recruited at intermediate amplitudes of bat cries. This collective response of many receptors may function to emphasize predator information in the sensory system, and correlates with the amplitude range at which ultrasound elicits evasive behavior in tettigoniids. We compare our results with spectral processing in crickets, and discuss that both groups evolved different adaptations for the perceptual tasks of mate and predator detection.     

Encoding of Natural Sounds at Multiple Spectral and Temporal Resolutions in the Human Auditory Cortex

Functional neuroimaging research provides detailed observations of the response patterns that natural sounds (e.g. human voices and speech, animal cries, environmental sounds) evoke in the human brain. The computational and representational mechanisms underlying these observations, however, remain largely unknown. Here we combine high spatial resolution (3 and 7 Tesla) functional magnetic resonance imaging (fMRI) with computational modeling to reveal how natural sounds are represented in the human brain. We compare competing models of sound representations and select the model that most accurately predicts fMRI response patterns to natural sounds. Our results show that the cortical encoding of natural sounds entails the formation of multiple representations of sound spectrograms with different degrees of spectral and temporal resolution. The cortex derives these multi-resolution representations through frequency-specific neural processing channels and through the combined analysis of the spectral and temporal modulations in the spectrogram. Furthermore, our findings suggest that a spectral-temporal resolution trade-off may govern the modulation tuning of neuronal populations throughout the auditory cortex. Specifically, our fMRI results suggest that neuronal populations in posterior/dorsal auditory regions preferably encode coarse spectral information with high temporal precision. Vice-versa, neuronal populations in anterior/ventral auditory regions preferably encode fine-grained spectral information with low temporal precision. We propose that such a multi-resolution analysis may be crucially relevant for flexible and behaviorally-relevant sound processing and may constitute one of the computational underpinnings of functional specialization in auditory cortex.     

Auditory information coding by modeled cochlear nucleus neurons

In this paper we use information theory to quantify the information in the output spike trains of modeled cochlear nucleus globular bushy cells (GBCs). GBCs are part of the sound localization pathway. They are known for their precise temporal processing, and they code amplitude modulations with high fidelity. Here we investigated the information transmission for a natural sound, a recorded vowel. We conclude that the maximum information transmission rate for a single neuron was close to 1,050 bits/s, which corresponds to a value of approximately 5.8 bits per spike. For quasi-periodic signals like voiced speech, the transmitted information saturated as word duration increased. In general, approximately 80% of the available information from the spike trains was transmitted within about 20 ms. Transmitted information for speech signals concentrated around formant frequency regions. The efficiency of neural coding was above 60% up to the highest temporal resolution we investigated (20 μs). The increase in transmitted information to that precision indicates that these neurons are able to code information with extremely high fidelity, which is required for sound localization. On the other hand, only 20% of the information was captured when the temporal resolution was reduced to 4 ms. As the temporal resolution of most speech recognition systems is limited to less than 10 ms, this massive information loss might be one of the reasons which are responsible for the lack of noise robustness of these systems.     

Computational models of temporal processing in the auditory thalamus

Previous work has shown that neurons in the medial geniculate body (MGB) of the echolocating bat, Myotis lucifugus, display response properties that are distinguishable from those of their afferents in the inferior colliculus (IC). Specifically, MGB neurons display phasic temporal discharge patterns, poor entrainment to trains of constant-amplitude sound pulses, and facilitated responses to amplitude-modulated trains of sound pulses (Llano and Feng 1999). In this study we used a modeling approach to examine the relative contributions of different known sources of inhibition on the temporal response properties of auditory thalamocortical neurons. We found that GABA_A-mediated post-excitatory inhibition resulting from coactivation of thalamocortical neurons and local inhibitory interneurons (in a triadic arrangement) is sufficient to reproduce many of the temporal response properties of MGB neurons. Addition of long-duration GABA_B-mediated inhibition gave the thalamocortical neuron temporal response characteristics that more closely resemble those seen in the experimental data. Neither recurrent inhibition from the thalamic reticular nucleus nor post-synaptic nonlinear mechanisms were necessary to reproduce the temporal transformations between the IC and MGB. This work suggests that feed-forward inhibitory circuitry, coupled with slow GABA_B-mediated inhibition, can emulate temporal information processing at the MGB. The transformation taking place in the MGB can be used to extract salient features from complex, time-varying stimuli, such as echoes returning from moving prey. Received: 11 August 1999 / Accepted in revised form: 5 April 2000     

Precognitive and cognitive elements in sound localization

Sound localization behavior is of great importance for an animal's survival. To localize a sound, animals have to detect a sound source and assign a location to it. In this review we discuss recent results on the underlying mechanisms and on modulatory influences in the barn owl, an auditory specialist with very well developed capabilities to localize sound. Information processing in the barn owl auditory pathway underlying the computations of detection and localization is well understood. This analysis of the sensory information primarily determines the following orienting behavior towards the sound source. However, orienting behavior may be modulated by cognitive (top-down) influences such as attention. We show how advanced stimulation techniques can be used to determine the importance of different cues for sound localization in quasi-realistic stimulation situations, how attentional influences can improve the response to behaviorally relevant stimuli, and how attention can modulate related neural responses. Taken together, these data indicate how sound localization might function in the usually complex natural environment.     

γ-氨基丁酸能抑制对大棕蝠听皮层神经元声反应特性的影响

为了探讨γ－氨基丁酸（γ-aminobutyric acid,GABA)能抑制对大棕蝠(Eptesicus fuscus)听皮层(auditory cortex,AC)神经元声反应特性的影响,采用多管微电极电泳方法,观察了8只大棕蝠AC神经元去ABA能抑制前后声刺激诱发的反应。结果显示,微电泳GABAa受体拮抗剂荷包牡丹碱(bicuculline,Bic)去ABA能抑制可改变声刺激诱发的反应模式;极大地增加神经元冲动发放率,缩短反应的潜伏期和降低反应的最小阈值;不同程度地改变强度－发放率和强度－潜伏期函数。结果提示：1、GABA能抑制对AC神经元声信号处理起重要作用;2、GABA能抑制可改变AC神经元兴奋性支配或输入的效应,并因此定型AC神经元的声反应性质,即发放模式、阈值、强度－发放率和强度－潜伏期函数;3、GABA能抑制为AC神经元的声诱发活动提供一种调制性抑制。