The expansion of information in ecological systems: Emergence as a quantifiable state

Although the term ‘emergence’ has received wide attention in the literature, most of this attention has been focused on epistemological discussions about the nature of what might be considered emergent behavior in self-organizing systems. For the concept of emergence to have any great utility for biologists, it must (1) be perceptible as a physical, quantitative property rather than just a philosophical one; (2) have a quantitative definition applicable to all levels of biological organization; and (3) be an essential component of biological system performance or evolution. Using an independent, cellular population model (running in the StarLogo system), we have developed a mutual information calculation to measure the information expansion when considering the interactions between a population of herbivores and an environment in comparison to the interactions between the individual herbivores and that environment. In self-organizing biological systems, the collective action of massively parallel units generates a greater potential complexity in the information processing capacity of the ‘whole’ system relative to the ‘individual’ parts, and as such, there is a demonstrable increase in mutual information content. From this perspective, we consider emergence to exist as a simple information expansion that is a default behavior of any system with multiple, component parts governed by a simple, probabilistic rule set. It is not a first principle of self-organizing biological systems, but rather a collective behavior that can be quantitatively described in practical terms for experimental biologists. With a quantitative formulation, the concept of emergence may become a useful information statistic in assessing the structure of biological systems.     

Disambiguating Multi–Modal Scene Representations Using Perceptual Grouping Constraints

In its early stages, the visual system suffers from a lot of ambiguity and noise that severely limits the performance of early vision algorithms. This article presents feedback mechanisms between early visual processes, such as perceptual grouping, stereopsis and depth reconstruction, that allow the system to reduce this ambiguity and improve early representation of visual information. In the first part, the article proposes a local perceptual grouping algorithm that — in addition to commonly used geometric information — makes use of a novel multi–modal measure between local edge/line features. The grouping information is then used to: 1) disambiguate stereopsis by enforcing that stereo matches preserve groups; and 2) correct the reconstruction error due to the image pixel sampling using a linear interpolation over the groups. The integration of mutual feedback between early vision processes is shown to reduce considerably ambiguity and noise without the need for global constraints.     

Estimating mutual information using B-spline functions – an improved similarity measure for analysing gene expression data

Background

The information theoretic concept of mutual information provides a general framework to evaluate dependencies between variables. In the context of the clustering of genes with similar patterns of expression it has been suggested as a general quantity of similarity to extend commonly used linear measures. Since mutual information is defined in terms of discrete variables, its application to continuous data requires the use of binning procedures, which can lead to significant numerical errors for datasets of small or moderate size.

Results

In this work, we propose a method for the numerical estimation of mutual information from continuous data. We investigate the characteristic properties arising from the application of our algorithm and show that our approach outperforms commonly used algorithms: The significance, as a measure of the power of distinction from random correlation, is significantly increased. This concept is subsequently illustrated on two large-scale gene expression datasets and the results are compared to those obtained using other similarity measures.A C++ source code of our algorithm is available for non-commercial use from kloska@scienion.de upon request.

Conclusion

The utilisation of mutual information as similarity measure enables the detection of non-linear correlations in gene expression datasets. Frequently applied linear correlation measures, which are often used on an ad-hoc basis without further justification, are thereby extended.

     

From autopoiesis to neurophenomenology: Francisco Varela's exploration of the biophysics of being

This paper reviews in detail Francisco Varela's work on subjectivity and consciousness in the biological sciences. His original approach to this "hard problem" presents a subjectivity that is radically intertwined with its biological and physical roots. It must be understood within the framework of his theory of a concrete, embodied dynamics, grounded in his general theory of autonomous systems. Through concepts and paradigms such as biological autonomy, embodiment and neurophenomenology, the article explores the multiple levels of circular causality assumed by Varela to play a fundamental role in the emergence of human experience. The concept of biological autonomy provides the necessary and sufficient conditions for characterizing biological life and identity as an emergent and circular self-producing process. Embodiment provides a systemic and dynamical framework for understanding how a cognitive self--a mind--can arise in an organism in the midst of its operational cycles of internal regulation and ongoing sensorimotor coupling. Global subjective properties can emerge at different levels from the interactions of components and can reciprocally constrain local processes through an ongoing, recursive morphodynamics. Neurophenomenology is a supplementary step in the study of consciousness. Through a rigorous method, it advocates the careful examination of experience with first-person methodologies. It attempts to create heuristic mutual constraints between biophysical data and data produced by accounts of subjective experience. The aim is to explicitly ground the active and disciplined insight the subject has about his/her experience in a biophysical emergent process. Finally, we discuss Varela's essential contribution to our understanding of the generation of consciousness in the framework of what we call his "biophysics of being."     

The fitness value of information

Communication and information are central concepts in evolutionary biology. In fact, it is hard to find an area of biology where these concepts are not used. However, quantifying the information transferred in biological interactions has been difficult. How much information is transferred when the first spring rainfall hits a dormant seed, or when a chick begs for food from its parent? One measure that is commonly used in such cases is fitness value: by how much, on average, an individual's fitness would increase if it behaved optimally with the new information, compared to its average fitness without the information. Another measure, often used to describe neural responses to sensory stimuli, is the mutual information – a measure of reduction in uncertainty, as introduced by Shannon in communication theory. However, mutual information has generally not been considered to be an appropriate measure for describing developmental or behavioral responses at the organismal level, because it is blind to function; it does not distinguish between relevant and irrelevant information. In this paper we show that there is in fact a surprisingly tight connection between these two measures in the important context of evolution in an uncertain environment. In this case, a useful measure of fitness benefit is the increase in the long‐term growth rate, or the fold increase in number of surviving lineages. We show that in many cases the fitness value of a developmental cue, when measured this way, is exactly equal to the reduction in uncertainty about the environment, as described by the mutual information.     

Time domain measures of inter-channel EEG correlations: a comparison of linear, nonparametric and nonlinear measures

Correlations between ten-channel EEGs obtained from thirteen healthy adult participants were investigated. Signals were obtained in two behavioral states: eyes open no task and eyes closed no task. Four time domain measures were compared: Pearson product moment correlation, Spearman rank order correlation, Kendall rank order correlation and mutual information. The psychophysiological utility of each measure was assessed by determining its ability to discriminate between conditions. The sensitivity to epoch length was assessed by repeating calculations with 1, 2, 3, …, 8 s epochs. The robustness to noise was assessed by performing calculations with noise corrupted versions of the original signals (SNRs of 0, 5 and 10 dB). Three results were obtained in these calculations. First, mutual information effectively discriminated between states with less data. Pearson, Spearman and Kendall failed to discriminate between states with a 1 s epoch, while a statistically significant separation was obtained with mutual information. Second, at all epoch durations tested, the measure of between-state discrimination was greater for mutual information. Third, discrimination based on mutual information was more robust to noise. The limitations of this study are discussed. Further comparisons should be made with frequency domain measures, with measures constructed with embedded data and with the maximal information coefficient.     

Prediction of protein secondary structure based on residue pair types and conformational states using dynamic programming algorithm

We have used a statistical approach for protein secondary structure prediction based on information theory and simultaneously taking into consideration pairwise residue types and conformational states. Since the prediction of residue secondary structure by one residue window sliding make ambiguity in state prediction, we used a dynamic programming algorithm to find the path with maximum score. A score system for residue pairs in particular conformations is derived for adjacent neighbors up to ten residue apart in sequence. The three state overall per-residue accuracy, Q3, of this method in a jackknife test with dataset created from PDBSELECT is more than 70%.     

A measure of phase ambiguity in pairs of SNPs in the presence of linkage disequilibrium

The extent of haplotype ambiguity in a string of single-nucleotide polymorphisms (SNPs) was quantified by Hodge et al. [Nat Genet 1999;21:360]. In their measure, the level of ambiguity increases with increasing numbers of loci and as loci become more polymorphic. That work assumed linkage equilibrium (LE). However, linkage disequilibrium (LD) provides additional information about the haplotypes at a site, thereby diluting the level of ambiguity. The ambiguity vanishes altogether when LD reaches its maximum value. Here, we introduce the ambiguity measure, Phi, to allow for LD (between pairs of SNPs). We derive the formula Phi = 4x(2)x(3) for ambiguity in individuals, where x(1), x(2), x(3) and x(4) are the probabilities of the A(1)A(2), A(1)B(2), B(1)A(2) and B(1)B(2) haplotypes, respectively, and w.l.o.g. x(1)x(4) > or = x(2)x(3). Alternatively, Phi can be expressed in terms of the allele frequencies and the LD parameter delta. We also extend the formula to triads of two parents plus one child. We estimate our measure Phi for relevant SNPs in the published lipoprotein lipase (LPL) gene dataset [Clark et al., Am J Hum Genet 1998;63:595; Nickerson et al., Nat Genet 1998;19:233], obtaining values ranging from a low of 0 to a high of 0.11 among adjacent pairs of sites. In genome-wide LD studies to map common disease genes, a dense map of SNPs may be utilized to detect association between a marker and disease. Therefore, the measurement of ambiguity can potentially help investigators to determine a more efficient map, designed to minimize ambiguity and subsequent information loss.     

Mutual Information Rate and Bounds for It

The amount of information exchanged per unit of time between two nodes in a dynamical network or between two data sets is a powerful concept for analysing complex systems. This quantity, known as the mutual information rate (MIR), is calculated from the mutual information, which is rigorously defined only for random systems. Moreover, the definition of mutual information is based on probabilities of significant events. This work offers a simple alternative way to calculate the MIR in dynamical (deterministic) networks or between two time series (not fully deterministic), and to calculate its upper and lower bounds without having to calculate probabilities, but rather in terms of well known and well defined quantities in dynamical systems. As possible applications of our bounds, we study the relationship between synchronisation and the exchange of information in a system of two coupled maps and in experimental networks of coupled oscillators.     

Mesoscopic Organization Reveals the Constraints Governing Caenorhabditis elegans Nervous System

One of the biggest challenges in biology is to understand how activity at the cellular level of neurons, as a result of their mutual interactions, leads to the observed behavior of an organism responding to a variety of environmental stimuli. Investigating the intermediate or mesoscopic level of organization in the nervous system is a vital step towards understanding how the integration of micro-level dynamics results in macro-level functioning. The coordination of many different co-occurring processes at this level underlies the command and control of overall network activity. In this paper, we have considered the somatic nervous system of the nematode Caenorhabditis elegans, for which the entire neuronal connectivity diagram is known. We focus on the organization of the system into modules, i.e., neuronal groups having relatively higher connection density compared to that of the overall network. We show that this mesoscopic feature cannot be explained exclusively in terms of considerations such as, optimizing for resource constraints (viz., total wiring cost) and communication efficiency (i.e., network path length). Even including information about the genetic relatedness of the cells cannot account for the observed modular structure. Comparison with other complex networks designed for efficient transport (of signals or resources) implies that neuronal networks form a distinct class. This suggests that the principal function of the network, viz., processing of sensory information resulting in appropriate motor response, may be playing a vital role in determining the connection topology. Using modular spectral analysis we make explicit the intimate relation between function and structure in the nervous system. This is further brought out by identifying functionally critical neurons purely on the basis of patterns of intra- and inter-modular connections. Our study reveals how the design of the nervous system reflects several constraints, including its key functional role as a processor of information.     

The evolution of multicellularity in animals as a shift in biological autonomy

The hypothesis is advanced that major evolutionary innovations are characterized by an increase of organismal autonomy in the sense of an emancipation from the environment. After a brief overview of the literature on this concept, increasing autonomy is defined as the evolutionary shift in the individual system-environment relationship, so that the direct influences of the environment are gradually reduced and a stabilization of self-referential, intrinsic functions within the system is generated. This is described as relative autonomy because numerous interconnections with the environment and dependencies upon it are retained. Elements of an increasing autonomy are spatial separations, an increase in homeostatic functions, internalizations and an increase in physiological and behavioral flexibility. These elements are described by taking the transition from single cells to metazoans as a case study. The principle of increasing autonomy is of central relevance for understanding this transition. The hypothesis does not contradict the principle of adaptation, but rather contributes to a further understanding of its elements as it supplies aspects for a reconsideration of the relationship between the outside and the inside, between organism and environment.     

The new physician as unwitting quantum mechanic: is adapting Dirac's inference system best practice for personalized medicine, genomics, and proteomics?

What is the Best Practice for automated inference in Medical Decision Support for personalized medicine? A known system already exists as Dirac's inference system from quantum mechanics (QM) using bra-kets and bras where A and B are states, events, or measurements representing, say, clinical and biomedical rules. Dirac's system should theoretically be the universal best practice for all inference, though QM is notorious as sometimes leading to bizarre conclusions that appear not to be applicable to the macroscopic world of everyday world human experience and medical practice. It is here argued that this apparent difficulty vanishes if QM is assigned one new multiplication function @, which conserves conditionality appropriately, making QM applicable to classical inference including a quantitative form of the predicate calculus. An alternative interpretation with the same consequences is if every i = radical-1 in Dirac's QM is replaced by h, an entity distinct from 1 and i and arguably a hidden root of 1 such that h2 = 1. With that exception, this paper is thus primarily a review of the application of Dirac's system, by application of linear algebra in the complex domain to help manipulate information about associations and ontology in complicated data. Any combined bra-ket can be shown to be composed only of the sum of QM-like bra and ket weights c(), times an exponential function of Fano's mutual information measure I(A; B) about the association between A and B, that is, an association rule from data mining. With the weights and Fano measure re-expressed as expectations on finite data using Riemann's Incomplete (i.e., Generalized) Zeta Functions, actual counts of observations for real world sparse data can be readily utilized. Finally, the paper compares identical character, distinguishability of states events or measurements, correlation, mutual information, and orthogonal character, important issues in data mining and biomedical analytics, as in QM.     

A Bayesian Alternative to Mutual Information for the Hierarchical Clustering of Dependent Random Variables

The use of mutual information as a similarity measure in agglomerative hierarchical clustering (AHC) raises an important issue: some correction needs to be applied for the dimensionality of variables. In this work, we formulate the decision of merging dependent multivariate normal variables in an AHC procedure as a Bayesian model comparison. We found that the Bayesian formulation naturally shrinks the empirical covariance matrix towards a matrix set a priori (e.g., the identity), provides an automated stopping rule, and corrects for dimensionality using a term that scales up the measure as a function of the dimensionality of the variables. Also, the resulting log Bayes factor is asymptotically proportional to the plug-in estimate of mutual information, with an additive correction for dimensionality in agreement with the Bayesian information criterion. We investigated the behavior of these Bayesian alternatives (in exact and asymptotic forms) to mutual information on simulated and real data. An encouraging result was first derived on simulations: the hierarchical clustering based on the log Bayes factor outperformed off-the-shelf clustering techniques as well as raw and normalized mutual information in terms of classification accuracy. On a toy example, we found that the Bayesian approaches led to results that were similar to those of mutual information clustering techniques, with the advantage of an automated thresholding. On real functional magnetic resonance imaging (fMRI) datasets measuring brain activity, it identified clusters consistent with the established outcome of standard procedures. On this application, normalized mutual information had a highly atypical behavior, in the sense that it systematically favored very large clusters. These initial experiments suggest that the proposed Bayesian alternatives to mutual information are a useful new tool for hierarchical clustering.     

Mutual Information for Testing Gene-Environment Interaction

Despite current enthusiasm for investigation of gene-gene interactions and gene-environment interactions, the essential issue of how to define and detect gene-environment interactions remains unresolved. In this report, we define gene-environment interactions as a stochastic dependence in the context of the effects of the genetic and environmental risk factors on the cause of phenotypic variation among individuals. We use mutual information that is widely used in communication and complex system analysis to measure gene-environment interactions. We investigate how gene-environment interactions generate the large difference in the information measure of gene-environment interactions between the general population and a diseased population, which motives us to develop mutual information-based statistics for testing gene-environment interactions. We validated the null distribution and calculated the type 1 error rates for the mutual information-based statistics to test gene-environment interactions using extensive simulation studies. We found that the new test statistics were more powerful than the traditional logistic regression under several disease models. Finally, in order to further evaluate the performance of our new method, we applied the mutual information-based statistics to three real examples. Our results showed that P-values for the mutual information-based statistics were much smaller than that obtained by other approaches including logistic regression models.     

A multilocus linkage disequilibrium measure based on mutual information theory and its applications

Evaluating the patterns of linkage disequilibrium (LD) is important for association mapping study as well as for studying the genomic architecture of human genome (e.g., haplotype block structures). Commonly used bi-allelic pairwise measures for assessing LD between two loci, such as r ² and D′, may not make full and efficient use of modern multilocus data. Though extended to multilocus scenarios, their performance is still questionable. Meanwhile, most existing measures for an entire multilocus region, such as normalized entropy difference, do not consider existence of LD heterogeneity across the region under investigation. Additionally, these existing multilocus measures cannot handle distant regions where long-range LD patterns may exist. In this study, we proposed a novel multilocus LD measure developed based on mutual information theory. Our proposed measure described LD pattern between two chromosome regions each of which may consist of multiple loci (including multi-allele loci). As such, the proposed measure can better characterize LD patterns between two arbitrary regions. As potential applications, we developed algorithms on the proposed measure for partitioning haplotype blocks and for selecting haplotype tagging SNPs (htSNPs), which were helpful for follow-up association tests. The results on both simulated and empirical data showed that our LD measure had distinct advantages over pairwise and other multilocus measures. First, our measure was more robust, and can capture comprehensively the LD information between neighboring as well as disjointed regions. Second, haplotype blocks were better described via our proposed measure. Furthermore, association tests with htSNPs from the proposed algorithm had improved power over tests on single markers and on haplotypes.     

Regime changes in ecological systems: an information theory approach

We present our efforts at developing an ecological system index using information theory. Specifically, we derive an expression for Fisher Information based on sampling of the system trajectory as it evolves in the space defined by the state variables of the system, i.e. its state space. The Fisher Information index, as we have derived it, is a measure of system order, and captures the characteristic variation in speed and acceleration along the system's periodic steady-state trajectories. When calculated repeatedly over the system period, this index tracks steady states and transient behavior. We believe that such an index could be useful in detecting system 'flips' associated with a regime change, i.e. determining when systems are in a transient between one steady state and another. We illustrate the concepts using model ecosystems.     

Towards an Autopoietic Redefinition of Life

In this paper we develop the autopoietic approach to the definition of the living developed by Maturana and Varela in the Seventies. Starting from very simple observations concerning the phenomenology of life, we propose a reformulation of the autopoietic original definition of life which integrates some of the contemporary criticism to it. Our definitional proposal, aiming to stimulate the further development of the autopoietic approach, expresses what remains implicit in the definition of the living originally given by Maturana and Varela: life, as self-production, is a process of cognitive coupling with the environment.     

Revisiting the operational RNA code for amino acids: Ensemble attributes and their implications

It has been suggested that tRNA acceptor stems specify an operational RNA code for amino acids. In the last 20 years several attributes of the putative code have been elucidated for a small number of model organisms. To gain insight about the ensemble attributes of the code, we analyzed 4925 tRNA sequences from 102 bacterial and 21 archaeal species. Here, we used a classification and regression tree (CART) methodology, and we found that the degrees of degeneracy or specificity of the RNA codes in both Archaea and Bacteria differ from those of the genetic code. We found instances of taxon-specific alternative codes, i.e., identical acceptor stem determinants encrypting different amino acids in different species, as well as instances of ambiguity, i.e., identical acceptor stem determinants encrypting two or more amino acids in the same species. When partitioning the data by class of synthetase, the degree of code ambiguity was significantly reduced. In cryptographic terms, a plausible interpretation of this result is that the class distinction in synthetases is an essential part of the decryption rules for resolving the subset of RNA code ambiguities enciphered by identical acceptor stem determinants of tRNAs acylated by enzymes belonging to the two classes. In evolutionary terms, our findings lend support to the notion that in the pre-DNA world, interactions between tRNA acceptor stems and synthetases formed the basis for the distinction between the two classes; hence, ambiguities in the ancient RNA code were pivotal for the fixation of these enzymes in the genomes of ancestral prokaryotes.