Conditional entropy approach for the evaluation of the coupling strength

A method that enables measurement of the degree of coupling between two signals is presented. The method is based on the definition of an uncoupling function calculating, by means of entropy rates, the minimum amount of independent information (i.e. the information carried by one signal which cannot be derived from the other). An estimator of the uncoupling function able to deal with short segments of data (a few hundred samples) is proposed, thus enabling the method to be used for usual experimental recordings. A synchronisation index is derived from the estimate of the uncoupling function by means of a minimisation procedure. It quantifies the maximum amount of information exchanged between the two signals. Simulations in which non-linear coordination schemes are produced and changes in the coupling strength are artificially induced are used to check the ability of the proposed index to measure the degree of synchronisation between signals. The synchronisation analysis is utilised to measure the coupling strength between the beat-to-beat variability of the sympathetic discharge and ventilation in decerebrate artificially ventilated cats and the degree of synchronisation between the beat-to-beat variability of the heart period and ventricular repolarisation interval in normal subjects and myocardial infarction patients. The sympathetic discharge and ventilation are strongly coupled and the coupling strength is not affected by manoeuvres capable of increasing or depressing sympathetic activity. The synchronisation is lost after spinalisation. The synchronisation analysis confirms that the heart period and ventricular repolarisation interval are well coordinated. In normal subjects, the synchronisation index is not modified by experimental conditions inducing changes in the sympathovagal balance. On the contrary, it strongly decreases after myocardial infarction, thus detecting and measuring the uncoupling between the heart period and ventricular repolarisation interval. Received: 29 October 1998 / Received in revised form: 4 March 1999     

Causal interactions between the cerebral cortex and the autonomic nervous system

Mental states such as stress and anxiety can cause heart disease.On the other hand,meditation can improve cardiac performance.In this study,the heart rate variability,directed transfer function and corrected conditional entropy were used to investigate the effects of mental tasks on cardiac performance,and the functional coupling between the cerebral cortex and the heart.When subjects tried to decrease their heart rate by volition,the sympathetic nervous system was inhibited and the heart rate decreased.When subjects tried to increase their heart rate by volition,the parasympathetic nervous system was inhibited and the sympathetic nervous system was stimulated,and the heart rate increased.When autonomic nervous system activity was regulated by mental tasks,the information flow from the post-central areas to the pre-central areas of the cerebral cortex increased,and there was greater coupling between the brain and the heart.Use of directed transfer function and corrected conditional entropy techniques enabled analysis of electroencephalographic recordings,and of the information flow causing functional coupling between the brain and the heart.     

Progressive decrease of heart period variability entropy-based complexity during graded head-up tilt.

Complexity (or its opposite, regularity) of heart period variability has been related to age and disease but never linked to a progressive shift of the sympathovagal balance. We compare several well established estimates of complexity of heart period variability based on entropy rates [i.e., approximate entropy (ApEn), sample entropy (SampEn), and correct conditional entropy (CCE)] during an experimental protocol known to produce a gradual shift of the sympathovagal balance toward sympathetic activation and vagal withdrawal (i.e., the graded head-up tilt test). Complexity analysis was carried out in 17 healthy subjects over short heart period variability series ( approximately 250 cardiac beats) derived from ECG recordings during head-up tilt with table inclination randomly chosen inside the set {0, 15, 30, 45, 60, 75, 90}. We found that 1) ApEn does not change significantly during the protocol; 2) all indices measuring complexity based on entropy rates, including ad hoc corrections of the bias arising from their evaluation over short data sequences (i.e., corrected ApEn, SampEn, CCE), evidence a progressive decrease of complexity as a function of the tilt table inclination, thus indicating that complexity is under control of the autonomic nervous system; 3) corrected ApEn, SampEn, and CCE provide global indices that can be helpful to monitor sympathovagal balance.     

Automatic classification of interference patterns in driven event series: application to single sympathetic neuron discharge forced by mechanical ventilation

This study proposes a method for the automatic classification of nonlinear interactions between a strictly periodical event series modelling the activity of an exogenous oscillator working at a fixed and well-known rate and an event series modelling the activity of a self-sustained oscillator forced by the exogenous one. The method is based on a combination of several well-known tools (probability density function of the cyclic relative phase, probability density function of the count of forced events per forcing cycle, conditional entropy of the cyclic relative phase sequence and a surrogate data approach). Classification is reached via a sequence of easily applicable decision rules, thus rendering classification virtually user-independent and fully reproducible. The method classifies four types of dynamics: full uncoupling, quasiperiodicity, phase locking and aperiodicity. In the case of phase locking, the coupling ratio (i.e. n:m) and the strength of the coupling are calculated. The method, validated on simulations of simple and complex phase-locking dynamics corrupted by different levels of noise, is applied to data derived from one anesthetized and artificially ventilated rat to classify the nonlinear interactions between mechanical ventilation and: (1) the discharges of two (contemporaneously recorded) single postganglionic sympathetic neurons innervating the caudal ventral artery in the tail and (2) arterial blood pressure. Under central apnea, the activity of the underlying sympathetic oscillators is perturbed by means of five different lung inflation rates (0.58, 0.64, 0.76, 0.95, 1.99 Hz). While ventilation and arterial pressure are fully uncoupled, ventilation is capable of phase locking sympathetic discharges, thus producing 40% of phase-locked patterns (one case of 2:5, 1:1, 3:2 and 2:2) and 40% of aperiodic dynamics. In the case of phase-locked patterns, the coupling strength is low, thus demonstrating that this pattern is sliding. Non-stationary interactions are observed in 20% of cases. The two discharges behave differently, suggesting the presence of a population of sympathetic oscillators working at different frequencies.     

Classification of coupling patterns among spontaneous rhythms and ventilation in the sympathetic discharge of decerebrate cats

 The spontaneous low- and high-frequency rhythms in the sympathetic discharge of decerebrate artificially ventilated cats are affected by external ventilation. Two graphical methods (i.e. the space-time separation plot and the frequency tracking locus) are used to classify the non-linear interactions. The observed behaviours in the sympathetic discharge consist of phase-locked periodic dynamics (at various frequency ratios with ventilation), quasiperiodic and aperiodic patterns. They depend on the experimental condition. In control condition the sympathetic discharge appears more frequently locked to each ventilatory cycle (1 : 1 dynamics). However, some cases of quasiperiodic dynamics are found. A sympathetic activation stimulus, such as inferior vena cava occlusion, is able to synchronise slow rhythms in the sympathetic discharge to a subharmonic of ventilation. During a sympathetic inhibition stimulus, such as aortic constriction, 1 : 1 dynamics is detected but the amplitude of the sympathetic responses can be modulated by unlocked slow rhythms. Moreover, some cases of aperiodic dynamics are observed. Vagotomy reduces the 1 : 1 coupling between sympathetic outflow and ventilation. Vagotomy plus spinalisation disrupts periodic dynamics in the sympathetic discharge so that irregular and complex patterns are found. Received: 19 July 1995/Accepted in revised form: 20 May 1996     

Markov process of maintained impulse activity in central single neurons

To clarify the stochast properties of the maintained impulse activity of the central nervous system, we proposed a measure of statistical dependency on the basis of Shannon's entropy. This measure could provide the Markov properties of the neural impulse sequences, representing the necessary and sufficient condition for the statistical dependence. The order of Markov process of the sequence is determined by the conditional entropy which is derived from the joint entropy. Here the joint entropy in the case of Gaussian process is directly related with the covariance matrix which is substituted for the matrix of the serial correlation coefficients. Therefore the condition to determine the order of Markov process is obtained by the equation of the matrices of the serial correlation coefficients. The order of Markov process of the neural impulse sequences recorded from the mesencephalic reticular formation (MRF), red nucleus (RN), and lateral geniculate nucleus (LGN) neurons has been estimated. The maintained impulse activity of the MRF and RN neurons had from the 2-nd to 4-th order Markov property, while that of the LGN had no Markov property, in the consecutive impulse sequences.     

Medullary lateral tegmental field: control of respiratory rate and vagal lung inflation afferent influences on sympathetic nerve discharge

We used spectral analysis and event-triggered averaging to determine the effects of chemical inactivation of the medullary lateral tegmental field (LTF) on 1) the relationship of intratracheal pressure (ITP, an index of vagal lung inflation afferent activity) to sympathetic nerve discharge (SND) and phrenic nerve activity (PNA) and 2) central respiratory rate in paralyzed, artificially ventilated dial-urethane-anesthetized cats. ITP-SND coherence value at the frequency of artificial ventilation was significantly (P<0.05; n=18) reduced from 0.73+/-0.04 (mean+/-SE) to 0.24+/-0.04 after bilateral microinjection of muscimol into the LTF. Central respiratory rate was unexpectedly increased in 12 of these experiments (0.28+/-0.03 vs. 0.95+/-0.25 Hz). The ITP-PNA coherence value was variably affected by chemical inactivation of the LTF. It was unchanged when central respiratory rate was also not altered, decreased when respiratory rate was increased above the rate of artificial ventilation, and increased when respiratory rate was raised from a value below the rate of artificial ventilation to the same frequency as the ventilator. Chemical inactivation of the LTF increased central respiratory rate in four of six vagotomized cats but did not significantly affect the PNA-SND coherence value. These data demonstrate that the LTF 1) plays a critical role in mediating the effects of vagal lung inflation afferents on SND but not PNA, 2) helps maintain central respiratory rate in the physiological range, but 3) is not involved in the coupling of central respiratory and sympathetic circuits.     

Secretory process regularity monitors neuroendocrine feedback and feedforward signaling strength in humans

The present experiments examine the neuroregulatory hypothesis that the degree of sample-by-sample regularity of hormone output by an interlinked hypothalamopituitary target-organ system monitors the strength of feedback and/or feedforward signaling. To test this postulate and assess its generality, we implemented a total of nine thematically complementary perturbation experiments. In particular, we altered feedback or feedforward signaling selectively in two distinct neuroendocrine systems; namely, the growth hormone (GH) insulin-like growth factor type I (IGF-I) and the luteinizing hormone-testosterone axes. Four experimental paradigms comprised preferential reduction vs. enhancement of IGF-I or testosterone feedback signal strength; and, conversely, five others entailed selective attenuation vs. augmentation of GH-releasing hormone and gonadotropin-releasing hormone feedforward signal intensity. In these independent interventions, quantitation of subordinate (nonpulsatile) secretory pattern reproducibility via the approximate entropy statistic unmasked salient changes (P values typically <10(-3)) in the conditional regularity of serial hormone output with high consistency (96-100%). In particular, approximate entropy quantified degradation of secretory subpattern orderliness under either muted feedback restraint or heightened feedforward drive. Assuming valid interpretation of the biological constraints imposed, these experimental observations coincide with earlier reductionist mathematical predictions, wherein increased irregularity of coupled parameter output mirrors attenuated feedback and/or augmented feedforward coupling within an integrative system.     

Consensus methods for finding and ranking DNA binding sites. Application to Escherichia coli promoters

There have been many different approaches employed to define the "consensus" sequence of various DNA binding sites and to use the definition obtained to locate and rank members of a given sequence family. The analysis presented here enlists two of these approaches, each in modified form, to develop a highly efficient search protocol for Escherichia coli promoters and to provide a relative ranking of these sites showing good agreement with in vitro measurements of promoter strength. Schneider et al. have applied Shannon's index of information content to evaluate the significance of each position within the consensus of a family of aligned sequences. In a formal sense, this index is only applicable to a group of sequences, providing at each position a negative entropy value between zero (random) and two bits (total conservation of a single base) for sequences in which all bases are equally represented. A method for evaluating how well an individual sequence conforms to the information content pattern of the consensus is described. A function is derived, by analogy to the information content of the sequence family, for application to individual sequences. Since this function is a measure of conformity, it can be used in a search protocol to identify new members of the family represented by the consensus. A protocol for locating E. coli promoters is presented. The Berg-von Hippel statistical-mechanical function is also tested in a similar application. While the information content function provides a superior search protocol, the Berg-von Hippel function, when scaled at each position by the information content, does well at ranking promoters according to their strength as measured in vitro.     

Entropies of short binary sequences in heart period dynamics

Dynamic aspects of R-R intervals have often been analyzed by means of linear and nonlinear measures. The goal of this study was to analyze binary sequences, in which only the dynamic information is retained, by means of two different aspects of regularity. R-R interval sequences derived from 24-h electrocardiogram (ECG) recordings of 118 healthy subjects were converted to symbolic binary sequences that coded the beat-to-beat increase or decrease in the R-R interval. Shannon entropy was used to quantify the occurrence of short binary patterns (length N = 5) in binary sequences derived from 10-min intervals. The regularity of the short binary patterns was analyzed on the basis of approximate entropy (ApEn). ApEn had a linear dependence on mean R-R interval length, with increasing irregularity occurring at longer R-R interval length. Shannon entropy of the same sequences showed that the increase in irregularity is accompanied by a decrease in occurrence of some patterns. Taken together, these data indicate that irregular binary patterns are more probable when the mean R-R interval increases. The use of surrogate data confirmed a nonlinear component in the binary sequence. Analysis of two consecutive 24-h ECG recordings for each subject demonstrated good intraindividual reproducibility of the results. In conclusion, quantification of binary sequences derived from ECG recordings reveals properties that cannot be found using the full information of R-R interval sequences.     

Finite time thermodynamic coupling in a biochemical network

The paper describes some thermodynamic constrains and relations in biochemical or metabolic network and provides a basis for entropy enthalpy compensation. Conventional definition of macroscopic forces and fluxes leads to a paradox namely, non-existence of positive efficiency of a chemically driven process. This paradox is resolved by deriving an appropriate definition of macroscopic force using the local balance equations. Entropy enthalpy compensation, whose thermodynamic basis is so far unclear, also follows. The method provides an account of how reactive pathways are coupled, the strength of coupling between a pathway pair depending on the product of their respective enthalpies. The obligatory role of the presence of a common chemical intermediate in defining coupling becomes unnecessary; such intermediate-free coupling being a key feature of metabolic energy transduction. The redefined flux and force can also be exploited to explain surface to volume ratio dependence of coupled networks. Lastly, the thermodynamic rationale for the Bergman’s eco-geographic rule, namely the reduced ability of larger animals to avoid stress follows from the generalized expression for coupling coefficients. Higher surface to volume ratio is shown to make the organism resistant to external perturbations.     

Practical limitations on the use of thermodynamic data from isothermal processes

Enthalpy, entropy and volume data obtained for processes studied in aqueous solvents generally have been assumed to apply to the solute process without consideration of the coupling between the process and the two-state equilibrium of water. Walrafen's confirmation of the latter in 1983 shows that long-debated model to be correct so the enthalpy and entropy contributions to a free-energy change to give unambiguous information must be corrected for the water contribution. The situation is further complicated by differential chemical interaction of amphiphilic solutes with the two water species since experimental complications make correction difficult or impossible. A more general source of error in isothermal experiments is the linkage to the thermal-equilibrium device. That thermal problem discovered only in 1967 is not yet treated in textbooks although it is always a complication in isothermal processes and responsible for a hierarchy of thermodynamic quantities with different levels of reliability. Major consequences for several familiar thermodynamic and extra-thermodynamic methods are examined in terms of relative reliability. In most cases the thermal corrections are restricted by changes in phase state on cooling.     

The Burrows-Wheeler similarity distribution between biological sequences based on Burrows-Wheeler transform

This work aims at the similarity of biological sequences. Based on the Burrows-Wheeler transform, a definition of Burrows-Wheeler similarity distribution of two sequences is proposed to compare two sequences. Some distance measures are naturally followed by the distribution. The expectation and entropy of the similarity distribution are used to construct phylogenetic trees on two independent data sets. The result demonstrates that the method is efficient and powerful.     

The Shannon information entropy of protein sequences.

A comprehensive data base is analyzed to determine the Shannon information content of a protein sequence. This information entropy is estimated by three methods: a k-tuplet analysis, a generalized Zipf analysis, and a "Chou-Fasman gambler." The k-tuplet analysis is a "letter" analysis, based on conditional sequence probabilities. The generalized Zipf analysis demonstrates the statistical linguistic qualities of protein sequences and uses the "word" frequency to determine the Shannon entropy. The Zipf analysis and k-tuplet analysis give Shannon entropies of approximately 2.5 bits/amino acid. This entropy is much smaller than the value of 4.18 bits/amino acid obtained from the nonuniform composition of amino acids in proteins. The "Chou-Fasman" gambler is an algorithm based on the Chou-Fasman rules for protein structure. It uses both sequence and secondary structure information to guess at the number of possible amino acids that could appropriately substitute into a sequence. As in the case for the English language, the gambler algorithm gives significantly lower entropies than the k-tuplet analysis. Using these entropies, the number of most probable protein sequences can be calculated. The number of most probable protein sequences is much less than the number of possible sequences but is still much larger than the number of sequences thought to have existed throughout evolution. Implications of these results for mutagenesis experiments are discussed.     

Regular heartbeat dynamics are associated with cardiac health

The human heartbeat series is more variable and, hence, more complex in healthy subjects than in congestive heart failure (CHF) patients. However, little is known about the complexity of the heart rate variations on a beat-to-beat basis. We present an analysis based on symbolic dynamics that focuses on the dynamic features of such beat-to-beat variations on a small time scale. The sequence of acceleration and deceleration of eight successive heartbeats is represented by a binary sequence consisting of ones and zeros. The regularity of such binary patterns is quantified using approximate entropy (ApEn). Holter electrocardiograms from 30 healthy subjects, 15 patients with CHF, and their surrogate data were analyzed with respect to the regularity of such binary sequences. The results are compared with spectral analysis and ApEn of heart rate variability. Counterintuitively, healthy subjects show a large amount of regular beat-to-beat patterns in addition to a considerable amount of irregular patterns. CHF patients show a predominance of one regular beat-to-beat pattern (alternation of acceleration and deceleration), as well as some irregular patterns similar to the patterns observed in the surrogate data. In healthy subjects, regular beat-to-beat patterns reflect the physiological adaptation to different activities, i.e., sympathetic modulation, whereas irregular patterns may arise from parasympathetic modulation. The patterns observed in CHF patients indicate a largely reduced influence of the autonomic nervous system. In conclusion, analysis of short beat-to-beat patterns with respect to regularity leads to a considerable increase of information compared with spectral analysis or ApEn of heart-rate variations.     

Backbone entropy of loops as a measure of their flexibility: Application to a Ras protein simulated by molecular dynamics

The flexibility of surface loops plays an important role in protein–protein and protein–peptide recognition; it is commonly studied by Molecular Dynamics or Monte Carlo simulations. We propose to measure the relative backbone flexibility of loops by the difference in their backbone conformational entropies, which are calculated here with the local states (LS) method of Meirovitch. Thus, one can compare the entropies of loops of the same protein or, under certain simulation conditions, of different proteins. These loops should be equal in size but can differ in their sequence of amino acids residues. This methodology is applied successfully to three segments of 10 residues of a Ras protein simulated by the stochastic boundary molecular dynamics procedure. For the first time estimates of backbone entropy differences are obtained, and their correlation with B factors is pointed out; for example, the segments which consist of residues 60–65 and 112–117 have average B factors of 67 and 18 Ų, respectively, and entropy difference T ΔS = 5.4 ± 0.1 kcal/mol at T = 300 K. In a large number of recent publications the entropy due to the fast motions (on the ps-ns time scale) of N–H and C–H vectors has been obtained from their order parameter, measured in nuclear magnetic resonance spin relaxation experiments. This enables one to estimate differences in the entropy of protein segments due to folding–unfolding transitions, for example. However, the vectors are assumed to be independent, and the effect of the neglected correlations is unknown; our method is expected to become an important tool for assessing this approximation. The present calculations, obtained with the LS method, suggest that the errors involved in experimental entropy differences might not be large; however, this should be verified in each case. Potential applications of entropy calculations to rational drug design are discussed. Proteins 29:127–140, 1997. © 1997 Wiley-Liss, Inc.     

Characterization of nucleotidic sequences using maximum entropy techniques

A statistical method for characterizing nucleotidic sequences based on maximum entropy techniques is presented. The method uses only codon usage tables and takes into account the length of sequences, and preserves the information contained in each codon by a punctual index. We present the methodological aspects of the analysis, showing an application relative to nucleotidic sequences of eukaryotes.     

熵变视角的吉林省城市化与生态环境的耦合关系判别

借鉴热力学第二定律中的熵变方程原理,结合熵值法和PSE模型,对2001—2011年吉林省城市化发展与生态环境效益的耦合关系模式展开分析.结果表明: 研究期间,吉林省城市化发展脱离了正常的演进轨道,经济城市化和空间城市化主宰了整个城市化过程,而过于忽视“人”的城市化,且资源(这里主要指土地资源)粗放、低效投入特征明显;基于PSE模型的生态环境水平,无论是压力指数、敏感性指数、弹性指数,还是综合水平在研究期间都得到了长足的增长,且都呈现出明显的阶段性特征,即先升后降再升;城市化系统的总熵值一直小于0,表现为一条不平稳的上升曲线;生态环境的总熵值时正时负,其中,2003、2006、2010、2011年是“拐点”,映射出生态环境系统是一条不平稳的波浪形曲线,时升时降;城市化子系统与生态环境子系统交互耦合的总熵值具有与生态环境子系统总熵值相似的变化趋势,即具有相同的“拐点”和曲线形状,且其“拐点”对应的演化模式是拮抗型耦合关系演化模式,其余年份对应的是协调型耦合关系演化模式.     

Coupling thermodynamics applied in chemical morphogenesis; influence of environmentally organized space-time structures upon nonequilibrium entropy functions of multivariate biosystems

In this paper is proposed the so-called coupling thermodynamics which deals with coupled variables or elementary units which are influenced by input-output relations. In contrast to the usual definition of nonequilibrium entropy, external influences are considered directly in the entropy function dependent on space and time. The employment in morphology leads to a model of organized growth.     

一个基于熵的对疾病基因精细定位的指数

针对人类疾病基因的精细定位,本文利用稠密的标记位点,通过比较标记的熵和条件熵,给出了一个基于熵的指数。该指数可以度量标记基因和性状位点间连锁不平衡(LD)程度。该指数的特性是它不依赖于标记基因的频率。同时它对应疾病易感位点(DSL)精细定位的哈迪-温伯格不平衡(HWD)指数。通过计算机模拟,文章调查了不同遗传参数下该指数的性质。模拟结果表明该指数用作疾病易感位点精细定位是有效的。