Entropy increase and information loss in Markov models of evolution

Markov models of evolution describe changes in the probability distribution of the trait values a population might exhibit. In consequence, they also describe how entropy and conditional entropy values evolve, and how the mutual information that characterizes the relation between an earlier and a later moment in a lineage’s history depends on how much time separates them. These models therefore provide an interesting perspective on questions that usually are considered in the foundations of physics—when and why does entropy increase and at what rates do changes in entropy take place? They also throw light on an important epistemological question: are there limits on what your observations of the present can tell you about the evolutionary past?     

An engineer's view on genetic information and biological evolution

We develop ideas on genome replication introduced in Battail [Europhys. Lett. 40 (1997) 343]. Starting with the hypothesis that the genome replication process uses error-correcting means, and the auxiliary one that nested codes are used to this end, we first review the concepts of redundancy and error-correcting codes. Then we show that these hypotheses imply that: distinct species exist with a hierarchical taxonomy, there is a trend of evolution towards complexity, and evolution proceeds by discrete jumps. At least the first two features above may be considered as biological facts so, in the absence of direct evidence, they provide an indirect proof in favour of the hypothesized error-correction system. The very high redundancy of genomes makes it possible. In order to explain how it is implemented, we suggest that soft codes and replication decoding, to be briefly described, are plausible candidates. Experimentally proven properties of long-range correlation of the DNA message substantiate this claim.     

Entropy in evolution

Daniel R. Brooks and E. O. Wiley have proposed a theory of evolution in which fitness is merely a rate determining factor. Evolution is driven by non-equilibrium processes which increase the entropy and information content of species together. Evolution can occur without environmental selection, since increased complexity and organization result from the likely capture at the species level of random variations produced at the chemical level. Speciation can occur as the result of variation within the species which decreases the probability of sharing genetic information. Critics of the Brooks-Wiley theory argue that they have abused terminology from information theory and t thermodynamics. In this paper I review the essentials of the theory, and give an account of hierarchical physical information systems within which the theory can be interpreted. I then show how the major conceptual objections can be answered.     

The evolution of genetic diversity.

The existence within natural populations of large amounts of genetic variation in molecules and morphology presents an evolutionary problem. The 'neutralist' solution to this problem, that the variation is usually unimportant to the organism displaying it, has now lost much of its strength. Interpretations that assume widespread heterozygous advantage also face serious difficulties. A resolution is possible in terms of frequency-dependent selection by predators, parasites and competitors. The evidence for pervasive frequency-dependent selection is now very strong. It appears to follow naturally from the behaviour of predators, from the evolutionary lability of parasites, from the ecology of competition and, at the molecular level, from the phenomena of enzyme kinetics. Such selection can explain the maintenance not only of conventional polymorphism but also of continuous variation in both molecular and morphological characters. It can account for the occurrence of diversity within groups of haploid and self-fertilizing organisms, and for the evolution of differences between individuals in their systems of genetic control.     

Speculations on the evolution of the genetic code.

An evolutionary scheme is postulated in which the bases enter the genetic code in a definite temporal sequence and the correlated amino acids are assigned definite functions in the evolving system. The scheme requires a singlet code (guanine coding for glycine) evolving into a doublet code (guanine-cytosine doublet coding for gly (GG), ala (GC), arg (CG), pro (CC). The doublet code evolves into a triplet code. Polymerization of nucleotides is thought to have been by block polymerization rather than by a template mechanism. The proteins formed at first were simple structural peptides. No direct nucleotide-amino acid stereo-chemical interaction was required. Rather an adaptor-type indirect mechanism is thought to have been functioning since the origin.     

Entropy and information in evolving biological systems

Integrating concepts of maintenance and of origins is essential to explaining biological diversity. The unified theory of evolution attempts to find a common theme linking production rules inherent in biological systems, explaining the origin of biological order as a manifestation of the flow of energy and the flow of information on various spatial and temporal scales, with the recognition that natural selection is an evolutionarily relevant process. Biological systems persist in space and time by transfor ming energy from one state to another in a manner that generates structures which allows the system to continue to persist. Two classes of energetic transformations allow this; heat-generating transformations, resulting in a net loss of energy from the system, and conservative transformations, changing unusable energy into states that can be stored and used subsequently. All conservative transformations in biological systems are coupled with heat-generating transformations; hence, inherent biological production, or genealogical proesses, is positively entropic. There is a self-organizing phenomenology common to genealogical phenomena, which imparts an arrow of time to biological systems. Natural selection, which by itself is time-reversible, contributes to the organization of the self-organized genealogical trajectories. The interplay of genealogical (diversity-promoting) and selective (diversity-limiting) processes produces biological order to which the primary contribution is genealogical history. Dynamic changes occuring on times scales shorter than speciation rates are microevolutionary; those occuring on time scales longer than speciation rates are macroevolutionary. Macroevolutionary processes are neither redicible to, nor autonomous from, microevolutionary processes.Authorship alphabetical     

Entropy and the complexity of graphs: III. Graphs with prescribed information content

The connection between the adjacency matrix and the automorphisms of a digraph is used to develop a method for studying the automorphism group and, thus, the information content (Mowshowitz 1968a, b) of a digraph. An algorithm is given for constructing digraphs with zero information content, and the properties of such digraphs are examined. Moreover, an algorithm for computing the automorphism group of a digraph is presented and is used to find conditions which insure that two digraphs have the same information content. This algorithm is further used to determine the information content of digraphs whose adjacency matrices have prescribed properties.     

Recent evidence for evolution of the genetic code.

The genetic code, formerly thought to be frozen, is now known to be in a state of evolution. This was first shown in 1979 by Barrell et al. (G. Barrell, A. T. Bankier, and J. Drouin, Nature [London] 282:189-194, 1979), who found that the universal codons AUA (isoleucine) and UGA (stop) coded for methionine and tryptophan, respectively, in human mitochondria. Subsequent studies have shown that UGA codes for tryptophan in Mycoplasma spp. and in all nonplant mitochondria that have been examined. Universal stop codons UAA and UAG code for glutamine in ciliated protozoa (except Euplotes octacarinatus) and in a green alga, Acetabularia. E. octacarinatus uses UAA for stop and UGA for cysteine. Candida species, which are yeasts, use CUG (leucine) for serine. Other departures from the universal code, all in nonplant mitochondria, are CUN (leucine) for threonine (in yeasts), AAA (lysine) for asparagine (in platyhelminths and echinoderms), UAA (stop) for tyrosine (in planaria), and AGR (arginine) for serine (in several animal orders) and for stop (in vertebrates). We propose that the changes are typically preceded by loss of a codon from all coding sequences in an organism or organelle, often as a result of directional mutation pressure, accompanied by loss of the tRNA that translates the codon. The codon reappears later by conversion of another codon and emergence of a tRNA that translates the reappeared codon with a different assignment. Changes in release factors also contribute to these revised assignments. We also discuss the use of UGA (stop) as a selenocysteine codon and the early history of the code.     

Ambiguity and the evolution of the genetic code

The evolution of the genetic code is an extremely complex problem. The addition of a new method by which the code could evolve, however, allows much to be explained about the way in which the present codes (₃ and ₃ ) originated. The idea that ambiguity would allow the length of the codon to change is very useful, since it predicts the distribution of the 4-blocs and 2-blocs in the code, determines where variations in the code are probable, and presents a scenario for the evolution of the code.     

Protein evolution drives the evolution of the genetic code and vice versa

A model for the developmental pathway of the genetic code, grounded on group theory and the thermodynamics of codon-anticodon interaction is presented. At variance with previous models, it takes into account not only the optimization with respect to amino acid attributes but, also physicochemical constraints and initial conditions. A 'simple-first' rule is introduced after ranking the amino acids with respect to two current measures of chemical complexity. It is shown that a primeval code of only seven amino acids is enough to build functional proteins. It is assumed that these proteins drive the further expansion of the code. The proposed primeval code is compared with surrogate codes randomly generated and with another proposal for primeval code found in the literature. The departures from the 'universal' code, observed in many organisms and cellular compartments, fit naturally in the proposed evolutionary scheme. A strong correlation is found between, on one side, the two classes of aminoacyl-tRNA synthetases, and on the other, the amino acids grouped by end-atom-type and by codon type. An inverse of Davydov's rules, to associate the amino acid end atoms (O/N and non-O/non-N) of 18 amino acids with codons containing a weak base (A/U), extended to the 20 amino acids, is derived.     

A genetic code Boolean structure. II. The genetic information system as a Boolean information system

A Boolean structure of the genetic code where Boolean deductions have biological and physicochemical meanings was discussed in a previous paper. Now, from these Boolean deductions we propose to define the value of amino acid information in order to consider the genetic information system as a communication system and to introduce the semantic content of information ignored by the conventional information theory. In this proposal, the value of amino acid information is proportional to the molecular weight of amino acids with a proportional constant of about 1.96×10²⁵ bits per kg. In addition to this, for the experimental estimations of the minimum energy dissipation in genetic logic operations, we present two postulates: (1) the energy E _i (i = 1, 2, ..., 20) of amino acids in the messages conveyed by proteins is proportional to the value of information, and (2) amino acids are distributed according to their energy E _i so the amino acid population in proteins follows a Boltzmann distribution. Specifically, in the genetic message carried by the DNA from the genomes of living organisms, we found that the minimum energy dissipation in genetic logic operations was close to kTLn(2) joules per bit.     

Language evolution and information theory

This paper places models of language evolution within the framework of information theory. We study how signals become associated with meaning. If there is a probability of mistaking signals for each other, then evolution leads to an error limit: increasing the number of signals does not increase the fitness of a language beyond a certain limit. This error limit can be overcome by word formation: a linear increase of the word length leads to an exponential increase of the maximum fitness. We develop a general model of word formation and demonstrate the connection between the error limit and Shannon's noisy coding theorem.     

Local stability and evolution of the genetic code

The standard genetic code is known to be robust to translation errors and point mutations. We studied how small modifications of the standard code affect its robustness. The robustness was assessed in terms of a proper stability function, the negative variations of which correspond to a more robust code. The fraction of more robust codes obtained under small modifications appeared to be unexpectedly high, about 0.1-0.4 depending on the choice of stability function and code modifications, yet significantly lower than the corresponding fraction in the random codes (about a half). In this sense the standard code ought to be considered distinctly non-random in accordance with previous observations. The distribution of the negative variations of stability function revealed very abrupt drop beyond one standard deviation, much sharper than for Gaussian distribution or for the random codes with the same number of codons in the sets coding for amino acids or stop-codons. This behavior holds for both the standard code as a whole and its binary NRN-NYN, NWN-NSN, and NMN-NKN blocks. Previously, it has been proved that such binary block structure is necessary for the robustness of a code and is inherent to the standard genetic code. The modifications of the standard code corresponding to more robust coding may be related to the different variants of the code. These effects may also contribute to the rates of replacements of amino acids. The observed features demonstrate the joint impact of random factors and natural selection during evolution of the genetic code.     

Entropy and the complexity of graphs: II. The information content of digraphs and infinite graphs

In a previous paper (Mowshowitz, 1968), a measureI _g (X) of the structural information content of an (undirected) graphX was defined, and its properties explored. The class of graphs on whichI _g is defined is here enlarged to included directed graphs (digraphs). Most of the properties ofI _g observed in the undirected case are seen to hold for digraphs. The greater generality of digraphs allows for a construction which shows that there exists a digraph having information content equal to the entropy of an arbitrary partition of a given positive integer. The measureI _g is also extended to a measure defined on infinite (undirected) graphs. The properties of this extension are discussed, and its applicability to the problem of measuring the complexity of algorithms is considered.