Limitations of entropy maximization in ecology: a reply to Haegeman and Loreau

Haegeman and Loreau published a paper that is primarily a criticism of a maximum entropy model of trait-based community assembly (by Shipley et al.) and purports to show the limitations of this method in ecology. However, they misunderstood the basic purpose, logic and justification of the maximum entropy formalism and, because of this, leveled criticisms of Shipley et al. that are unfounded. Part of the confusion can be traced to sloppy presentation of the underlying approach in Shipley et al. The confusion arises because maximum entropy models are justified based on information theory and Bayesian logic while the interpretation that Haegeman and Loreau present is based on substantive empirical assumptions about microstate allocations and a combinatorial argument that do not apply to maximum entropy models and which I do not apply to my model in particular.     

Trivial and non-trivial applications of entropy maximization in ecology: a reply to Shipley

Entropy maximization (EM) is becoming an increasingly popular modelling technique in ecology, but its potential and limitations are still poorly understood. In our previous contribution (Haegeman and Loreau 2008), we showed that even a trivial application of EM can yield predictions that provide an excellent fit to empirical data. In his response, Shipley (2009) distinguishes two different versions of the EM procedure, an information-theoretical version and a combinatorial version, to justify a trivial application of EM. Here we first provide a brief user's guide to EM to clarify the various steps involved in the procedure. We then show that the information-theoretical and combinatorial rationales for EM are but complementary views on the same procedure. Lastly, we attempt to identify the conditions that lead to trivial and non-trivial applications of EM. We discuss how non-trivial applications of EM can yield valuable new insights in ecology.     

Power-law inter-spike interval distributions infer a conditional maximization of entropy in cortical neurons

The brain is considered to use a relatively small amount of energy for its efficient information processing. Under a severe restriction on the energy consumption, the maximization of mutual information (MMI), which is adequate for designing artificial processing machines, may not suit for the brain. The MMI attempts to send information as accurate as possible and this usually requires a sufficient energy supply for establishing clearly discretized communication bands. Here, we derive an alternative hypothesis for neural code from the neuronal activities recorded juxtacellularly in the sensorimotor cortex of behaving rats. Our hypothesis states that in vivo cortical neurons maximize the entropy of neuronal firing under two constraints, one limiting the energy consumption (as assumed previously) and one restricting the uncertainty in output spike sequences at given firing rate. Thus, the conditional maximization of firing-rate entropy (CMFE) solves a tradeoff between the energy cost and noise in neuronal response. In short, the CMFE sends a rich variety of information through broader communication bands (i.e., widely distributed firing rates) at the cost of accuracy. We demonstrate that the CMFE is reflected in the long-tailed, typically power law, distributions of inter-spike intervals obtained for the majority of recorded neurons. In other words, the power-law tails are more consistent with the CMFE rather than the MMI. Thus, we propose the mathematical principle by which cortical neurons may represent information about synaptic input into their output spike trains.     

最大熵原理及其在生态学研究中的应用

最大熵原理(the principle of maximum entropy)起源于信息论和统计力学,是基于有限的已知信息对未知分布进行无偏推断的一种数学方法.这一方法在很多领域都有成功应用,但只是近几年才被应用到生态学研究中,并且还存在很多争论.我们从基本概念和方法出发,用掷骰子的例子阐明了最大熵原理的概念,并提出运用最大熵原理解决问题需要遵从的步骤.最大熵原理在生态学中的应用主要包括以下方面:(1)用群落水平功能性状的平均值作为约束条件来预测群落物种相对多度的模型;(2)基于气候、海拔、植被等环境因子构建物种地理分布的生态位模型;(3)对物种多度分布、种一面积关系等宏生态学格局的推断;(4)对物种相互作用的推断;(5)对食物网度分布的研究等等.最后我们综合分析了最大熵原理在生态学应用中所存在的争议,包括相应模型的有效性、可靠性等方面,介绍了一些对最大熵原理预测能力及其局限性的检验结果,强调了生态学家应用最大熵原理需要注意的问题,比如先验分布的选择、约束条件的设置等等.在物种相互作用、宏生态学格局等方面对最大熵原理更广泛的讨论与应用可能会给生态学带来新的发展机会.     

Biological diversity: Distinct distributions can lead to the maximization of Rao’s quadratic entropy

Rao’s quadratic entropy (QE) is a diversity index that includes the abundances of categories (e.g. alleles, species) and distances between them. Here we show that, once the distances between categories are fixed, QE can be maximized with a reduced number of categories and by several different distributions of relative abundances of the categories. It is shown that Rao’s coefficient of distance (DISC), based on QE, can equal zero between two maximizing distributions, even if they have no categories in common. The consequences of these findings on the relevance of QE for understanding biological diversity are evaluated in three case studies. The behaviour of QE at its maximum is shown to be strongly dependent on the distance metric. We emphasize that the study of the maximization of a diversity index can bring clarity to what exactly is measured and enhance our understanding of biological diversity.     

Reinterpreting maximum entropy in ecology: a null hypothesis constrained by ecological mechanism

Simplified mechanistic models in ecology have been criticised for the fact that a good fit to data does not imply the mechanism is true: pattern does not equal process. In parallel, the maximum entropy principle (MaxEnt) has been applied in ecology to make predictions constrained by just a handful of state variables, like total abundance or species richness. But an outstanding question remains: what principle tells us which state variables to constrain? Here we attempt to solve both problems simultaneously, by translating a given set of mechanisms into the state variables to be used in MaxEnt, and then using this MaxEnt theory as a null model against which to compare mechanistic predictions. In particular, we identify the sufficient statistics needed to parametrise a given mechanistic model from data and use them as MaxEnt constraints. Our approach isolates exactly what mechanism is telling us over and above the state variables alone.     

Foraging ecology of bison at the landscape and plant community levels: the applicability of energy maximization principles

Predictions of animal distribution and resource use require multi-scale consideration because animals can use different sets of selection criteria at different scales. We investigated whether patterns of distribution and resource use by free-ranging bison (Bison bison) in Prince Albert National Park, Saskatchewan, follow rules of energy maximization that hold across multiple scales. Optimality theory predicts specialization on Carex atherodes and frequency-independent selection among plant species; that is, local variation in C. atherodes biomass should not influence diet choice but only the time spent in individual patches. The summer use of resources within meadows was closely related to energy maximization principles. C. atherodes dominated the diet of bison, was selected in all meadows, and diet choice was frequency-independent among meadows in the bison range. In winter, diet was still dominated by C. atherodes, but frequency-dependent selection of Scolochloa festucacea and the relative use of Cirsium arvense were inconsistent with theoretical predictions. At a larger spatial scale, however, the probability of meadow use was not positively related to the abundance of Carex atherodes. During summer and winter, general landscape features within the daily radius of bison (2 km in summer and 1.3 km in winter), together with abiotic characteristics of meadows, had the major influence on the probability of meadow utilization. Our study suggests that bison distribution and resource use are influenced by abiotic and biotic factors which vary in importance at different spatio-temporal scales.     

Tinnitus-like “hallucinations” elicited by sensory deprivation in an entropy maximization recurrent neural network

Sensory deprivation has long been known to cause hallucinations or “phantom” sensations, the most common of which is tinnitus induced by hearing loss, affecting 10–20% of the population. An observable hearing loss, causing auditory sensory deprivation over a band of frequencies, is present in over 90% of people with tinnitus. Existing plasticity-based computational models for tinnitus are usually driven by homeostatic mechanisms, modeled to fit phenomenological findings. Here, we use an objective-driven learning algorithm to model an early auditory processing neuronal network, e.g., in the dorsal cochlear nucleus. The learning algorithm maximizes the network’s output entropy by learning the feed-forward and recurrent interactions in the model. We show that the connectivity patterns and responses learned by the model display several hallmarks of early auditory neuronal networks. We further demonstrate that attenuation of peripheral inputs drives the recurrent network towards its critical point and transition into a tinnitus-like state. In this state, the network activity resembles responses to genuine inputs even in the absence of external stimulation, namely, it “hallucinates” auditory responses. These findings demonstrate how objective-driven plasticity mechanisms that normally act to optimize the network’s input representation can also elicit pathologies such as tinnitus as a result of sensory deprivation.     

Thermodynamics and muscle contraction. II. Consequences of the principle of maximization of the rate of entropy production]

For non-elastic irreversible mechanochemical machines of the muscular type, the above-mentioned principle (by H. Ziegler) leads to the following results. The velocity of shortening, v, decreases when the force F increases. The mechanical power of the system Fv, is active when 0 greater than v greater than vF=0 or 0 greater than F greater than Fv=0. At F=0 the rate of chemical reaction has the maximum and dQ/dF = -v greater than 0 (where Q is the rate of heat production). At v = 0 the rate of entropy production has the minimum and d/dv = F/A greater than 0 (where A is an affinity of the chemical reaction). The Onsager symmetry relation holds at v=0 only.     

Time-based reward maximization

Humans and animals time intervals from seconds to minutes with high accuracy but limited precision. Consequently, time-based decisions are inevitably subjected to our endogenous timing uncertainty, and thus require temporal risk assessment. In this study, we tested temporal risk assessment ability of humans when participants had to withhold each subsequent response for a minimum duration to earn reward and each response reset the trial time. Premature responses were not penalized in Experiment 1 but were penalized in Experiment 2. Participants tried to maximize reward within a fixed session time (over eight sessions) by pressing a key. No instructions were provided regarding the task rules/parameters. We evaluated empirical performance within the framework of optimality that was based on the level of endogenous timing uncertainty and the payoff structure. Participants nearly tracked the optimal target inter-response times (IRTs) that changed as a function of the level of timing uncertainty and maximized the reward rate in both experiments. Acquisition of optimal target IRT was rapid and abrupt without any further improvement or worsening. These results constitute an example of optimal temporal risk assessment performance in a task that required finding the optimal trade-off between the ‘speed’ (timing) and ‘accuracy’ (reward probability) of timed responses for reward maximization.     

Cell-mass maximization in fed-batch cultures

Complete solutions are provided for cell-mass maximization for free and fixed final times and constant and variable yields. The optimal feed rate profile is a concatenation of maximum, minimum and singular feed rates. The exact sequence and duration of each feed rate depends primarily on the initial substrate concentration, and degenerate cases arise due to the magnitude constraint on the feed rate and the length of final time t _f. When the final time is free and not in the performance index, it is infinite for constant yield so that any form of feed rate leads to the same amount of cells, while for variable yield the singular feed rate is exponential and maximizes the yield. For fixed final time the singular feed rate for constant yield is exponential and maximizes the specific growth rate by maintaining the substrate concentration constant, while for variable yield, it is semi-exponential and the substrate concentration starts near the maximum specific growth rate and moves toward the maximum yield. A simple sufficient condition for existence of singular feed rate requires an existence of a region bounded by the maxima of specific growth and cellular yield. Otherwise, the optimal feed rate profile is a bang-bang type and the bioreactor operates in batch mode.     

Violations of transitivity under fitness maximization

We present a novel demonstration that violations of transitive choice can result from decision strategies that maximize fitness. Our results depend on how the available options, including options not currently chosen, influence a decision-maker's expectations about the future. In particular, they depend on how the presence of an option may act as an insurance against a run of bad luck in the future.     

A plea for more ecology in phytoplankton ecology

When looking for a pattern of phytoplankton behaviour across trophic gradients, we need to cross the boundaries between different disciplinary areas, from autoecology to systems ecology, because eutrophication is a complex process which involves different time scales and different levels of community structure. Thus, we submit our observations to the muddled conceptual world of assemblage ecology. These inaccuracies arise, for example, from both species and community arguments; eutrophication as a fertilization or a metabolic phenomenon; and the notions frequently interwoven of pattern, process and rules. We suggest that it is advantageous to tackle this issue from the perspective of general ecology, rather than from a specifically planktonic orientation. In this way, useful general ecological tools, for example, time series and assembly-rule studies, can be used. Time-series study allows the dynamics of any variable to be described or to show that long term variable fluctuations may sometimes be unregulated, in response to some exogenous factor. Rules of assembly help us to resolve which traits are selectively involved during the eutrophication process. In this context, we advocate (1) the use of traits instead of morphospecies in phytoplankton studies, (2) looking for the dynamic patterns of phytoplankton with eutrophication, (3) the use of time series techniques to study phytoplankton trajectories, (4) the use of assembly rules to discern patterns in the formation of multispecies assemblages, (5) the consideration of the pelagic food-web in studies of phytoplankton dynamics and, as an overall suggestion, to borrow knowledge and inspiration from general ecology.