The maximum entropy formalism and the idiosyncratic theory of biodiversity

Why does the neutral theory, which is based on unrealistic assumptions, predict diversity patterns so accurately? Answering questions like this requires a radical change in the way we tackle them. The large number of degrees of freedom of ecosystems pose a fundamental obstacle to mechanistic modelling. However, there are tools of statistical physics, such as the maximum entropy formalism (MaxEnt), that allow transcending particular models to simultaneously work with immense families of models with different rules and parameters, sharing only well‐established features. We applied MaxEnt allowing species to be ecologically idiosyncratic, instead of constraining them to be equivalent as the neutral theory does. The answer we found is that neutral models are just a subset of the majority of plausible models that lead to the same patterns. Small variations in these patterns naturally lead to the main classical species abundance distributions, which are thus unified in a single framework.     

Reinterpreting maximum entropy in ecology: a null hypothesis constrained by ecological mechanism

Simplified mechanistic models in ecology have been criticised for the fact that a good fit to data does not imply the mechanism is true: pattern does not equal process. In parallel, the maximum entropy principle (MaxEnt) has been applied in ecology to make predictions constrained by just a handful of state variables, like total abundance or species richness. But an outstanding question remains: what principle tells us which state variables to constrain? Here we attempt to solve both problems simultaneously, by translating a given set of mechanisms into the state variables to be used in MaxEnt, and then using this MaxEnt theory as a null model against which to compare mechanistic predictions. In particular, we identify the sufficient statistics needed to parametrise a given mechanistic model from data and use them as MaxEnt constraints. Our approach isolates exactly what mechanism is telling us over and above the state variables alone.     

Statistical mechanics unifies different ecological patterns

Recently there has been growing interest in the use of maximum relative entropy (MaxREnt) as a tool for statistical inference in ecology. In contrast, here we propose MaxREnt as a tool for applying statistical mechanics to ecology. We use MaxREnt to explain and predict species abundance patterns in ecological communities in terms of the most probable behaviour under given environmental constraints, in the same way that statistical mechanics explains and predicts the behaviour of thermodynamic systems. We show that MaxREnt unifies a number of different ecological patterns: (i) at relatively local scales a unimodal biodiversity-productivity relationship is predicted in good agreement with published data on grassland communities, (ii) the predicted relative frequency of rare vs. abundant species is very similar to the empirical lognormal distribution, (iii) both neutral and non-neutral species abundance patterns are explained, (iv) on larger scales a monotonic biodiversity-productivity relationship is predicted in agreement with the species-energy law, (v) energetic equivalence and power law self-thinning behaviour are predicted in resource-rich communities. We identify mathematical similarities between these ecological patterns and the behaviour of thermodynamic systems, and conclude that the explanation of ecological patterns is not unique to ecology but rather reflects the generic statistical behaviour of complex systems with many degrees of freedom under very general types of environmental constraints.     

Comparison of two views of maximum entropy in biodiversity: Frank (2011) and Pueyo et al. (2007)

An increasing number of authors agree in that the maximum entropy principle (MaxEnt) is essential for the understanding of macroecological patterns. However, there are subtle but crucial differences among the approaches by several of these authors. This poses a major obstacle for anyone interested in applying the methodology of MaxEnt in this context. In a recent publication, Frank (2011) gives some arguments why his own approach would represent an improvement as compared to the earlier paper by Pueyo et al. (2007) and also to the views by Edwin T. Jaynes, who first formulated MaxEnt in the context of statistical physics. Here I show that his criticisms are flawed and that there are fundamental reasons to prefer the original approach.     

A statistical explanation of MaxEnt for ecologists

MaxEnt is a program for modelling species distributions from presence‐only species records. This paper is written for ecologists and describes the MaxEnt model from a statistical perspective, making explicit links between the structure of the model, decisions required in producing a modelled distribution, and knowledge about the species and the data that might affect those decisions. To begin we discuss the characteristics of presence‐only data, highlighting implications for modelling distributions. We particularly focus on the problems of sample bias and lack of information on species prevalence. The keystone of the paper is a new statistical explanation of MaxEnt which shows that the model minimizes the relative entropy between two probability densities (one estimated from the presence data and one, from the landscape) defined in covariate space. For many users, this viewpoint is likely to be a more accessible way to understand the model than previous ones that rely on machine learning concepts. We then step through a detailed explanation of MaxEnt describing key components (e.g. covariates and features, and definition of the landscape extent), the mechanics of model fitting (e.g. feature selection, constraints and regularization) and outputs. Using case studies for a Banksia species native to south‐west Australia and a riverine fish, we fit models and interpret them, exploring why certain choices affect the result and what this means. The fish example illustrates use of the model with vector data for linear river segments rather than raster (gridded) data. Appropriate treatments for survey bias, unprojected data, locally restricted species, and predicting to environments outside the range of the training data are demonstrated, and new capabilities discussed. Online appendices include additional details of the model and the mathematical links between previous explanations and this one, example code and data, and further information on the case studies.     

Limitations of entropy maximization in ecology

Applying ideas of statistical mechanics in ecology have recently received quite some attention. The entropy maximization (EM) formalism looks particularly attractive, as it provides a simple algorithm to infer detailed system variables from a limited number of constraints. However, we point out that a blind application of this formalism can easily lead to wrong conclusions. To illustrate this, we reanalyze an ecological data set that has been used to claim the good performance of EM in predicting species abundances from trait measurements. We show that these results are entirely due to the restrictive constraints, and do not provide any support for the applicability of EM in ecology. By comparing with a simple example from physics, we indicate which characteristic mechanism of EM, and of statistical mechanics in general, is missing for the ecological example. This analysis introduces a series of methods to evaluate future attempts to apply EM in ecology.     

Phylogenetic beta diversity: linking ecological and evolutionary processes across space in time

A key challenge in ecological research is to integrate data from different scales to evaluate the ecological and evolutionary mechanisms that influence current patterns of biological diversity. We build on recent attempts to incorporate phylogenetic information into traditional diversity analyses and on existing research on beta diversity and phylogenetic community ecology. Phylogenetic beta diversity (phylobetadiversity) measures the phylogenetic distance among communities and as such allows us to connect local processes, such as biotic interactions and environmental filtering, with more regional processes including trait evolution and speciation. When combined with traditional measures of beta diversity, environmental gradient analyses or ecological niche modelling, phylobetadiversity can provide significant and novel insights into the mechanisms underlying current patterns of biological diversity.     

The scale dependence of fungal community distribution in paddy soil driven by stochastic and deterministic processes

Fungal communities play important roles in terrestrial ecosystem functioning. Unraveling the relative importance of stochastic versus deterministic processes in shaping biogeographic patterns of fungal communities has long been a challenge in microbial ecology, owing to high biodiversity and difficulties in identifying fungal taxa. Using a unique anthropogenic system of geographically isolated paddy ‘islands’, we collected 198 soil samples with a spatially explicit design to examine how ecological processes shaped fungal biogeographic patterns. Fungal community structure showed scale-dependent distance-decay relationships. Stochastic processes (dispersal and drift) contributed more to community assembly than deterministic processes (selection) at the local scale, which was largely attributed to drift. In contrast, deterministic processes contributed more to community assembly than stochastic processes at the regional scale, with soil dissolved organic carbon being the most important measured factor. Collectively, scale dependence of fungal biogeographical patterns in paddy soils is influenced by differential contribution of deterministic and stochastic processes.     

Ecological hierarchies and self-organisation – Pattern analysis,modelling and process integration across scales

A continuing discussion in applied and theoretical ecology focuses on the relationship of different organisational levels and on how ecological systems interact across scales. We address principal approaches to cope with complex across-level issues in ecology by applying elements of hierarchy theory and the theory of complex adaptive systems. A top-down approach, often characterised by the use of statistical techniques, can be applied to analyse large-scale dynamics and identify constraints exerted on lower levels. Current developments are illustrated with examples from the analysis of within-community spatial patterns and large-scale vegetation patterns. A bottom-up approach allows one to elucidate how interactions of individuals shape dynamics at higher levels in a self-organisation process; e.g., population development and community composition. This may be facilitated by various modelling tools, which provide the distinction between focal levels and resulting properties. For instance, resilience in grassland communities has been analysed with a cellular automaton approach, and the driving forces in rodent population oscillations have been identified with an agent-based model. Both modelling tools illustrate the principles of analysing higher level processes by representing the interactions of basic components.The focus of most ecological investigations on either top-down or bottom-up approaches may not be appropriate, if strong cross-scale relationships predominate. Here, we propose an ‘across-scale-approach’, closely interweaving the inherent potentials of both approaches. This combination of analytical and synthesising approaches will enable ecologists to establish a more coherent access to cross-level interactions in ecological systems.     

Community versus single‐species distribution models for British plants

Aim Species distribution models are increasingly used to predict the impacts of global change on whole ecological communities by modelling the individualistic niche responses of large numbers of species. However, it is not clear whether this single‐species ensemble approach is preferable to community‐wide strategies that represent interspecific associations or shared responses to environmental gradients. Here, we test the performance of two multi‐species modelling approaches against equivalent single‐species models. Location Great Britain. Methods Single‐ and multi‐species distribution models were fitted for 701 native British plant species at a 10‐km grid scale. Two machine learning methods were used – classification and regression trees (CARTs) and artificial neural networks (ANNs). The single‐species versions are widely used in ecology but their multivariate extensions are less well known and have not previously been evaluated against one another. We compared their abilities to predict species distributions, community compositions and species richness in an independent geographical region reserved from model‐fitting. Results The single‐ and multi‐species models performed similarly, although the community models gave slightly poorer predictive accuracy by all measures. However, from the point of view of the whole community they were much simpler than the array of single‐species models, involving orders of magnitude fewer parameters. Multi‐species approaches also left greater residual spatial autocorrelation than the individualistic models and, contrary to expectation, were relatively less accurate for rarer species. However, the fitted multi‐species response curves had lower tendency for pronounced discontinuities that are unlikely to be a feature of realized niche responses. Main conclusions Although community distribution models were slightly less accurate than single‐species models, they offered a highly simplified way of modelling spatial patterns in British plant diversity. Moreover, an advantage of the multi‐species approach was that the modelling of shared environmental responses resolved more realistic response curves. However, there was a slight tendency for community models to predict rare species less accurately, which is potentially disadvantageous for conservation applications. We conclude that multi‐species distribution models may have potential for understanding and predicting the structure of ecological communities, but were slightly inferior to single‐species ensembles for our data.     

Predicting habitat suitability for Castor fiber reintroduction: MaxEnt vs SWOT-Spatial multicriteria approach

Species reintroductions are high-investment ecological interventions that require feasibility studies and careful planning. In this context, Ecological Niche Models are a useful tool to identify habitat suitability, estimate spatiotemporal dynamics and manage reintroductions or conservation initiatives. The present paper proposes an application of Ecological Niche Models to explore potentials of the Italian territory for the reintroduction of the Eurasian beaver (Castor fiber), by identifying suitable areas and estimating the possible population harboured. The MaxEnt model was chosen from several modelling methods working with different statistical algorithms and inputs. MaxEnt was applied to the whole territory of the region of Piedmont and to two focus areas, the Natural Park of “La Mandria” and the Ivrea lakes area. The habitat suitability results were compared with previous research, conducted in 2019 that applied an integrated SWOT-Spatial multicriteria based approach for the construction of a Map of Suitable Areas for beavers reintroduction in Piedmont. The MaxEnt predictive maps show several potential habitats mostly located along main watercourses in foothill and in plain zones. Moreover, habitat distribution is comparable with the SWOT-Spatial Multicriteria Analsysis, although predictive maps could be minimally dependent on the choice of sample points within the map just mentioned. The MaxEnt model and, generally, the Ecological Niche models represent a useful tool to evaluate and select appropriate reintroduction sites and can support the complex problem of localization in reintroduction planning.     

Using generalized dissimilarity modelling to analyse and predict patterns of beta diversity in regional biodiversity assessment

Generalized dissimilarity modelling (GDM) is a statistical technique for analysing and predicting spatial patterns of turnover in community composition (beta diversity) across large regions. The approach is an extension of matrix regression, designed specifically to accommodate two types of nonlinearity commonly encountered in large-scaled ecological data sets: (1) the curvilinear relationship between increasing ecological distance, and observed compositional dissimilarity, between sites; and (2) the variation in the rate of compositional turnover at different positions along environmental gradients. GDM can be further adapted to accommodate special types of biological and environmental data including, for example, information on phylogenetic relationships between species and information on barriers to dispersal between geographical locations. The approach can be applied to a wide range of assessment activities including visualization of spatial patterns in community composition, constrained environmental classification, distributional modelling of species or community types, survey gap analysis, conservation assessment, and climate-change impact assessment.     

Statistical ecology comes of age

The desire to predict the consequences of global environmental change has been the driver towards more realistic models embracing the variability and uncertainties inherent in ecology. Statistical ecology has gelled over the past decade as a discipline that moves away from describing patterns towards modelling the ecological processes that generate these patterns. Following the fourth International Statistical Ecology Conference (1–4 July 2014) in Montpellier, France, we analyse current trends in statistical ecology. Important advances in the analysis of individual movement, and in the modelling of population dynamics and species distributions, are made possible by the increasing use of hierarchical and hidden process models. Exciting research perspectives include the development of methods to interpret citizen science data and of efficient, flexible computational algorithms for model fitting. Statistical ecology has come of age: it now provides a general and mathematically rigorous framework linking ecological theory and empirical data.     

Unifying conspecific-encounter index v and Moran's I index

Single-species distributional patterns have been well studied and quantified using a negative binomial model. However, it is still unclear how to quantify multi-species distribution patterns. A previous study (Chen et al. 2019) developed a simple conspecific-encounter index for quantifying multi-species distributional aggregation patterns when biodiversity data are collected in a consecutive scheme along line transects. Here, through simple derivation, we reported that the previously proposed conspecific-encounter index had a close association with Moran's I index, which has been widely applied in spatial ecology for describing spatial autocorrelation of ecological data. Our proof unified the conspecific-encounter index that is applied under the context of line-transect sampling and Moran's I index that is applied under the context of spatial ecology. The unification sheds light on the development of new statistical metrics for quantifying biological diversity and distribution patterns when line transects are used in field surveys.     

Accounting for dispersal and biotic interactions to disentangle the drivers of species distributions and their abundances

Although abiotic factors, together with dispersal and biotic interactions, are often suggested to explain the distribution of species and their abundances, species distribution models usually focus on abiotic factors only. We propose an integrative framework linking ecological theory, empirical data and statistical models to understand the distribution of species and their abundances together with the underlying community assembly dynamics. We illustrate our approach with 21 plant species in the French Alps. We show that a spatially nested modelling framework significantly improves the model's performance and that the spatial variations of species presence-absence and abundances are predominantly explained by different factors. We also show that incorporating abiotic, dispersal and biotic factors into the same model bring new insights to our understanding of community assembly. This approach, at the crossroads between community ecology and biogeography, is a promising avenue for a better understanding of species co-existence and biodiversity distribution.     

Empirical assessment of the relation between ecological connectivity and land complexity based on information-theoretic metrics

Habitat fragmentation and connectivity loss pose significant threats to biodiversity at both local and landscape levels. Strategies to increase ecological connectivity and preserve strong connectivity are important for dealing with the potential threat of habitat degradation. Various metrics have been used to measure (i.e., quantify) landscape composition and configuration in landscape ecology. However, their relationship with ecological connectivity must be understood to interpret landscape patterns comprehensively. In the present study, correlations between ecological connectivity and land complexity are examined based on information-theory metrics. Two primary questions are explored: (1) to what extent are landscape mosaic measures of entropy correlated with ecological connectivity, with landscape gradient-based measures, and with each other? (2) are landscape gradient-based entropy measures correlated with ecological connectivity more than discrete entropy measures? Results show that all information theoretic metrics are statistically significant (p < 0.05) for modelling ecological connectivity. Among categorically-based indices, the relationship between ECI and joint entropy was the most significant, while a generalized additive model indicated that Boltzmann entropy could predict the ecological connectivity index, explaining ∼60% of the variance. Therefore, configurational entropy can be used for improving ecological connectivity models.     

Movement‐mediated community assembly and coexistence

Organismal movement is ubiquitous and facilitates important ecological mechanisms that drive community and metacommunity composition and hence biodiversity. In most existing ecological theories and models in biodiversity research, movement is represented simplistically, ignoring the behavioural basis of movement and consequently the variation in behaviour at species and individual levels. However, as human endeavours modify climate and land use, the behavioural processes of organisms in response to these changes, including movement, become critical to understanding the resulting biodiversity loss. Here, we draw together research from different subdisciplines in ecology to understand the impact of individual‐level movement processes on community‐level patterns in species composition and coexistence. We join the movement ecology framework with the key concepts from metacommunity theory, community assembly and modern coexistence theory using the idea of micro–macro links, where various aspects of emergent movement behaviour scale up to local and regional patterns in species mobility and mobile‐link‐generated patterns in abiotic and biotic environmental conditions. These in turn influence both individual movement and, at ecological timescales, mechanisms such as dispersal limitation, environmental filtering, and niche partitioning. We conclude by highlighting challenges to and promising future avenues for data generation, data analysis and complementary modelling approaches and provide a brief outlook on how a new behaviour‐based view on movement becomes important in understanding the responses of communities under ongoing environmental change.     

Opportunities for improved distribution modelling practice via a strict maximum likelihood interpretation of MaxEnt

Maximum entropy (MaxEnt) modelling, as implemented in the Maxent software, has rapidly become one of the most popular methods for distribution modelling. Originally, MaxEnt was described as a machine‐learning method. More recently, it has been explained from principles of Bayesian estimation. MaxEnt offers numerous options (variants of the method) and settings (tuning of parameters) to the users. A widespread practice of accepting the Maxent software's default options and settings has been established, most likely because of ecologists’ lack of familiarity with machine‐learning and Bayesian statistical concepts and the ease by which the default models are obtained in Maxent. However, these defaults have been shown, in many cases, to be suboptimal and exploration of alternatives has repeatedly been called for. In this paper, we derive MaxEnt from strict maximum likelihood principles, and point out parallels between MaxEnt and standard modelling tools like generalised linear models (GLM). Furthermore, we describe several new options opened by this new derivation of MaxEnt, which may improve MaxEnt practice. The most important of these is the option for selecting variables by subset selection methods instead of the ?₁‐regularisation method, which currently is the Maxent software default. Other new options include: incorporation of new transformations of explanatory variables and user control of the transformation process; improved variable contribution measures and options for variation partitioning; and improved output prediction formats. The new options are exemplified for a data set for the plant species Scorzonera humilis in SE Norway, which was analysed by the standard MaxEnt procedure in a previously published paper. We recommend that thorough comparisons between the proposed alternative options and default procedures and variants thereof be carried out.     

Convolutional neural networks improve species distribution modelling by capturing the spatial structure of the environment

Convolutional Neural Networks (CNNs) are statistical models suited for learning complex visual patterns. In the context of Species Distribution Models (SDM) and in line with predictions of landscape ecology and island biogeography, CNN could grasp how local landscape structure affects prediction of species occurrence in SDMs. The prediction can thus reflect the signatures of entangled ecological processes. Although previous machine-learning based SDMs can learn complex influences of environmental predictors, they cannot acknowledge the influence of environmental structure in local landscapes (hence denoted “punctual models”). In this study, we applied CNNs to a large dataset of plant occurrences in France (GBIF), on a large taxonomical scale, to predict ranked relative probability of species (by joint learning) to any geographical position. We examined the way local environmental landscapes improve prediction by performing alternative CNN models deprived of information on landscape heterogeneity and structure (“ablation experiments”). We found that the landscape structure around location crucially contributed to improve predictive performance of CNN-SDMs. CNN models can classify the predicted distributions of many species, as other joint modelling approaches, but they further prove efficient in identifying the influence of local environmental landscapes. CNN can then represent signatures of spatially structured environmental drivers. The prediction gain is noticeable for rare species, which open promising perspectives for biodiversity monitoring and conservation strategies. Therefore, the approach is of both theoretical and practical interest. We discuss the way to test hypotheses on the patterns learnt by CNN, which should be essential for further interpretation of the ecological processes at play.     

Temporal scaling of bacterial taxa is influenced by both stochastic and deterministic ecological factors

Microorganisms operate at a range of spatial and temporal scales acting as key drivers of ecosystem properties. Therefore, many key questions in microbial ecology require the consideration of both spatial and temporal scales. Spatial scaling, in particular the species-area relationship (SAR), has a long history in ecology and has recently been addressed in microbial ecology. However, the temporal analogue of the SAR, the species-time relationship, has received far less attention even in the science of general ecology. Here we focus upon the role of temporal scaling in microbial ecological patterns by coupling molecular characterization of bacterial communities in discrete island (bioreactor) systems with a macroecological approach. Our findings showed that the temporal scaling exponent (slope), and therefore taxa turnover of the bacterial taxa-time relationship decreased as selective pressure (industrial wastewater concentration) increased. Also, as the concentration of industrial wastewater increased across the bioreactors, we observed a gradual switch from stochastic community assembly to more deterministic (niche)-based considerations. The identification of broad-scale statistical patterns is particularly relevant to microbial ecology, as it is frequently difficult to identify individual species or their functions. In this study, we identify wide-reaching statistical patterns of diversity and show that they are shaped by the prevalent underlying ecological factors.