Conceptual and statistical problems with the use of the Shannon-Weiner entropy index in bioacoustic analyses

Information theory and its indices were developed for human communication to predict the amount of information transferred in a message. One such index, the Shannon-Weiner index (SWI), has often been used to analyse information from other fields in which its application may not be appropriate. In ecoacoustics, SWI is used to compare acoustic diversity (i.e. a measure derived by integrating the richness and abundance of animal sounds) between locations. In animal communication, SWI is used to quantify repertoire complexity (i.e. a measure derived by integrating the number and abundance of sound types produced by individuals or species) as an approach to understanding signal evolution. We discuss problems associated with using the SWI in ecoacoustics and animal communication. Specifically, we discuss conceptual and statistical problems associated with the SWI and then illustrate these problems using hypothetical data. In ecoacoustics, the SWI’s assumptions of random variables and independent samples are often violated. In animal communication, the SWI fails to distinguish among repertoires in which the number of sound types and the abundance of each sound type differ. We also show that other methods do capture these differences. We conclude that the SWI does not adequately represent acoustic diversity or repertoire complexity due to the multiple conceptual and statistical issues associated with its use. We recommend other analytical methods to more fully describe these biological systems, including goodness of fit, Morisita similarity index and Markov chain analysis. These methods provide more information for future comparisons and permit researchers to test hypotheses more directly.     

Simultaneous confidence intervals for comparing biodiversity indices estimated from overdispersed count data

Diversity indices might be used to assess the impact of treatments on the relative abundance patterns in species communities. When several treatments are to be compared, simultaneous confidence intervals for the differences of diversity indices between treatments may be used. The simultaneous confidence interval methods described until now are either constructed or validated under the assumption of the multinomial distribution for the abundance counts. Motivated by four example data sets with background in agricultural and marine ecology, we focus on the situation when available replications show that the count data exhibit extra‐multinomial variability. Based on simulated overdispersed count data, we compare previously proposed methods assuming multinomial distribution, a method assuming normal distribution for the replicated observations of the diversity indices and three different bootstrap methods to construct simultaneous confidence intervals for multiple differences of Simpson and Shannon diversity indices. The focus of the simulation study is on comparisons to a control group. The severe failure of asymptotic multinomial methods in overdispersed settings is illustrated. Among the bootstrap methods, the widely known Westfall–Young method performs best for the Simpson index, while for the Shannon index, two methods based on stratified bootstrap and summed count data are preferable. The methods application is illustrated for an example.     

A new statistical approach for assessing similarity of species composition with incidence and abundance data

The classic Jaccard and Sørensen indices of compositional similarity (and other indices that depend upon the same variables) are notoriously sensitive to sample size, especially for assemblages with numerous rare species. Further, because these indices are based solely on presence–absence data, accurate estimators for them are unattainable. We provide a probabilistic derivation for the classic, incidence‐based forms of these indices and extend this approach to formulate new Jaccard‐type or Sørensen‐type indices based on species abundance data. We then propose estimators for these indices that include the effect of unseen shared species, based on either (replicated) incidence‐ or abundance‐based sample data. In sampling simulations, these new estimators prove to be considerably less biased than classic indices when a substantial proportion of species are missing from samples. Based on species‐rich empirical datasets, we show how incorporating the effect of unseen shared species not only increases accuracy but also can change the interpretation of results.     

Similarity indices,sample size and diversity

Summary The effect of sample size and species diversity on a variety of similarity indices is explored. Real values of a similarity index must be evaluated relative to the expected maximum value of that index, which is the value obtained for samples randomly drawn from the same universe, with the diversity and sample sizes of the real samples. It is shown that these expected maxima differ from the theoretical maxima, the values obtained for two identical samples, and that the relationship between expected and theoretical maxima depends on sample size and on species diversity in all cases, without exception. In all cases but one (the Morisita index) the expected maxima depend strongly to fairly strongly on sample size and diversity. For some of the more useful indices empirical equations are given to calculate the expected maximum value of the indices to which the observed values can be related at any combination of sample sizes. It is recommended that the Morisita index be used whenever possible to avoid the complex dealings with effects of sample size and diversity; however, when previous logarithmic transformation of the data is required, which often may be the case, the Morisita-Horn or the Renkonen indices are recommended.     

Why do traditional dispersion indices used for analysis of spatial distribution of plants tend to become obsolete?

It is common to characterize the spatial distribution of plant patterns as random, aggregate, or uniform. In this context, a major challenge for the researcher is the choice of the method to identify the spatial pattern correctly as well as the factors related to it. The vast literature on the subject is not recent, especially regarding the dispersion indices. The aim of this review was to conduct a critical and temporal analysis of these dispersion indices and test their effectiveness in determining the spatial distribution of Paepalanthus chiquitensis Herzog (Eriocaulaceae). This species is a meaningful model due to its occurrence in specific sites. The Lexis, Charlier, dispersion, relative variance, aggregation, Green, inverse of k of the negative binomial, Morisita, and standardized Morisita indices were limited to indicating that the individuals of the species are aggregate and did not provide information on neither spatial dimension (scale) where the aggregation occurs, nor the factors related to this aggregation. Although they have distinct magnitudes, the algebraic expressions of dispersion, relative variance, aggregation, Green, inverse of k, Morisita, and standardized Morisita indices exhibited a close relationship with each other and little progress from their precursors Lexis and Charlier. By disregarding the possibility of spatial dependence, these indices make it impossible to generate important hypotheses for the investigation of factors related to spatial structure. Therefore, they became obsolete and are falling into disuse. It should be noted that these measurements accomplished their role and contributed to science in times of limited technologies for spatial data.     

Abundance-based similarity indices and their estimation when there are unseen species in samples

A wide variety of similarity indices for comparing two assemblages based on species incidence (i.e., presence/absence) data have been proposed in the literature. These indices are generally based on three simple incidence counts: the number of species shared by two assemblages and the number of species unique to each of them. We provide a new probabilistic derivation for any incidence-based index that is symmetric (i.e., the index is not affected by the identity ordering of the two assemblages) and homogeneous (i.e., the index is unchanged if all counts are multiplied by a constant). The probabilistic approach is further extended to formulate abundance-based indices. Thus any symmetric and homogeneous incidence index can be easily modified to an abundance-type version. Applying the Laplace approximation formulas, we propose estimators that adjust for the effect of unseen shared species on our abundance-based indices. Simulation results show that the adjusted estimators significantly reduce the biases of the corresponding unadjusted ones when a substantial fraction of species is missing from samples. Data on successional vegetation in six tropical forests are used for illustration. Advantages and disadvantages of some commonly applied indices are briefly discussed.     

Genetic relationships among Neisseria species assessed by comparative enzyme electrophoresis

The electrophoretic mobilities of 12 enzymes from 19 Neisseria species (including 6 strains of N. perflava), Gemella haemolysans, Escherichia coli and Branhamella catarrhalis were characterized by polyacrylamide slab gel electrophoresis. All strains and species tested exhibited qualitatively different zymogram patterns. Species and strain relationships were quantified by pairwise comparisons of all 12 enzyme systems to obtain similarity indices; these data were subjected to numerical clustering methods to obtain groups and a phenogram. The electrophoretic classification compared favourably with those obtained by other criteria. In addition, the quantitative clustering data indicated that N. ovis and N. caviae are sufficiently different from the other Neisseria species to warrant their separation into a distinct group. These two species also lacked the characteristic NADPH-diaphorase zymogram pattern found in all the other Neisseria species. Intra-species similarity indices were generally greater than the inter-species index values. However, certain species such as N. meningitidis and N. gonorrhoeae had similarity index values in the range of inter-strain index values.     

A similarity measure sensitive to the contribution of rare species and its use in investigation of variation in marine benthic communities

Summary We propose a new measure of similarity, the normalized expected species shared or NESS. The measure is based on the expected number of species shared between random samples of size, m, drawn from a population. The NESS measure is shown to be a generalization of Morisita's similarity and is demonstrated to be less biased than other commonly used measures. The contribution of dominant and rare species is explicit according to the sample size, m, chosen. For large m, NESS is sensitive to the less common species in the populations to be compared.The NESS measure has been used to cluster a 2-year sequence of subtidal benthic samples taken after a severe disturbance. The NESS measure is responsive to the less common species so that both a temporal progression in community recovery and seasonal variation are revealed.Contribution No. 3699 from the Woods Hole Oceanographic Institution     

Ecological quality scales based on phytoplankton for the implementation of Water Framework Directive in the Eastern Mediterranean

Structural changes of phytoplankton communities, often expressed through ecological indices, constitute one of the metrics for the implementation of the European Water Framework Directive (WFD). In the current study a thorough analysis of the efficiency of 22 ecological indices was performed and a small number was selected for the development of five-level water quality scales (High, Good, Moderate, Poor, and Bad). The analysis was performed on simulated communities free of the noise of field communities due to uncontrolled factors or stochastic processes. Two criteria were set for the sensitivity of indices, namely their monotonicity and linearity across the studied eutrophication spectrum. The whole procedure was based on the development of a five-level quality assessment scheme based on phytoplankton abundance. Among the indices tested, the Menhinick diversity index and three indices of evenness were the most efficient, showing consistency (monotonic behavior) and linearity and were therefore used for the development of quality scales for the WFD. An Integrated Phytoplankton Index (IPI) based on three phytoplankton metrics, chlorophyll a, abundance, and diversity is also proposed. The efficiency of these indices was evaluated for a number of sites in the Aegean, already classified in the past by various methods based on nutrient concentrations or phytoplankton data. The results indicate that the various phytoplankton metrics (chlorophyll a, abundance, and diversity) assessed or proposed in the current study, carry their own information showing differences in the final classification of areas. Therefore the establishment of synthetic indices as the IPI seems to be advantageous for the integrated assessment of coastal water quality in the framework of European policies as the WFD.     

Quantifying sexual selection: a comparison of competing indices with mating system data from a terrestrially breeding salamander

Calculations for quantifying the potential for sexual selection remain controversial. Many indices have been promoted in the literature, but each has unique sets of advantages and disadvantages. Using marbled salamanders, I evaluated the performance of several measures by manipulating intensity of sexual selection in experimental breeding replicates of varying operational sex ratio. Theory predicts that sexual selection among males will be higher when sex ratio is male‐biased and lower when female‐biased. I used microsatellite data to assign hatchling parentage, estimate adult fitness, and calculate several indices of inequality for quantifying sexual selection. Opportunity for selection and Morisita index always conformed to theoretical expectations, which was not the case for index of resource monopolization, standardized Morisita index, or binomial skew index. Although I conclude that opportunity for selection is advantageous in sexual selection studies because of its link to formal theory, this should be tested against the null hypothesis of random variation in ambiguous cases. In the present study, although variation in both reproductive and mating success was high when quantified using opportunities for selection, it was only significantly greater than random expectations for reproductive success. This study provides further empirical support for the continued use of opportunity for selection in sexual selection studies. © 2010 The Linnean Society of London, Biological Journal of the Linnean Society, 2010, 99 , 73–83.     

The Role of Landscape Patterns of Habitat Types on Plant Species Diversity of a Tropical Forest in Mexico

The relationships among landscape characteristics and plant diversity in tropical forests may be used to predict biodiversity. To identify and characterize them, the number of species, as well as Shannon and Simpson diversity indices were calculated from 157 sampling quadrats (17,941 individuals sampled) while the vegetation classes were obtained from multi-spectral satellite image classification in four landscapes located in the southeast of Quintana Roo, Mexico. The mean number of species of trees, shrubs and vines as well as the mean value of the total number of species and the other two diversity indices were calculated for four vegetation classes in every one of the four landscapes. In addition, the relationships between landscape patterns metrics of patch types and diversity indices were explored. The multiple statistical analyses revealed significant predictor variables for the three diversity indices. Moreover, the shape, similarity and edge contrast metrics of patch types might serve as useful indicators for the number of species and the other two diversity variables at the landscape scale. Although the association between the three diversity indices and patch types metrics showed similar behavior, some differences were appreciated. The Shannon diversity index, with its greater sensitivity to rare species, should be considered as having a greater importance in interpretation analysis than Simpson index.     

Distance-Based Functional Diversity Measures and Their Decomposition: A Framework Based on Hill Numbers

Hill numbers (or the “effective number of species”) are increasingly used to characterize species diversity of an assemblage. This work extends Hill numbers to incorporate species pairwise functional distances calculated from species traits. We derive a parametric class of functional Hill numbers, which quantify “the effective number of equally abundant and (functionally) equally distinct species” in an assemblage. We also propose a class of mean functional diversity (per species), which quantifies the effective sum of functional distances between a fixed species to all other species. The product of the functional Hill number and the mean functional diversity thus quantifies the (total) functional diversity, i.e., the effective total distance between species of the assemblage. The three measures (functional Hill numbers, mean functional diversity and total functional diversity) quantify different aspects of species trait space, and all are based on species abundance and species pairwise functional distances. When all species are equally distinct, our functional Hill numbers reduce to ordinary Hill numbers. When species abundances are not considered or species are equally abundant, our total functional diversity reduces to the sum of all pairwise distances between species of an assemblage. The functional Hill numbers and the mean functional diversity both satisfy a replication principle, implying the total functional diversity satisfies a quadratic replication principle. When there are multiple assemblages defined by the investigator, each of the three measures of the pooled assemblage (gamma) can be multiplicatively decomposed into alpha and beta components, and the two components are independent. The resulting beta component measures pure functional differentiation among assemblages and can be further transformed to obtain several classes of normalized functional similarity (or differentiation) measures, including N-assemblage functional generalizations of the classic Jaccard, Sørensen, Horn and Morisita-Horn similarity indices. The proposed measures are applied to artificial and real data for illustration.     

Species interactions mediate phylogenetic community structure in a hyperdiverse lizard assemblage from arid Australia

Evolutionary history can exert a profound influence on ecological communities, but few generalities have emerged concerning the relationships among phylogeny, community membership, and niche evolution. We compared phylogenetic community structure and niche evolution in three lizard clades (Ctenotus skinks, agamids, and diplodactyline geckos) from arid Australia. We surveyed lizard communities at 32 sites in the northwestern Great Victoria Desert and generated complete species-level molecular phylogenies for regional representatives of the three clades. We document a striking pattern of phylogenetic evenness within local communities for all groups: pairwise correlations in species abundance across sites are negatively related to phylogenetic similarity. By modeling site suitability on the basis of species' habitat preferences, we demonstrate that phylogenetic evenness generally persists even after controlling for habitat filtering among species. This phylogenetic evenness is coupled with evolutionary lability of habitat-associated traits, to the extent that closely related species are more divergent in habitat use than distantly related species. In contrast, lizard diets are phylogenetically conserved, and pairwise dietary overlap between species is negatively related to phylogenetic distance in two of the three clades. Our results suggest that contemporary and historical species interactions have led to similar patterns of community structure across multiple clades in one of the world's most diverse lizard communities.     

Host ontogeny and the temporal decay of similarity in parasite communities of marine fish

Geographical distances between host populations are key determinants of how many parasite species they share. In principle, decay in similarity should also occur with increasing distance along any other dimension that characterizes some form of separation between communities. Here, we apply the biogeographical concept of distance decay in similarity to ontogenetic changes in the metazoan parasite communities of three species of marine fish from the Atlantic coast of South America. Using differences in body length between all possible pairs of size classes as measures of ontogenetic distances, we find that, using an index of similarity (Bray-Curtis) that takes into account the abundance of each parasite species, the similarity in parasite communities showed a very clear decay pattern; using an index (Jaccard) based on presence/absence of species only, we obtained slightly weaker but nevertheless similar patterns. As we predicted, the slope of the decay relationship was significantly steeper in the fish Cynoscion guatucupa, which goes through clear ontogenetic changes in diet and therefore in exposure to parasites, than in the other species, Engraulis anchoita and Micropogonias furnieri, which maintain a roughly similar diet throughout their lives. In addition, we found that for any given ontogenetic distance, i.e. for a given length difference between two size classes, the similarity in parasite communities was almost always higher if they were adult size classes, and almost always lower if they were juvenile size classes. This, combined with comparisons among individual fish within size classes, shows that parasite communities in juvenile fish are variable and subject to stochastic effects. We propose the distance decay approach as a rigorous and quantitative method to measure rates of community change as a function of host age, and for comparisons across host species to elucidate the role of host ecology in the development of parasite assemblages.     

Diversity and distribution of soil fungal communities in a semiarid grassland

The fungal loop model of semiarid ecosystems integrates microtopographic structures and pulse dynamics with key microbial processes. However limited data exist about the composition and structure of fungal communities in these ecosystems. The goal of this study was to characterize diversity and structure of soil fungal communities in a semiarid grassland. The effect of long-term nitrogen fertilization on fungi also was evaluated. Samples of rhizosphere (soil surrounding plant roots) and biological soil crust (BSC) were collected in central New Mexico, USA. DNA was amplified from the samples with fungal specific primers. Twelve clone libraries were generated with a total of 307 (78 operational taxonomic units, OTUs) and 324 sequences (67 OTUs) for BSC and rhizosphere respectively. Approximately 40% of soil OTUs were considered novel (less than 97% identity when compared to other sequences in NCBI using BLAST). The dominant organisms were dark-septate (melanized fungi) ascomycetes belonging to Pleosporales. Effects of N enrichment on fungi were not evident at the community level; however the abundance of unique sequences, sampling intensity and temporal variations may be uncovering the effect of N in composition and diversity of fungal communities. The fungal communities of rhizosphere soil and BSC overlapped substantially in composition, with a Jaccard abundance similarity index of 0.75. Further analyses are required to explore possible functions of the dominant species colonizing zones of semiarid grassland soils.     

Detecting spatial patterns in species composition with multiple plot similarity coefficients and singularity measures

Recently, several multiple plot similarity indices have been presented that cure some of the problems associated with the approaches for the calculation of compositional similarity for groups of plots by averaging pairwise similarities. These new indices calculate the similarity between more than two plots whilst considering the species composition on all compared plots. The resulting similarity value is true for the whole group of plots considered (called neighborhood in the following). Here, we review the possibilities for multiple plot similarity calculation and additionally explore coefficients that examine multiple plot similarity between a reference plot (named focal plot in the following) and any number of surrounding plots. The latter represent measures of singularity. Further, we establish a framework for applying these two kinds of multiple plot measures to gridded data including an algorithm for testing the significance of calculated values against random expectations. The capability of multiple plot measures for detecting species compositional gradients and local/regional hotspots within this framework is tested. For this purpose, several artificial data sets with known gradients in species composition (random, gradient, central hotspot, hotspot bottom right) are constructed on the basis of a real data set from a Tundra ecosystem in northern Sweden (Abisko). The coefficients that best reflect the positions of the plots on the realized gradients in species composition are considered as performing best with regard to pattern detection. The tested measures of multiple plot similarity and singularity produced considerably different results when applied to one real and 4 artificial data sets. The newly proposed symmetric singularity coefficient has the best overall performance which makes it suitable for local/regional hotspot detection and for incorporating local to regional similarity analyses in reserve selection procedures.     

The biogeography of parasitism in sticklebacks: distance,habitat differences and the similarity in parasite occurrence and abundance

Similarity in parasite community composition often decreases with both increasing geographic distance and environmental dissimilarity between localities, though it is unknown whether similarity in local abundance of selected parasite species follows similar rules. We tested this using data on metazoan parasites in 126 stickleback (Gasterosteus aculeatus) populations, with locations from Eurasia, eastern North America, and western North America treated separately. Similarity values were regressed against pairwise distances between localities; after correcting for distance, the effect of environmental dissimilarity was assessed by splitting similarity values into those between pairs of localities with either similar, moderately different or very different salinity (freshwater, marine or brackish). For selected parasite species, pairwise similarity in abundance (mean no. parasites per host) were computed across all localities, and treated as above. Similarity in parasite community composition decreased with increasing distance between localities in all three geographic regions. A significant effect of environmental difference was found in all regions: for a given distance between two sites, their parasite communities were more similar if they were of the same salinity. Slopes for distance decay in similarity were consistently higher for eastern North America than for Eurasia. Among the 12 parasite species for which sufficient data were available, only 4 showed the expected relationship, i.e. the greater the geographical separation between host populations, the greater the difference in parasite abundance; also, significant effects of environmental differences in salinity were only found for 3 of these species. Our findings show that parasite communities of sticklebacks are structured by geographical distance and local salinity conditions. The results indicate that strong effects at the community level do not translate into corresponding effects at the population level, suggesting that parasite dispersal and population dynamics are controlled by different processes.     

Heavy-weighting rare species in dissimilarity indices improve recovery of multivariate groups

Dissimilarity indices differ in the relative weight given to rare species. Heavy-weighting of rare species may be justified in terms of sampling. An index may erroneously estimate high dissimilarity between two identical communities if they are composed of many rare species and the sampling effort is insufficient to observe most of them in both samples. Heavy-weighting of rare species is thought to compensate for this negative bias. I evaluated two quantitative indices that heavy-weight rare species, NNESS (New Normalized Expected Species Shared) and Goodall, and two probability versions of the Sørensen index, one that takes into account shared unseen rare species and the other that does not. They were compared against the widely used Bray-Curtis (or Sørensen quantitative) and the Morisita-Horn. Indices were computed using raw abundance data or coded data that heavy-weight rare species (frequency in sample units, log-transformation and standardization by the maximum abundance within species). Indices were evaluated for their ability to distinguish, using distance-based MANOVA, season-defined (summer, winter) groups of samples of stream macroinvertebrates and groups of samples obtained by simulation. Sørensen corrected for unseen shared species performed poorly in the empirical study and intermediate in the simulations. NNESS was good in the empirical study and intermediate in the simulations. Goodall scored inversely as NNESS, being intermediate in the empirical assessment and very good in the simulations. The Sørensen uncorrected for unseen shared species, Bray-Curtis and the Morisita-Horn presented poor or intermediate results using raw abundance data. Their performance, however, improved consistently using coded data that heavy-weight rare species and made them good or very good. I conclude that heavy-weighting rare species improves the ability to detect multivariate groups. Heavy-weighting of rare species may be achieved either by using specific formulae (NNESS, Goodall) or using coded data.     

Diatom Model Affinity (DMA), a New Index for Water Quality Assessment

A new index, diatom model affinity (DMA) was erected as a reference-based community metric, derived from generic and species composition. DMA provides an assessment of water quality by the calculation of percent similarity to a model community, which can be viewed as a reference standard. High similarity to the model indicates communities that are minimally disturbed, while lower similarity suggests water quality problems. The index is similar to that developed for macroinvertebrate communities by Novak & Bode (1992, Journal of the North American Benthological Society 11: 80-85). The index correlated well with environmental factors operating on hierarchical scales. Compared to other commonly used diatom indices, DMA-based community assessments demonstrated high correspondence to patterns derived from multivariate statistics.