Optimizing performance of nonparametric species richness estimators under constrained sampling

Understanding the functional relationship between the sample size and the performance of species richness estimators is necessary to optimize limited sampling resources against estimation error. Nonparametric estimators such as Chao and Jackknife demonstrate strong performances, but consensus is lacking as to which estimator performs better under constrained sampling. We explore a method to improve the estimators under such scenario. The method we propose involves randomly splitting species‐abundance data from a single sample into two equally sized samples, and using an appropriate incidence‐based estimator to estimate richness. To test this method, we assume a lognormal species‐abundance distribution (SAD) with varying coefficients of variation (CV), generate samples using MCMC simulations, and use the expected mean‐squared error as the performance criterion of the estimators. We test this method for Chao, Jackknife, ICE, and ACE estimators. Between abundance‐based estimators with the single sample, and incidence‐based estimators with the split‐in‐two samples, Chao2 performed the best when CV < 0.65, and incidence‐based Jackknife performed the best when CV > 0.65, given that the ratio of sample size to observed species richness is greater than a critical value given by a power function of CV with respect to abundance of the sampled population. The proposed method increases the performance of the estimators substantially and is more effective when more rare species are in an assemblage. We also show that the splitting method works qualitatively similarly well when the SADs are log series, geometric series, and negative binomial. We demonstrate an application of the proposed method by estimating richness of zooplankton communities in samples of ballast water. The proposed splitting method is an alternative to sampling a large number of individuals to increase the accuracy of richness estimations; therefore, it is appropriate for a wide range of resource‐limited sampling scenarios in ecology.     

Performance of nonparametric species richness estimators in a high diversity plant community

Abstract. The efficiency of four nonparametric species richness estimators — first‐order Jackknife, second‐order Jackknife, Chao2 and Bootstrap — was tested using simulated quadrat sampling of two field data sets (a sandy ‘Dune’ and adjacent ‘Swale’) in high diversity shrublands (kwongan) in south‐western Australia. The data sets each comprised > 100 perennial plant species and > 10 000 individuals, and the explicit (x‐y co‐ordinate) location of every individual. We applied two simulated sampling strategies to these data sets based on sampling quadrats of unit sizes 1/400th and 1/100th of total plot area. For each site and sampling strategy we obtained 250 independent sample curves, of 250 quadrats each, and compared the estimators’ performances by using three indices of bias and precision: MRE (mean relative error), MSRE (mean squared relative error) and OVER (percentage overestimation). The analysis presented here is unique in providing sample estimates derived from a complete, field‐based population census for a high diversity plant community. In general the true reference value was approached faster for a comparable area sampled for the smaller quadrat size and for the swale field data set, which was characterized by smaller plant size and higher plant density. Nevertheless, at least 15–30% of the total area needed to be sampled before reasonable estimates of S_t (total species richness) were obtained. In most field surveys, typically less than 1% of the total study domain is likely to be sampled, and at this sampling intensity underestimation is a problem. Results showed that the second‐order Jackknife approached the actual value of S_t more quickly than the other estimators. All four estimators were better than S_obs (observed number of species). However, the behaviour of the tested estimators was not as good as expected, and even with large sample size (number of quadrats sampled) all of them failed to provide reliable estimates. First‐ and second‐order Jackknives were positively biased whereas Chao2 and Bootstrap were negatively biased. The observed limitations in the estimators’ performance suggests that there is still scope for new tools to be developed by statisticians to assist in the estimation of species richness from sample data, especially in communities with high species richness.     

Assessing the performance of nonparametric estimators of species richness in meadows

To accurately measure the number of species in a biological community, a complete inventory should be performed, which is generally unfeasible; hopefully, estimators of species richness can help. Our main objectives were (i) to assess the performance of nonparametric estimators of plant species richness with real data from a small set of meadows located in the Basque campiña (northern Spain), and (ii) to apply the best estimator to a larger dataset to test the effects on plant species richness caused by environmental conditions and human practices. Two non-asymptotic and seven asymptotic accumulation functions were fitted to a randomized sample-based rarefaction curve computed with data from three well sampled meadows, and information theoretic methods were used to select the best fitting model; this was the Morgan-Mercer-Flodin, and its asymptote was taken as our best guess of true richness. Then, five nonparametric estimators were computed: ICE, Chao 2, Jackknife 1 and 2, and Bootstrap; MMRuns and MMMeans were also assessed. According to the criteria set for our performance assessment (i.e., bias, precision, and accuracy), the best estimator was Jackknife 1. Finally, Jackknife 1 was applied to assess the effects of terrain slope and soil parent material, and also fertilization, grazing, and mowing, on plant species richness from a larger dataset (20 meadows). Results suggested that grass cutting was causing a loss of richness close to 30%, as compared to unmowed meadows. It is concluded that the use of nonparametric estimators of species richness can improve the evaluation of biodiversity responses to human management practices.     

Could we obtain better estimates of plot species richness from multiple‐observer plant censuses?

Question: Could we better estimate plot species richness by asking several botanists to survey the same plots and using non‐parametric estimators of richness? Location: Two French deciduous forests. Methods: Using replicated, independent censuses made by 11 professional botanists on the same eight 100‐m² forest plots, the relative performance of different richness estimators (Lincoln‐Petersen, Jackknife 1&2, Chao 1&2, Bootstrap, Chao Mth, Darroch) and the variation in their performance with the number of botanists involved (teams with two to eight botanists) were investigated. The sensitivity of these estimators to the presence of misidentifications in the data was also assessed. Results: When misidentifications are removed, Chao Mth estimators converged fastest to true richness, but none of the tested estimators correctly accounted for differences in exhaustiveness between the teams. Finally, all estimators were highly sensitive to misidentifications. Conclusions: Richness estimators are of little help in the presence of misidentifications and are ineffective at removing between‐team discrepancies, thus strongly limiting their usefulness in practice. Methods are presented to show how surveys can be designed to remove misidentifications and limit between‐team discrepancies. A sensible sampling design for 100‐m² plots in temperate forests would involve triplets of botanists and correcting data with the Chao N1. Pairs of botanists would already significantly improve the richness estimates, but such estimates would still be biased low. However, further research is needed to design new richness estimators that are more robust to observer effects.     

The relationship between percentage of singletons and sampling effort: A new approach to reduce the bias of richness estimates

Estimate the richness of a community with accuracy despite differences in sampling effort is a key aspect to monitoring high diverse ecosystems. We compiled a worldwide multitaxa database, comprising 185 communities, in order to study the relationship between the percentage of species represented by one individual (singletons) and the intensity of sampling (number of individuals divided by the number of species sampled). The database was used to empirically adjust a correction factor to improve the performance of non-parametrical estimators under conditions of low sampling effort. The correction factor was tested on seven estimators (Chao1, Chao2, Jack1, Jack2, ACE, ICE and Bootstrap). The correction factor was able to reduce the bias of all estimators tested under conditions of undersampling, while converging to the original uncorrected values at higher intensities. Our findings led us to recommend the threshold of 20 individuals/species, or less than 21% of singletons, as a minimum sampling effort to produce reliable richness estimates of high diverse ecosystems using corrected non-parametric estimators. This threshold rise for 50 individuals/species if non-corrected estimators are used which implies in an economy of 60% of sampling effort if the correction factor is used.     

Evaluating the performance of species richness estimators: sensitivity to sample grain size

1. Fifteen species richness estimators (three asymptotic based on species accumulation curves, 11 nonparametric, and one based in the species-area relationship) were compared by examining their performance in estimating the total species richness of epigean arthropods in the Azorean Laurisilva forests. Data obtained with standardized sampling of 78 transects in natural forest remnants of five islands were aggregated in seven different grains (i.e. ways of defining a single sample): islands, natural areas, transects, pairs of traps, traps, database records and individuals to assess the effect of using different sampling units on species richness estimations. 2. Estimated species richness scores depended both on the estimator considered and on the grain size used to aggregate data. However, several estimators (ACE, Chao 1, Jackknifel and 2 and Bootstrap) were precise in spite of grain variations. Weibull and several recent estimators [proposed by Rosenzweig et al. (Conservation Biology, 2003, 17, 864-874), and Ugland et al. (Journal of Animal Ecology, 2003, 72, 888-897)] performed poorly. 3. Estimations developed using the smaller grain sizes (pair of traps, traps, records and individuals) presented similar scores in a number of estimators (the above-mentioned plus ICE, Chao2, Michaelis-Menten, Negative Exponential and Clench). The estimations from those four sample sizes were also highly correlated. 4. Contrary to other studies, we conclude that most species richness estimators may be useful in biodiversity studies. Owing to their inherent formulas, several nonparametric and asymptotic estimators present insensitivity to differences in the way the samples are aggregated. Thus, they could be used to compare species richness scores obtained from different sampling strategies. Our results also point out that species richness estimations coming from small grain sizes can be directly compared and other estimators could give more precise results in those cases. We propose a decision framework based on our results and on the literature to assess which estimator should be used to compare species richness scores of different sites, depending on the grain size of the original data, and of the kind of data available (species occurrence or abundance data).     

Application of species richness estimators for the assessment of fungal diversity

Species richness and distribution patterns of wood-inhabiting fungi and mycetozoans (slime moulds) were investigated in the canopy of a Central European temperate mixed deciduous forest. Species richness was described with diversity indices and species-accumulation curves. Nonmetrical multidimensional scaling was used to assess fungal species composition on different tree species. Different species richness estimators were used to extrapolate species richness beyond our own data. The reliability of the abundance-based coverage estimator, Chao, Jackknife and other estimators of species richness was evaluated for mycological surveys. While the species-accumulation curve of mycetozoans came close to saturation, that of wood-inhabiting fungi was continuously rising. The Chao 2 richness estimator was considered most appropriate to predict the number of species at the investigation site if sampling were continued. Gray's predictor of species richness should be used if statements of the number of species in larger areas are required. Multivariate analysis revealed the importance of different tree species for the conservation and maintenance of fungal diversity within forests, because each tree species possessed a characteristic fungal community. The described mathematical approaches of estimating species richness possess great potential to address fungal diversity on a regional, national, and global scale.     

Endoparasite Species Richness of Iberian Carnivores: Influences of Host Density and Range Distribution

We investigate the determinants of macroparasite species richness of Iberian carnivores. For this, we used the parasitological data collected on 14 species of carnivores over a 10-year period. These previously unpublished data permitted to estimate parasite species richness using estimators of species richness, i.e. Jackknife first order and Chao 2. Most of the parasite species were rare, with low prevalence. Potential determinants were investigated as possible factors explaining the variability of parasites species richness among carnivores host body mass, host geographical range, host longevity and host density. Using independent contrasts, we found positive relationships between residuals of estimates of parasite species richness and residuals in host density, and between residuals of estimates of parasite species richness and residuals in host range. These results are discussed in terms of risk of extinction and invasion abilities related to a possible investment in immune defences correlated with parasite diversity.     

Efficient sampling of plant diversity in arid deserts using non-parametric estimators

Surveying plant diversity in arid desert areas is extremely difficult because of the harsh climate, hostile terrain, lack of roads, and insecurity, which is why it is particularly important to improve the sampling efficiency, but few relevant studies have been done. The performance of non-parametric estimators was assessed with first-hand field data to determine (a) the threshold of the proportion of uniques (number of species that occur in exactly one plot divided by the number of species sampled) that involves the least sampling effort and (b) the method of locating plots to obtain a more reliable estimate of species richness. The study area (Gurbantunggut desert, China) was divided into five sub-regions based on variation in physical environment and vegetation. The following common correction factors were selected: ACE, Chao1, Bootstrap, Chao2, ICE, Jack1, and Jack2. The estimates for each sub-region (partition) and for the entire region (without partition), the threshold of proportion of uniques, and the method of determining sampling locations (including prior sampling of plots that show large differences in habitats) were compared in terms of their ability to predict the number of species more accurately. We found that ACE and Chao1 (which use abundance data) showed more biased estimates than the other factors (incidence data), and best estimator is Jack1. Species richness was significantly underestimated for the region, but the non-parametric estimators could estimate the species richness for each sub-region reliably. Sampling locations affected the performance of non-parametric estimators significantly. The threshold of minimum sampling was 15% and that of uniques was 30%; the two were able to limit the bias within 5 and 10%, respectively. It is concluded that the non-parametric estimators can estimate the plant diversity of arid deserts reliably from the data on incidence. The study area (on the scale of a region) should be partitioned to improve the performance of the non-parametric estimators. The plots with larger differences in habitats should be sampled more extensively based on the threshold of the proportion of uniques.     

Testing species-richness estimation methods on single-sample collection data using the Danish Diptera

Estimating species richness through extrapolation is becomingincreasingly important for conservation decision making. We present the resultsof a first test of four abundance-based estimation procedures, ACE, Chao1, Lognormal and Poisson lognormal based on single-sample museum collection data consisting of more than 150000specimens of 47 families of Danish Diptera. All four estimators considerablyunderestimate true species richness as assessed by species distributions, expertopinions, and a species–area curve. In our samples 3326 species wererepresented. The different estimators predicted the Danish fauna to consist of3490–3805 species, although at least 4361 are already known from theliterature. Expert opinion and the species–area curve indicate that theDanish fauna likely contains 5400–5800 species. The Poisson lognormalmethod displays a rather erratic behavior, but nonetheless performs slightlybetter than the other estimators. We discuss the inherent problems concerningthe use of collection data in this context as well as the influence of patchydistributions and sample size on estimator performance. We conclude thatabundance-based estimators should preferably be applied to almost completesamples of randomly distributed organisms.     

Limited sampling hampers “big data” estimation of species richness in a tropical biodiversity hotspot

Macro‐scale species richness studies often use museum specimens as their main source of information. However, such datasets are often strongly biased due to variation in sampling effort in space and time. These biases may strongly affect diversity estimates and may, thereby, obstruct solid inference on the underlying diversity drivers, as well as mislead conservation prioritization. In recent years, this has resulted in an increased focus on developing methods to correct for sampling bias. In this study, we use sample‐size‐correcting methods to examine patterns of tropical plant diversity in Ecuador, one of the most species‐rich and climatically heterogeneous biodiversity hotspots. Species richness estimates were calculated based on 205,735 georeferenced specimens of 15,788 species using the Margalef diversity index, the Chao estimator, the second‐order Jackknife and Bootstrapping resampling methods, and Hill numbers and rarefaction. Species richness was heavily correlated with sampling effort, and only rarefaction was able to remove this effect, and we recommend this method for estimation of species richness with “big data” collections.     

The net result: evaluating species richness extrapolation techniques for littoral pond invertebrates

1. Total species richness for an assemblage or site is a valuable measure in conservation monitoring and assessment, but protocols for sampling and species richness determination in wetland habitats such as ponds, bogs or mires remain largely unrefined. 2. Techniques for estimation of total richness of an assemblage based upon replicated sampling offer the opportunity to derive useful estimates of total richness based upon small numbers of samples, and limit sampling‐derived disturbance which can be particularly problematic in small aquatic habitats. 3. We quantified the performance of eight of the most commonly encountered estimators of species richness for a variety of littoral zone macrofauna from ponds, comparing estimated richness to maximum richness derived from sampling. 4. Estimates using non‐parametric techniques based on species incidence provided the most accurate and precise estimates. The estimators Chao 2 and incidence‐based coverage estimator (ICE) from this category were reliable and consistent slight over‐estimators; the abundance‐based estimator Chao1 also performed well. 5. Species inventory based on relatively small numbers of samples might be significantly improved by use of non‐parametric estimators for quantification of species richness. 6. Use of non‐parametric estimators of species richness can assist biodiversity inventory by preventing erroneous rankings of habitat richness based upon observed species numbers from limited sampling.     

Temporal Patterns of Species Accumulation in a Survey of Lepidoptera in a Beech-Maple Forest

One of the most significant challenges to insect conservation is lack of information concerning species diversity and distribution. Because a complete inventory of all species in an area is virtually impossible, interest has turned to developing statistical techniques to guide sampling design and to estimate total species richness within a site. We used two such techniques, diversity partitioning and non-parametric richness estimation, to determine how variation in sampling effort over time affected species accumulation for a survey of Lepidoptera in an old-growth beech-maple forest. Temporal scaling of sampling effort had significant effects on two measures of species diversity. Increases in species richness were primarily driven by changes in species occurrences with season, while Shannon diversity was largely determined at the scale of individual sampling units (i.e. by spatial effects). Variation in sampling effort affected the values of the two most widely regarded richness estimators (ICE and Chao 2); neither diversity estimator achieved stable values across a range of sampling efforts. Even after 52 trap-nights and accounting for seasonality, rare species (singletons and uniques) remained a significant component of the moth community. To the extent that moth communities in other forest systems are similarly comprised of many rare species, non-parametric richness estimators should be expected to yield variable estimates with increased effort and should only be used to provide a minimum benchmark for predicting the number of species remaining to be sampled. Our results suggest the best strategy for a short-term survey of forest Lepidoptera should emphasize spreading sampling intervals throughout a given year rather than focusing on intensive sampling during a short time period or prolonged sampling over many years.     

Species richness estimation of bird communities: how to control for sampling effort?

Since estimates of total species richness increase with sampling effort, methods to control for this sampling effect need to be tested and used. We present seven non-parametric and 12 accumulation curve methods that have been used recently in the ecological literature. To test their performance, we used data from bird communities in the Queen Charlotte Islands, Canada. The performance of each method was evaluated by calculating the bias and precision of its estimates against the known total species richness. For our data set, the two Chao estimators were the overall least biased and most precise estimation methods, followed by the two jackknife estimators, thus supporting results of previous studies. Nonparametric estimators tended to perform better than accumulation curve models. Most estimation methods had the problem that they tended to underestimate species richness for early samples, but slightly overestimated it for late samples. We briefly discuss the practical use of these methods which may greatly increase our ability to answer ecological questions and to guide conservation decisions, especially for species-rich tropical bird communities.     

Species richness patterns of obligate subterranean beetles (Insecta: Coleoptera) in a global biodiversity hotspot – effect of scale and sampling intensity

We studied species richness patterns of obligate subterranean (troglobiotic) beetles in the Dinaric karst of the western Balkans, using five grid sizes with cells of 80 × 80, 40 × 40, 20 × 20, 10 × 10, and 5 × 5 km. The same two hotspots could be recognized at all scales, although details differed. Differences in sampling intensity were not sufficient to explain these patterns. Correlations between number of species and number of sampled localities increased with increasing cell size. Additional species are expected to be found in the region, as indicated by jackknife 1, jackknife 2, Chao2, bootstrap, and incidence‐based coverage (ICE) species richness estimators. All estimates increased with increasing cell size, except Chao2, with the lowest prediction at the intermediate 20 × 20 km cell size. Jackknife 2 and ICE gave highest estimates and jackknife 1 and bootstrap the lowest. Jackknife 1 and bootstrap estimates changed least with cell size, while the number of single cell species increased. In the highly endemic subterranean fauna with many rare species, bootstrap may be most appropriate to consider. Positive autocorrelation of species numbers was highest at 20 × 20 km scale, so we used this cell size for further analyses. At this scale we added 137 localities with less positional accuracy to 1572 previously considered, and increased 254 troglobiotic species considered to 276. Previously discovered hotspots and their positions did not change, except for a new species‐rich cell which appeared in the south‐eastern region. There are two centres of troglobiotic species richness in the Dinaric karst. The one in the north‐west exhibited high species richness of Trechinae (Carabidae), while in the south‐east, the Leptodirinae (Cholevidae) were much more diverse. These centres of species richness should serve as the starting point for establishing a conservation network of important subterranean areas in Dinaric karst.     

Massively parallel rRNA gene sequencing exacerbates the potential for biased community diversity comparisons due to variable library sizes

Technologies for massively parallel sequencing are revolutionizing microbial ecology and are vastly increasing the scale of ribosomal RNA (rRNA) gene studies. Although pyrosequencing has increased the breadth and depth of possible rRNA gene sampling, one drawback is that the number of reads obtained per sample is difficult to control. Pyrosequencing libraries typically vary widely in the number of sequences per sample, even within individual studies, and there is a need to revisit the behaviour of richness estimators and diversity indices with variable gene sequence library sizes. Multiple reports and review papers have demonstrated the bias in non-parametric richness estimators (e.g. Chao1 and ACE) and diversity indices when using clone libraries. However, we found that biased community comparisons are accumulating in the literature. Here we demonstrate the effects of sample size on Chao1, ACE, CatchAll, Shannon, Chao-Shen and Simpson's estimations specifically using pyrosequencing libraries. The need to equalize the number of reads being compared across libraries is reiterated, and investigators are directed towards available tools for making unbiased diversity comparisons.     

Evaluation of methods for estimating macroinvertebrate species richness using individual stones in tropical streams

1. The most straightforward way to assess diversity in a site is the species count. However, a relatively large sample is needed for a reliable result because of the presence of many rare species in rich assemblages. The use of richness estimation methods is suggested by many authors as a solution for this problem in many cases.
2. We examined the performance of 13 methods for estimating richness of stream macroinvertebrates inhabiting riffles both at local (stream) and regional (catchment) scales. The evaluation was based on (1) the smallest sub-sample size needed to estimate total richness in the sample, (2) constancy of this size, (3) lack of erratic behaviour in curve shape and (4) similarity in curve shape through different data sets. Samples were from three single stream sites (local) and three from several streams within the same catchment basin (regional). All collections were made from protected forest areas in south-east Brazil.
3. All estimation methods were dependent on sub-sample size, producing higher estimates when using larger sub-sample sizes. The Stout and Vandermeer method estimated total richness in the samples with the smallest sub-sample size, but showed some erratic behaviour at small sub-sample sizes, and the estimated curves were not similar among the six samples. The Bootstrap method was the best estimator in relation to constancy of sub-sample sizes, but needed an unacceptably large sub-sample to estimate total richness in the samples. The second order Jackknife method was the second best estimator both for minimum sub-sample size and constancy of this size and we suggest its use in future studies of diversity in tropical streams. Despite the inferior performance of several other methods, some produced acceptable results. Comments are made on the utility of using these estimators for predicting species richness in an area and for comparative purposes in diversity studies.     

The use of incidence-based species richness estimators,species accumulation curves and similarity measures to appraise ethnobotanical inventories from South Africa

The incorporation of suitable quantitative methods into ethnobotanical studies enhances the value of the research and the interpretation of the results. Prediction of sample species richness and the use of species accumulation functions have been addressed little in applied ethnobotany. In this paper, incidence-based species richness estimators, species accumulation curves and similarity measures are used to compare and predict species richness, evaluate sampling effort and compare the similarity of species inventories for ethnobotanical data sets derived from the trade in traditional medicine in Johannesburg and Mpumalanga, South Africa. EstimateS was used to compute estimators of species richness (e.g. Jackknife), rarefaction curves, species accumulation curves and complimentarity. Results showed that while the Michaelis–Menten Means estimator appeared to be the best estimator because the curve approached a horizontal asymptote, it was not able to accurately predict species richness for one of the data sets when two of its subsamples were individually tested. Instead, the first-order Jackknife estimator best approximated the known richness.     

Estimating the richness of species with variable mobility

The vast majority of species are animals that, unlike most plants and fungi, are variably and often highly mobile. While species' mobility affects species' probabilities of being sampled, effects of movement on the estimation of species richness have yet to be systematically investigated. Information-rich abundance-based estimators may be able to address variably mobile species but the accuracy of these estimators has also yet to be investigated. Here, we address both issues by variably sampling simulated landscapes with up to 250 species and evaluating the performance of ten non-parametric estimators and one species accumulation curve. Our results show that some abundance-based estimators are as accurate as better known and tested incidence-based estimators. Increased movement heterogeneity between the species reduced estimator performance by reducing the sample coverage, which systematically determined which estimator was most accurate. Based on these findings, we present the first decision framework for choosing the most accurate of many available abundance-based species-richness estimators. These decisions, based on data coverage, can significantly improve investigators' ability to estimate faunal species richness.