Macroinvertebrate-based multimetric predictive models for evaluating the human impact on biotic condition of Bolivian streams

We developed and validated a single multimetric index based on predictive models that could evaluate anthropogenic disturbances in streams of three disparate ecoregions of Bolivia. To do so, we examined 45 candidate metrics reflecting different aspects of macroinvertebrate assemblage structure and function gleaned from available literature and for their potential to indicate degradation. More importantly, we integrated functional trait metrics to improve the sensitivity of our index. To quantify possible deviation from reference conditions, we first established and validated statistical models describing metric responses to natural environmental differences in the absence of any significant anthropogenic disturbance. We considered that the residual distributions of these models described the response range of each metric, independently of natural environmental influence. After testing the sensitivity of these residuals to a gradient of anthropogenic disturbance, we retained eight metrics that were used in the final assemblage index, four metrics based on richness and composition and four metrics based on biological traits. Our index performed well in discriminating between reference and disturbed sites, giving a significant negative linear response to a gradient of physical and chemical anthropogenic disturbances. After employing a probability survey design and sampling a relatively small number of sites throughout all major ecoregions of Bolivia, we believe our methodology can be used to develop a monitoring tool to evaluate status and trends in biological condition for streams of the entire country despite its complex and heterogeneous geology and climate.     

Circadian rhythms vary over the growing season and correlate with fitness components

Circadian clocks have evolved independently in all three domains of life, suggesting that internal mechanisms of time‐keeping are adaptive in contemporary populations. However, the performance consequences of either discrete or quantitative clock variation have rarely been tested in field settings. Clock sensitivity of diverse segregating lines to the environment remains uncharacterized as do the statistical genetic parameters that determine evolutionary potential. In field studies with Arabidopsis thaliana, we found that major perturbations to circadian cycle length (referred to as clock period) via mutation reduce both survival and fecundity. Subtler adjustments via genomic introgression of naturally occurring alleles indicated that clock periods slightly >24 hr were adaptive, consistent with prior models describing how well the timing of biological processes is adjusted within a diurnal cycle (referred to as phase). In segregating recombinant inbred lines (RILs), circadian phase varied up to 2 hr across months of the growing season, and both period and phase expressed significant genetic variances. Performance metrics including developmental rate, size and fruit set were described by principal components (PC) analyses and circadian parameters correlated with the first PC, such that period lengths slightly >24 hr were associated with improved performance in multiple RIL sets. These experiments translate functional analyses of clock behaviour performed in controlled settings to natural ones, demonstrating that quantitative variation in circadian phase is highly responsive to seasonally variable abiotic factors. The results expand upon prior studies in controlled settings, showing that discrete and quantitative variation in clock phenotypes correlates with performance in nature.     

An unbiased sensitivity analysis reveals important parameters controlling periodicity of circadian clock

To assess the importance of model parameters in kinetic models, sensitivity analysis is generally employed to provide key measures. However, it is quite often that no information is available for a significant number of parameters in biochemical models. Therefore, the results of sensitivity analysis that heavily rely on the accuracy of parameters are largely ambiguous. In this study, we propose a computational approach to determine the relative importance of parameters controlling the performance of the circadian clock in Drosophila. While previous attempts to sensitivity analysis largely depend on the knowledge of model parameters which are generally unknown, our study depicts a consistent picture of sensitivity assessment for a large number of parameters, even when the values of these parameters are not available in vivo. The resulting parametric sensitivity analysis suggests that PER/TIM negative loop is critical to maintain the stable periodicity of the circadian clock, which is consistent to the previously experimental and computational findings. Furthermore, our analysis generates a rich hypothesis of important parameters in the circadian clock that can be further tested experimentally. This approach can also be extended to assess the sensitivity of parameters in any biochemical system where a large number of parameters have unknown values. Biotechnol. Bioeng. 2010; 105: 250–259. © 2009 Wiley Periodicals, Inc.     

Modeling wetland plant metrics to improve the performance of vegetation-based indices of biological integrity

The objective of this study was to determine if the accuracy and precision of wetland plant indices of biological integrity (IBIs) could be improved through the use of modeling techniques. To do this, we developed a modeled vegetation IBI (MVIBI) based on metrics previously used to develop vegetation indices of biological integrity (VIBIs) for Ohio wetlands (e.g. % invasive grass, % sensitive species, shrub richness). We selected 82 emergent, forested, and shrub-dominated reference sites distributed across the State of Ohio and built Random Forest models to predict plant metric scores at reference wetlands from naturally occurring environmental features related to climate, hydrology, geology, soils, and landscape position. The models explained between 14 and 52% of the variance in the scores of 21 metrics indicating that variation in wetland plant assemblages was significantly associated with naturally occurring environmental gradients. We used principal component analysis to identify ten groups of statistically independent metrics and selected one metric from each group that discriminated most strongly between reference and most degraded sites based on t-scores. Two axes did not contain discriminating metrics so we used eight metrics in the MVIBI. Analysis of variance of reference site MVIBI scores indicated that we could use one distribution of reference site scores to assess multiple wetland types, thus eliminating the need to separately designate wetland types. We used the MVIBI to assess 170 test sites and compared the accuracy, precision, responsiveness, and sensitivity of the MVIBI to those of the original VIBIs. The MVIBI was up to twice as accurate and precise as the original VIBIs, indicating that modeling can be used to improve the performance of vegetation-based IBIs. The use of model-based IBIs for wetland plants should reduce assessment errors associated with natural variation in plant metrics and should increase confidence in wetland assessments.     

Regional fish community indicators of landscape disturbance to catchments of the conterminous United States

Biological assessments of river conditions are increasingly conducted at regional and continental scales that match the extent of large-scale river management efforts. Multimetric indices composed of biological community indicators are commonly used to assess ecological condition and indices have recently been applied in large regions. Methods for large-scale multimetric index creation emphasize repeatability, comparability across regions, and objective selection of candidate metrics. Here we used an extensive fish dataset to create a large pool of fish community metrics which were screened to create multimetric indices (MMIs) in eight ecoregions covering the conterminous U.S. Candidate metrics were tested for metric range, corrected for natural gradients using boosted regression trees, and then tested for repeatability and sensitivity to landscape disturbance. Temporally stable and repeatable metrics were then evaluated for redundancy and used to compose MMIs for each region. Our MMIs were significantly correlated to independently developed MMIs, accurately reproducing prior index values with moderate to high precision and little bias. Our study demonstrates the utility of boosted regression tree models for correcting metric values for natural abiotic gradients and shows that the order of screening tests has a potentially important influence on metric selection. The resultant regional indices and component metrics provide a basis for assessing condition and testing hypotheses about landscape influences on aquatic ecosystems at a national scale in the US.     

Interaction strength revisited—clarifying the role of energy flux for food web stability

Interaction strength (IS) has been theoretically shown to play a major role in governing the stability and dynamics of food webs. Nonetheless, its definition has been varied and problematic, including a range of recent definitions based on biological rates associated with model parameters (e.g., attack rate). Results from food web theory have been used to argue that IS metrics based on energy flux ought to have a clear relationship with stability. Here, we use simple models to elucidate the actual relationship between local stability and a number of common IS metrics (total flux and per capita fluxes) as well as a more recently suggested metric. We find that the classical IS metrics map to stability in a more complex way than suggested by existing food web theory and that the new IS metric has a much clearer, and biologically interpretable, relationship with local stability. The total energy flux metric falls off existing theoretical predictions when the total resource productivity available to the consumer is reduced despite increased consumer attack rates. The density of a consumer can hence decrease when its attack rate increases. This effect, called the paradox of attack rate, is similar to the well-known hydra effect and can even cascade up a food chain to exclude a predator when consumer attack rate is increased.     

Ecological niche modeling re‐examined: A case study with the Darwin's fox

Many previous studies have attempted to assess ecological niche modeling performance using receiver operating characteristic (ROC) approaches, even though diverse problems with this metric have been pointed out in the literature. We explored different evaluation metrics based on independent testing data using the Darwin's Fox (Lycalopex fulvipes) as a detailed case in point. Six ecological niche models (ENMs; generalized linear models, boosted regression trees, Maxent, GARP, multivariable kernel density estimation, and NicheA) were explored and tested using six evaluation metrics (partial ROC, Akaike information criterion, omission rate, cumulative binomial probability), including two novel metrics to quantify model extrapolation versus interpolation (E‐space index I) and extent of extrapolation versus Jaccard similarity (E‐space index II). Different ENMs showed diverse and mixed performance, depending on the evaluation metric used. Because ENMs performed differently according to the evaluation metric employed, model selection should be based on the data available, assumptions necessary, and the particular research question. The typical ROC AUC evaluation approach should be discontinued when only presence data are available, and evaluations in environmental dimensions should be adopted as part of the toolkit of ENM researchers. Our results suggest that selecting Maxent ENM based solely on previous reports of its performance is a questionable practice. Instead, model comparisons, including diverse algorithms and parameterizations, should be the sine qua non for every study using ecological niche modeling. ENM evaluations should be developed using metrics that assess desired model characteristics instead of single measurement of fit between model and data. The metrics proposed herein that assess model performance in environmental space (i.e., E‐space indices I and II) may complement current methods for ENM evaluation.     

Evaluating True BCI Communication Rate through Mutual Information and Language Models

Brain-computer interface (BCI) systems are a promising means for restoring communication to patients suffering from “locked-in” syndrome. Research to improve system performance primarily focuses on means to overcome the low signal to noise ratio of electroencephalogric (EEG) recordings. However, the literature and methods are difficult to compare due to the array of evaluation metrics and assumptions underlying them, including that: 1) all characters are equally probable, 2) character selection is memoryless, and 3) errors occur completely at random. The standardization of evaluation metrics that more accurately reflect the amount of information contained in BCI language output is critical to make progress. We present a mutual information-based metric that incorporates prior information and a model of systematic errors. The parameters of a system used in one study were re-optimized, showing that the metric used in optimization significantly affects the parameter values chosen and the resulting system performance. The results of 11 BCI communication studies were then evaluated using different metrics, including those previously used in BCI literature and the newly advocated metric. Six studies'' results varied based on the metric used for evaluation and the proposed metric produced results that differed from those originally published in two of the studies. Standardizing metrics to accurately reflect the rate of information transmission is critical to properly evaluate and compare BCI communication systems and advance the field in an unbiased manner.     

Incorporating natural variability in the bioassessment of stream condition in the Atlantic Forest biome,Brazil

Most bioassessment programs in Brazil face difficulties when scaling up from small spatial scales because larger scales usually encompass great environmental variability. Covariance of anthropogenic pressures with natural environmental gradients can be a confounding factor in the evaluation of biologic responses to anthropogenic pressures. The objective of this study was to develop a multimetric index (MMI) with macroinvertebrates for two stream types and two ecoregions in the Atlantic Forest biome in Rio de Janeiro state, Brazil. We hypothesized that by using two approaches – (1) testing and adjusting metrics to landscape parameters, and (2) selecting metrics using a cluster analysis to avoid metrics redundancy – the final MMI would perform better than the traditional approach (unadjusted metrics, one metric representing each category). Four MMIs were thus developed: MMI-1 – adjusted MMI with metrics selected after cluster analysis); MMI-2 – adjusted MMI with one metric from each category; MMI-3 – unadjusted MMI with metrics selected after cluster analysis; MMI-4 – unadjusted MMI with one metric from each category. We used three decision criteria to assess MMI’s performance: precision, responsiveness and sensitivity. In addition, we tested the MMI’s by using an independent set of sites to validate the results. Although all MMIs performed well in the three criteria, adjusting metrics to natural variation increased MMI response and sensitivity to impairment. In addition, the selected MMI-2 was able to classify sites of two stream types and two ecoregions. The use of cluster analysis, however, did not avoid high redundancy between metrics of different branches. The MMI-4 had the poorest performance among all tested MMIs and it was not able to distinguish adequately reference and impaired sites from both ecoregions. We present some considerations on the use of metrics and on the development of MMI’s in Brazil and elsewhere.     

The impact of rare taxa on a fish index of biotic integrity

The index of biotic integrity (IBI) is a commonly used bioassessment tool that integrates abundance and richness measures to assess water quality. In developing IBIs that are both responsive to human disturbance and resistant to natural variability and sampling error, water managers must decide how to weigh information about rare and abundant taxa, which in turn requires an understanding of the sensitivity of indices to rare taxa. Herein, we investigated the influence of rare fish taxa (within the lower 5% of rank abundance curves) on IBI metric and total scores for stream sites in two of Minnesota's major river basins, the St. Croix (n = 293 site visits) and Upper Mississippi (n = 210 site visits). We artificially removed rare taxa from biological samples by (1) separately excluding each individual taxon that fell within the lower 5% of rank abundance curves; (2) simultaneously excluding all taxa that had an abundance of one (singletons) or two (doubletons); and (3) simultaneously excluding all taxa that fell within the lower 5% of rank abundance curves. We then compared IBI metric and total scores before and after removal of rare taxa using the normalized root mean square error (nRMSE) and regression analysis. The difference in IBI metric and total scores increased as more taxa were removed. Moreover, when multiple rare taxa were removed, the nRMSE was related to sample abundance and to total taxa richness, with greater nRMSE observed in samples with a larger number of taxa or sample abundance. Metrics based on relative abundance of fish taxa were less sensitive to the loss of rare taxa, whereas those based on taxa richness were more sensitive, because taxa richness metrics give more weight to rare taxa compared to the relative abundance metrics.     

Estimating confidence of European WFD ecological status class and WISER Bioassessment Uncertainty Guidance Software (WISERBUGS)

The Water Framework Directive (WFD) requires estimates of the confidence and precision associated with any scheme for assessing and monitoring the ecological status class of any European rivers, lakes, transitional or coastal waters. This is a complex important issue, especially for waterbody assessments based on multiple metrics and/or two or more taxonomic groups. This paper aims to contribute towards improving understanding and providing practical approaches to assessing confidence of class by (i) discussing the various sources and causes of uncertainty, (ii) using UK rivers macroinvertebrate datasets to illustrate the estimation of replicate, temporal and spatial variance components and the implications for water body metric precision, confidence of class and optimal sampling design, (iii) introducing new freely available general software WISER Bioassessment Uncertainty Guidance Software (WISERBUGS) which uses prior sampling uncertainty estimates with user-specified metrics, class limits and metric combination rules to simulate the joint sampling uncertainty in metric EQR values and provide estimates of confidence of class based on individual metrics, (optionally weighted) multi-metric indices and/or multi-metric classification rules (worst case, mean or median class) based on one or more WFD biological quality elements.     

Phase sensitivity analysis of circadian rhythm entrainment

As a biological clock, circadian rhythms evolve to accomplish a stable (robust) entrainment to environmental cycles, of which light is the most obvious. The mechanism of photic entrainment is not known, but two models of entrainment have been proposed based on whether light has a continuous (parametric) or discrete (nonparametric) effect on the circadian pacemaker. A novel sensitivity analysis is developed to study the circadian entrainment in silico based on a limit cycle approach and applied to a model of Drosophila circadian rhythm. The comparative analyses of complete and skeleton photoperiods suggest a trade-off between the contribution of period modulation (parametric effect) and phase shift (nonparametric effect) in Drosophila circadian entrainment. The results also give suggestions for an experimental study to (in)validate the two models of entrainment.     

An automatic EEG-based sleep staging system with introducing NAoSP and NAoGP as new metrics for sleep staging systems

Different biological signals are recorded in sleep labs during sleep for the diagnosis and treatment of human sleep problems. Classification of sleep stages with electroencephalography (EEG) is preferred to other biological signals due to its advantages such as providing clinical information, cost-effectiveness, comfort, and ease of use. The evaluation of EEG signals taken during sleep by clinicians is a tiring, time-consuming, and error-prone method. Therefore, it is clinically mandatory to determine sleep stages by using software-supported systems. Like all classification problems, the accuracy rate is used to compare the performance of studies in this domain, but this metric can be accurate when the number of observations is equal in classes. However, since there is not an equal number of observations in sleep stages, this metric is insufficient in the evaluation of such systems. For this purpose, in recent years, Cohen’s kappa coefficient and even the sensitivity of NREM1 have been used for comparing the performance of these systems. Still, none of them examine the system from all dimensions. Therefore, in this study, two new metrics based on the polygon area metric, called the normalized area of sensitivity polygon and normalized area of the general polygon, are proposed for the performance evaluation of sleep staging systems. In addition, a new sleep staging system is introduced using the applications offered by the MATLAB program. The existing systems discussed in the literature were examined with the proposed metrics, and the best systems were compared with the proposed sleep staging system. According to the results, the proposed system excels in comparison with the most advanced machine learning methods. The single-channel method introduced based on the proposed metrics can be used for robust and reliable sleep stage classification from all dimensions required for real-time applications.Electronic supplementary materialThe online version of this article (10.1007/s11571-020-09641-2) contains supplementary material, which is available to authorized users.     

Seagrass Quality Index (SQI), a Water Framework Directive compliant tool for the assessment of transitional and coastal intertidal areas

This work describes a new method for assessing the ecological quality of intertidal seagrass in estuaries and coastal systems, the Seagrass Quality Index (SQI). The design of the SQI aims to fulfil the Water Framework Directive requirements in terms of compliance (e.g., metrics, sampling procedure, pressure relationship, uncertainty of misclassification and comparability to other methodologies in terms of concept). The index includes three common and easy-to-measure structural parameters of seagrass (i.e., the no. of taxa, bed extent and shoot density) combined in a calculation rule that allows the index to report all five of the quality classes (i.e., high, good, moderate, poor and bad). The present study contains analyses of the relationships between the ecosystem-quality results produced by the index and the pressures measured in the system as well as the relationships between the SQI and the seagrass parameters composing it (both the correlation between the SQI and metrics and the SQI sensitivity to the individual variation of each metric). These relationships were tested using a Spearman rank-correlation analysis, producing significant correlations between the biological metrics and the index results as well as between the index results and the environmental quality-pressure category (i.e., the concentration of winter DIN and turbidity). In terms of management, it is possible to apply the methodology on a broad geographical scale in systems where the reference condition for the number of taxa is even higher than one (for the Mondego studied here, the reference value was one species). The tool fulfilled the WFD requirements, had a robust sampling design and proved to be able to track the inertia that usually exists from the moment the pressure is alleviated as well as the biological response that characterises the recovery phase in systems under restoration.     

Biological indices applied to benthic macroinvertebrates at reference conditions of mountain streams in two ecoregions (Poland, the Slovak Republic)

The study was carried out from 2007 to 2010 in two ecoregions: the Carpathians and the Central Highlands. The objectives of our survey were to test the existing biological index metric based on benthic macroinvertebrates at reference conditions in the high- and mid-altitude mountain streams of two ecoregions according to the requirements of the EU WFD and to determine which environmental factors influence the distribution of benthic macroinvertebrates. Our results revealed statistically significant differences in the values of the physical and chemical parameters of water as well as the mean values of metrics between the types of streams at the sampling sites. RDA analysis showed that the temperature of the water, pH, conductivity, the stream gradient, values of the HQA index, and altitude were the parameters most associated with the distribution of benthic macroinvertebrate taxa and the values of the metrics. The values of biological indices should be considered according to the stream typology including altitude and geology. At the reference conditions, the suggested border values of biological indices are very harsh. The values of the biological indices of most sampling sites did not correspond to the requirements of the high status in rivers. The streams at altitudes above 1,200 m a.s.l. should be treated as another river type and new reference values should be established.     

Enhancing life cycle impact assessment from climate science: Review of recent findings and recommendations for application to LCA

Since the Global Warming Potential (GWP) was first presented in the Intergovernmental Panel on Climate Change (IPCC) First Assessment Report, the metric has been scrutinized and alternative metrics have been suggested. The IPCC Fifth Assessment Report gives a scientific assessment of the main recent findings from climate metrics research and provides the most up-to-date values for a subset of metrics and time horizons. The objectives of this paper are to perform a systematic review of available midpoint metrics (i.e. using an indicator situated in the middle of the cause-effect chain from emissions to climate change) for well-mixed greenhouse gases and near-term climate forcers based on the current literature, to provide recommendations for the development and use of characterization factors for climate change in life cycle assessment (LCA), and to identify research needs. This work is part of the ‘Global Guidance on Environmental Life Cycle Impact Assessment’ project held by the UNEP/SETAC Life Cycle Initiative and is intended to support a consensus finding workshop. In an LCA context, it can make sense to use several complementary metrics that serve different purposes, and from there get an understanding about the robustness of the LCA study to different perspectives and metrics. We propose a step-by-step approach to test the sensitivity of LCA results to different modelling choices and provide recommendations for specific issues such as the consideration of climate-carbon feedbacks and the inclusion of pollutants with cooling effects (negative metric values).     

Comparative study of parameter sensitivity analyses of the TCR-activated Erk-MAPK signalling pathway

Parameter estimation is a major challenge for mathematical modelling of biological systems. Given the uncertainties associated with model parameters, it is important to understand how sensitive the model output is to variations in parameter values. A local sensitivity analysis determines the model sensitivity to parameter variations over a localised region around the nominal parameter values, whereas a global sensitivity analysis (GSA) investigates the sensitivity over the entire parameter space. Using a T-cell receptor-activated Erk-MAPK signalling pathway model as an example, the authors present a comparative study of a variety of different sensitivity analysis techniques. These techniques include: local sensitivity analysis, existing GSA methods of partial rank correlation coefficient, Sobol's, extended Fourier amplitude sensitivity test, as well as a weighted average of local sensitivities and a new GSA method to extract global parameter sensitivities from a parameter identification routine. Results of this study revealed critical reactions in the signalling pathway and their impact on the signalling dynamics and provided insights into embedded regulatory mechanisms such as feedback loops in the pathway. From this study, a recommendation emerges for a general sensitivity analysis strategy to efficiently and reliably infer quantitative, dynamic as well as topological properties from systems biology models.     

Sensitivity Analysis of the APEX Model for Assessing Sustainability of Switchgrass Grown for Biofuel Production in Central Texas

Crop simulation models are increasingly being used to understand the feasibility of large-scale cellulosic biofuel production along with the multi-dimensional impacts on environmental sustainability. However, how the uncertainty in model parameters impacts model performance for sustainability is unclear. In this case study, sensitivity analyses were conducted for three switchgrass sustainability metrics: total biomass production, nitrogen loss, and soil carbon change using the APEX (Agricultural Policy/Environmental eXtender) model. Fifteen out of the 45 parameters (25 crop growth (CROP) parameters and 20 additional model parameters (PARM)) were identified as influential for the three sustainability metrics for three lowland genotypes (WBC, AP13, and KAN) across two locations (Temple, TX, and Austin, TX). Our sensitivity results showed that parameter importance was not dependent on the genotypes but depended on the variables of interest, and differed only slightly between locations. Influential belowground-related CROP and PARM parameters were identified for each sustainability metric, indicating that belowground-related parameters are just as important as commonly measured aboveground CROP parameters. Further investigation of the linear or non-linear relationships and the two-way interactions between each of the individual influential parameters with the three sustainability metrics reflected the functions and characteristics within the APEX model and the interrelations among different processes. Strong interactions between the most influential parameters for total biomass, nitrogen loss, and soil carbon change also highlighted the importance of accurately setting these parameters. Identification of influential model parameters for switchgrass sustainability may help guide field measurements and provide further understanding of the interrelated processes in the APEX model. Furthermore, future field experiments can be designed to measure these influential parameters and understand the non-linear relationships identified between influential parameters and response variables. More accurate model parameterization will help improve APEX model performance and our understanding of the possible underlying physiological mechanisms.     

An evaluation of methods for modelling species distributions

Aim Various statistical techniques have been used to model species probabilities of occurrence in response to environmental conditions. This paper provides a comprehensive assessment of methods and investigates whether errors in model predictions are associated to specific kinds of geographical and environmental distributions of species. Location Portugal, Western Europe. Methods Probabilities of occurrence for 44 species of amphibians and reptiles in Portugal were modelled using seven modelling techniques: Gower metric, Ecological Niche Factor Analysis, classification trees, neural networks, generalized linear models, generalized additive models and spatial interpolators. Generalized linear and additive models were constructed with and without a term accounting for spatial autocorrelation. Model performance was measured using two methods: sensitivity and Kappa index. Species were grouped according to their spatial (area of occupancy and extent of occurrence) and environmental (marginality and tolerance) distributions. Two‐way comparison tests were performed to detect significant interactions between models and species groups. Results Interaction between model and species groups was significant for both sensitivity and Kappa index. This indicates that model performance varied for species with different geographical and environmental distributions. Artificial neural networks performed generally better, immediately followed by generalized additive models including a covariate term for spatial autocorrelation. Non‐parametric methods were preferred to parametric approaches, especially when modelling distributions of species with a greater area of occupancy, a larger extent of occurrence, lower marginality and higher tolerance. Main conclusions This is a first attempt to relate performance of modelling techniques with species spatial and environmental distributions. Results indicate a strong relationship between model performance and the kinds of species distributions being modelled. Some methods performed generally better, but no method was superior in all circumstances. A suggestion is made that choice of the appropriate method should be contingent on the goals and kinds of distributions being modelled.