Estimates and comparisons of the effects of sampling variation using ‘national’ macroinvertebrate sampling protocols on the precision of metrics used to assess ecological status

The Water Framework Directive (WFD) of the European Union requires all member countries to provide information on the level of confidence and precision of results in their river monitoring programmes to assess the ecological status class of river sites. As part of the European Union project STAR, the overall effects of sampling variation for a wide range of commonly used metrics and sampling methods were assessed. Replicate samples were taken in each of two seasons at 2–6 sites of varying ecological status class within each of 18 stream types spread over 12 countries, using both the STAR-AQEM method and a national sampling method or, where unavailable, the RIVPACS sampling protocol. The sampling precision of a combination of sampling method and metric was estimated by expressing the replicate sampling variance as a percentage P_samp of the total variance in metric values with a stream type; low values of P_samp indicate high precision. Most metrics had percentage sampling variances less than 20% for all or most stream types and methods. Most national methods including RIVPACS had sampling precisions at least as good as those for the STAR-AQEM method as used in their country at the same sites; the main exceptions were the national methods used in Latvia and Sweden. The national methods used in the Czech Republic, Denmark, France, Poland and the RIVPACS method used in the UK and Austria all had percentage sampling variances of less than 10% for the majority of metrics assessed. In contrast, none of the metrics had percentage sampling variances less than 10% when based on either the Italian (IBE) method, which used bank-side sorting, or the Latvian national method which identifies only a limited set of taxa. P_samp was lowest on average for the two stream types sampled in the Czech Republic using either the PERLA national method or the STAR-AQEM method. Averaged over all stream types and methods, the three Saprobic-based metrics had the lowest average percentage sampling variances (3–6%) amongst the 26 metrics assessed. These estimates of sampling standard deviation can be used to help assess the uncertainty in single or multi-metric systems for estimating site ecological status using the general STAR Bioassessment Guidance Software (STARBUGS) developed within the STAR project.     

Uncertainty associated with river health assessment in a varying environment: The case of a predictive fish-based index in France

Sensitive biological measures of river ecosystem quality are needed to assess, maintain or restore ecological conditions of water bodies. Since our understanding of these complex systems is imperfect, decision-making requires recognizing uncertainty. In this study, a new predictive multi-metric index based on fish functional traits was developed to assess French rivers. Information on fish assemblage structure, local environment and human-induced disturbances of 1654 French river sites was compiled. A Bayesian framework was used to predict theoretical metric values in absence of human pressure and to estimate the uncertainty associated with these predictions. The uncertainty associated with the index score gives the confidence associated with the evaluation of site ecological conditions.Among the 228 potential metrics tested, only 11 were retained for the index computation. The final index is independent from natural variability and sensitive to human-induced disturbances. In particular, it is affected by the accumulation of different degradations and specific degradations including hydrological perturbations. Predictive uncertainty is globally lower for IPR+ than for underlying metrics.This new methodology seems appropriate to develop bio-indication tools accounting for uncertainty related to reference condition definition and could be extended to other biological groups and areas. Our results support the use of multi-metric indexes to assess rivers and strengthen the idea that examination of uncertainty could contribute greatly to the improvement of the assessment power of bio-indicators.     

The future of fish-based ecological assessment of European rivers: from traditional EU Water Framework Directive compliant methods to eDNA metabarcoding-based approaches

Most of the present EU Water Framework Directive (WFD) compliant fish-based assessment methods of European rivers are multi-metric indices computed from traditional electrofishing (TEF) samples, but this method has known shortcomings, especially in large rivers. The probability of detecting rare species remains limited, which can alter the sensitivity of the indices. In recent years, environmental (e)DNA metabarcoding techniques have progressed sufficiently to allow applications in various ecological domains as well as eDNA-based ecological assessment methods. A review of the 25 current WFD-compliant methods for river fish shows that 81% of the metrics used in these methods are expressed in richness or relative abundance and thus compatible with eDNA samples. However, more than half of the member states' methods include at least one metric related to age or size structure and would have to adapt their current fish index if reliant solely on eDNA-derived information. Most trait-based metrics expressed in richness are higher when computed from eDNA than when computed from TEF samples. Comparable values are obtained only when the TEF sampling effort increases. Depending on the species trait considered, most trait-based metrics expressed in relative abundance are significantly higher for eDNA than for TEF samples or vice versa due to over-estimation of sub-surface species or under-estimation of benthic and rare species by TEF sampling, respectively. An existing predictive fish index, adapted to make it compatible with eDNA data, delivers an ecological assessment comparable with the current approved method for 22 of the 25 sites tested. Its associated uncertainty is lower than that of current fish indices. Recommendations for the development of future fish eDNA-based indices and the associated eDNA water sampling strategy are discussed.     

Effects of sampling and sub-sampling variation using the STAR-AQEM sampling protocol on the precision of macroinvertebrate metrics

As part of the extensive field sampling programme within the European Union STAR project, replicate macroinvertebrate samples were taken using the STAR-AQEM sampling method at each of 2–13 sites of varying ecological quality within each of 15 stream types spread over 12 countries throughout Europe. The STAR-AQEM method requires the sub-sampling and taxonomic identification of at least one-sixth of the sample and at least 700 individuals. Replicate sub-samples were also taken at most of these sites. Sub-sampling effects caused more than 50% of the overall variance between replicate samples values for 12 of the 27 macroinvertebrate metrics analysed and was generally greatest for metrics that depend on the number of taxa present. The sampling precision of each metric was estimated by the overall replicate sampling variance as a percentage P_samp of the total variance in metric values within a stream type. Average over all stream types, the three Saprobic indices had the lowest percentage sampling variances with median values of only 3–6%. Most of the metrics had typical replicate sampling variances of 8–18% of the total variability within a stream type; this gives rise to estimated rates of mis-classifying sites to ecological status class of between 22 and 55% with an average of about 40%. This suggests that the precision of such metrics based on the STAR-AQEM method is only sufficient to indicate gross changes in the ecological status of sites, but there will be considerable uncertainty in the assignment of sites to adjacent status classes. These estimates can be used to provide information on the effects of STAR-AQEM sampling variation on the expected uncertainty in multi-metric assessments of the ecological status of sites in the same or similar stream types, where only one sample has been taken at a point in time and thus there is no replication.     

Quantifying uncertainties in biologically-based water quality assessment: A pan-European analysis of lake phytoplankton community metrics

Lake phytoplankton are adopted world-wide as a sensitive indicator of water quality. European environmental legislation, the EU Water Framework Directive (WFD), formalises this, requiring the use of phytoplankton to assess the ecological status of lakes and coastal waters. Here we provide a rigorous assessment of a number of proposed phytoplankton metrics for assessing the ecological quality of European lakes, specifically in response to nutrient enrichment, or eutrophication, the most widespread pressure affecting lakes. To be useful indicators, metrics must have a small measurement error relative to the eutrophication signal we want them to represent among lakes of different nutrient status. An understanding of variability in metric scores among different locations around a lake, or due to sampling and analytical variability can also identify how best this measurement error is minimised.To quantify metric variability, we analyse data from a multi-scale field campaign of 32 European lakes, resolving the extent to which seven phytoplankton metrics (including chlorophyll a, the most widely used metric of lake quality) vary among lakes, among sampling locations within a lake and through sample replication and processing. We also relate these metrics to environmental variables, including total phosphorus concentration as an indicator of eutrophication.For all seven metrics, 65–96% of the variance in metric scores was among lakes, much higher than variability occurring due to sampling/sample processing. Using multi-model inference, there was strong support for relationships between among-lake variation in three metrics and differences in total phosphorus concentrations. Three of the metrics were also related to mean lake depth. Variability among locations within a lake was minimal (<4%), with sub-samples and analysts accounting for much of the within-lake metric variance. This indicates that a single sampling location is representative and suggests that sub-sample replication and standardisation of analyst procedures should result in increased precision of ecological assessments based upon these metrics.For three phytoplankton metrics being used in the WFD: chlorophyll a concentration, the Phytoplankton Trophic Index (PTI) and cyanobacterial biovolume, >85% of the variance in metric scores was among-lakes and total phosphorus concentration was well supported as a predictor of this variation. Based upon this study, we can recommend that these three proposed metrics can be considered sufficiently robust for the ecological status assessment of European lakes in WFD monitoring schemes.     

Assessment of sources of uncertainty in macrophyte surveys and the consequences for river classification

The application of macrophytes in freshwater monitoring is still relatively limited and studies on their intercalibration and sources of variation are required. Therefore, the aim of the study was to compare selected indices and metrics based on macrophytes and to quantify their variability. During the STAR project, several aspects influencing uncertainty in estimation of the ecological quality of river were assessed. Results showed that several metrics based on the indicative value of plant species can be used in evaluation of the ecological status of rivers. Among estimated sources of variance in metric values the inter-surveyor differences had the lowest effect and slightly stronger were the influences of temporal variation (years and seasons) and shading. The impact of habitat modification was the most important factor. Analysis showed that some of macrophyte-based metrics (notably MTR and IBMR) are of sufficient precision in terms of sampling uncertainty, that they could be useful for estimating the ecological status of rivers in accordance with the aims of the Water Framework Directive.     

Errors and uncertainty in bioassessment methods – major results and conclusions from the STAR project and their application using STARBUGS

The STAR project’s extensive replicated sampling programmes have provided the first ever quantitative comparative studies of the susceptibility of a wide range of national macroinvertebrate sampling methods and taxonomic metrics to uncertainty resulting from the effects of field sampling variability and subsequent sub-sampling and laboratory (or bank-side) procedures and protocols. We summarise six STAR project papers examining various aspects of the potential sources of uncertainty in the observed fauna and observed metric values. The use of new simulation software STARBUGS (STAR Bioassessment Uncertainty Software System) to incorporate the effects of these potential errors into quantitative assessments of the uncertainty in assigning water bodies to WFD ecological status classes is discussed.     

The development of an ecological quality assessment and classification system for Greek running waters based on benthic macroinvertebrates

A first approach towards an integrated, macroinvertebrate based, river type specific system for the assessment and classification of the ecological quality in Greek rivers affected by nutrient enrichment is presented. This system, which was developed within the framework of the AQEM project application in Greece, is regionally adapted and integrated, since it contains a multitude of biotic and abiotic factors affecting taxa presences and abundances. The methodology applied is very promising for the application of the Water Framework Directive (WFD) in Greece. According to the AQEM project, a seasonal `multi habitat sampling' procedure, at 45 sampling sites, scattered over three river types, was applied. In addition, a large set of hydrological, morphological, geological, biological, hydrochemical parameters and anthropogenic factors were considered. The system is based on a new, river type specific biotic metric, adapted for Greek conditions (BMG) and a new nutrient pollution metric. The developed metrics showed high correlations with each other. Hence, BMG seems to be suitable for the ecological quality assessment of Greek running waters affected by nutrients. Moreover, by applying BMG, a final classification of the rivers into five quality classes, according to the demands of the WFD, was achieved.     

Sensitivity analysis to explore responsiveness and dynamic range of multi-metric fish-based indices for assessing the ecological status of estuaries and lagoons

Fish-based multi-metric indices are an integral tool in implementing effective water policy initiatives for transitional waters. This study analysed the behaviour of three fish indices (TFCI in the UK, ELFI in France and EFAI in Portugal) developed for monitoring in line with the European Water Framework Directive, by applying a supervised multi-way sensitivity analysis with national monitoring data. The relative variation in the index outcome was analysed under different simulation scenarios by setting metric values at various levels and accounting for the co-variation between metrics. Subsets of key metrics were identified based on their higher influence in determining the index output. Index results under manipulations of metric scores clearly indicated that metric type, number of metrics used and correlations between metrics are important in determining their behaviour. This has implications for implementing management/conservation plans, for example, by prioritising restoration and/or conservation of metrics influencing more the ecological status. Indices including uncorrelated metrics (e.g. EFAI) or metrics with a skewed distribution (e.g. TFCI) are less affected by extreme metric changes, reducing the effectiveness of management actions aimed at improving the ecological status of a water body and thus adjustments may be needed to increase their sensitivity to changes in their metrics.     

Evaluating the relative contributions of hydroperiod and soil fertility on growth of south Florida mangroves

The requirements of the European Water Framework Directive (WFD), aimed at an integrative assessment methodology for evaluating the ecological status of water bodies are frequently being achieved through multimetric techniques, i.e. by combining several indices, which address different stressors or different components of the biocoenosis. This document suggests a normative methodology for the development and application of Multimetric Indices as a tool with which to evaluate the ecological status of running waters. The methodology has been derived from and tested on a European scale within the framework of the AQEM and STAR research projects, and projects on the implementation of the WFD in Austria and Germany. We suggest a procedure for the development of Multimetric Indices, which is composed of the following steps: (1) selection of the most suitable form of a Multimetric Index; (2) metric selection, broken down into metric calculation, exclusion of numerically unsuitable metrics, definition of a stressor gradient, correlation of stressor gradients and metrics, selection of candidate metrics, selection of core metrics, distribution of metrics within the metric types, definition of upper and lower anchors and scaling; (3) generation of a Multimetric Index (general or stressor-specific approach); (4) setting class boundaries; (5) interpretation of results. Each step is described by examples.     

Development and validation of a fish-based index for the assessment of 'river health' in France

SUMMARY 1. In Europe, water policy is currently undergoing considerable change as emphasised by the recent European Water Framework Directive (WFD), which requires the restoration and maintenance of 'healthy' aquatic ecosystems by the assessment of their hydromorphological, chemical and biological characteristics. If the requirements of the WFD are to be met, effective biological tools are needed to measure the 'health' of rivers at scales large enough to be useful for management. These tools need to be ecologically based, efficient, rapid and applicable in different ecological regions. Among potential indicators, fish assemblages are of particular interest because of their ability to integrate environmental variability at different spatial scales. To meet the goals of the WFD, the French Water Agencies and the Ministry of the Environment initiated a research programme to develop a fish-based index that would be applicable nationwide.
2. A variety of metrics based on occurrence and abundance data and reflecting different aspects of the fish assemblage structure and function were selected from available literature and for their potential to indicate degradation.
3. Logistic and multiple linear regression procedures were applied, using an initial data set of 650 reference sites fairly evenly distributed across French rivers and defined by some easily measured regional and local characteristics, to elaborate the simplest possible response model that adequately explained the observed patterns of each metric for a given site.
4. Models obtained for each metric were validated using two independent data sets of 88 reference sites and 88 disturbed sites. These procedures allowed us to select the most effective metrics in discriminating between reference and disturbed sites.     

广西河池地区河流基于鱼类的生物完整性指数筛选及其环境质量评估

为促进生物完整性评价法在中国的应用,加强珠江上游广西河池地区河流环境质量监测,该研究以鱼类为研究对象,初步构建了包含21个指标在内的广西河池地区河流基于鱼类的生物完整性评价的指标体系。根据鱼类调查所收集数据,结合历史资料的记录,对这些指标进行筛选和赋值,最终确立了6个指标,用于建立适合广西河池地区河流的基于鱼类的生物完整性指标体系;应用此指标体系对该地区部分河流(河段)进行了评价。评价的结果表明,小环江鱼类完整性为好;红水河、龙江和大环江为一般;刁江为极差。评价的结果与河流受人为干扰的实际情况相吻合,研究构建的评价体系可供河池周边地区使用和借鉴。生物完整性评价是水域环境质量监测的重要手段,为使生物完整性评价更加科学和客观,应加强生物完整性与环境因子之间关系的研究。     

River macrophyte indices: not the Holy Grail!

1. Recent studies have demonstrated that there is generally no unambiguous relationship between plant species composition and specific environmental conditions in rivers. Nevertheless, indices of environmental pressures based on macrophytes are flourishing, because of the requirements of the Water Framework Directive (WFD). 2. We first reviewed nine such indices against 13 criteria for bioindicators. Then, using data from France and England, we tested whether the IBMR (Macrophyte Biological Index for Rivers) and LEAFPACS (predictions and classification system for macrophytes) methods could reliably indicate nutrient and hydromorphological pressures. Finally, we used an improved bootstrapping method to estimate accuracy. 3. Currently, most indices lack ecological meaning for a variety of reasons, including partial sampling (backwaters are excluded); reliance on list of taxa (there are identification difficulties) rather than structure and functions; correlation rather than causation; application within a limited biogeographical area; reliance on ‘expert’ judgement; high precision but poor accuracy; poorly defined reference conditions; lack of independent tests; and an inability to discriminate reliably between the target pressures of interest from confounding background variables. 4. IBMR was a far better indicator of pH (or HCO₃‐pCO₂) than it was of soluble reactive phosphorus, SRP (or SRP‐NH₄). While there was a highly significant correlation between IBMR and SRP after removing the effect of pH, the relationship was weak (r² = 0.08, n = 215, P < 0.001). 5. LEAFPACS is a multi‐metric method summing up five individual indices, each compliant with the WFD. Its individual metrics were not better correlated with nutrient and hydromorphological pressures (with r² < 0.1, n = 62, P < 0.05) than was the IBMR. The meaning of the overall metric is questionable. 6. There are problems in determining the precision of the indices, owing to uncertainties in recording, but they are less than the uncertainties in determining accuracy (because species optima and tolerances are sometimes poorly known). 7. Reliable information is needed to improve the state of our rivers. Macrophyte indices are able to detect statistically significant pressures from a large population of sites but cannot be applied at specific sites, as required by the WFD, owing to large uncertainties and low explanatory power. Typically, more than 90% of the variability in macrophyte indices is attributed to factors other than human pressure. The WFD would be better served by a simpler, holistic approach based on our current mechanistic understanding of river processes. These findings are likely to apply also to other taxonomic groups (macroinvertebrates, diatoms, fish) used in the assessment of purported ecological quality and to palaeolimnological measures of reference status.     

Combination of multiple biological quality elements into waterbody assessment of surface waters

The Water Framework Directive (WFD) requires EU Member States to classify the ecological status of surface waters by using multiple biological quality elements (BQEs). According to the WFD Classification Guidance, a ‘one-out-all-out’ (OOAO) rule should be applied when integrating multiple BQEs into an overall biological status of a waterbody, i.e. classification is determined by the lowest status BQE. Using both simulated and monitoring datasets, we analyzed the effects of different combination rules in classification outcome and classification reliability. The OOAO represented the strictest combination rule in terms of increased probabilities of waterbodies being in moderate or worse status in comparison to other rules. The OOAO approach gave acceptable results when different BQEs were complementary, showing the effects of different pressures, and when level of uncertainty in the metrics used in the assessment was not high. Increasing the number of BQEs used in the assessment affected the classification outcome when using the OOAO approach; this was especially problematic if all BQEs address the same pressure. Our study showed that grouping of metrics and metrics uncertainty has a large influence on classification outcomes and that this should be carefully considered to ensure that final classification adequately reflects ecological status.     

An angiosperm quality index (AQI) for Cantabrian estuaries

Angiosperms are an important biological parameter for establishing the status in transitional waters. Angiosperms grow from subtidal to upper intertidal zones and are sensitive to the most important pressures present in estuaries. A multi-metric for determining the status of Cantabrian estuaries has been developed by taking into account the angiosperms that live in these environments. This metric integrates two important directives: the Water Framework Directive (WFD) and the Habitats Directive. To be precise, the diversity of the existing habitats, the coverage in relative terms with respect to the optimal situation, and the loss of extension are considered in the metric. The application of the method to three Cantabrian estuaries subjected to different pressures demonstrates that the metric captures the degree of pressures of each estuary. The multi-metric method allows us to identify the type of problems to be tackled by the relevant authorities in the programme of measures of the river basin management plan.     

Long-term monitoring studies as a powerful tool in marine ecosystem research

The EU-funded research project WISER (“Water bodies in Europe: Integrative Systems to assess Ecological status and Recovery”) developed new assessment methods required by the EU Water Framework Directive (WFD) for lakes, coastal and transitional waters. WISER also addressed the recovery of biotic assemblages from degradation. The results are summarised in five key messages, supported by papers in this special issue and by WISER results published elsewhere: (1) Response to stress differs between organism groups, water types and stressors; a conceptual model is proposed summarising how the individual organism groups respond to different types of degradation in rivers, lakes, transitional and coastal waters. (2) The sources of uncertainty differ between BQEs and water types, leading to methodological suggestions on how to design WFD sampling programmes. (3) Results from about 300 current assessment methods indicate geographical variations in metrics but assessments are comparable at an aggregated level (“ecological status”). (4) Scale and time matter; restoration requires action at (sub)-basin levels and recovery may require decades. (5) Long-term trends require consideration; the effects of both degradation and restoration at the water body or river basin scales is increasingly superimposed by multiple stressors acting at large scales, in particular by climate change.     

Biological indices applied to benthic macroinvertebrates at reference conditions of mountain streams in two ecoregions (Poland, the Slovak Republic)

The study was carried out from 2007 to 2010 in two ecoregions: the Carpathians and the Central Highlands. The objectives of our survey were to test the existing biological index metric based on benthic macroinvertebrates at reference conditions in the high- and mid-altitude mountain streams of two ecoregions according to the requirements of the EU WFD and to determine which environmental factors influence the distribution of benthic macroinvertebrates. Our results revealed statistically significant differences in the values of the physical and chemical parameters of water as well as the mean values of metrics between the types of streams at the sampling sites. RDA analysis showed that the temperature of the water, pH, conductivity, the stream gradient, values of the HQA index, and altitude were the parameters most associated with the distribution of benthic macroinvertebrate taxa and the values of the metrics. The values of biological indices should be considered according to the stream typology including altitude and geology. At the reference conditions, the suggested border values of biological indices are very harsh. The values of the biological indices of most sampling sites did not correspond to the requirements of the high status in rivers. The streams at altitudes above 1,200 m a.s.l. should be treated as another river type and new reference values should be established.     

Addressing a gap in the Water Framework Directive implementation: Rocky shores assessment based on benthic macroinvertebrates

A gap in the European Water Framework Directive (WFD) is addressed, aiming for the development of an ecological quality status assessment tool based solely on the Biological Quality Element benthic macroinvertebrates from intertidal rocky shores. The proposed Rocky shore Macroinvertebrates Assessment Tool (RMAT) was tested and validated along disturbance gradients (organic enrichment). During the whole process, the response of widely used metrics (e.g. Hurlbert index, Shannon-Wiener index, AZTI’s Marine Biotic Index; Bentix biotic index) and models (i.e., metrics combined) was compared to results provided by the Marine Macroalgae Assessment Tool to the same sampling sites.The RMAT is a multimetric index compliant with the WFD based on the benthic macroinvertebrates community, combining ‘abundance’ (Hurlbert index) and ‘taxonomic composition’ (Bentix index using density and biomass data) metrics. It performed well along anthropogenic disturbance gradients, showing ecological quality increasing from close to far away from the disturbance.The RMAT is a promising tool for rocky shore ecological assessment in the scope of the WFD or other monitoring activities worldwide.     

Seagrass Quality Index (SQI), a Water Framework Directive compliant tool for the assessment of transitional and coastal intertidal areas

This work describes a new method for assessing the ecological quality of intertidal seagrass in estuaries and coastal systems, the Seagrass Quality Index (SQI). The design of the SQI aims to fulfil the Water Framework Directive requirements in terms of compliance (e.g., metrics, sampling procedure, pressure relationship, uncertainty of misclassification and comparability to other methodologies in terms of concept). The index includes three common and easy-to-measure structural parameters of seagrass (i.e., the no. of taxa, bed extent and shoot density) combined in a calculation rule that allows the index to report all five of the quality classes (i.e., high, good, moderate, poor and bad). The present study contains analyses of the relationships between the ecosystem-quality results produced by the index and the pressures measured in the system as well as the relationships between the SQI and the seagrass parameters composing it (both the correlation between the SQI and metrics and the SQI sensitivity to the individual variation of each metric). These relationships were tested using a Spearman rank-correlation analysis, producing significant correlations between the biological metrics and the index results as well as between the index results and the environmental quality-pressure category (i.e., the concentration of winter DIN and turbidity). In terms of management, it is possible to apply the methodology on a broad geographical scale in systems where the reference condition for the number of taxa is even higher than one (for the Mondego studied here, the reference value was one species). The tool fulfilled the WFD requirements, had a robust sampling design and proved to be able to track the inertia that usually exists from the moment the pressure is alleviated as well as the biological response that characterises the recovery phase in systems under restoration.