Integrated assessment of ecological status and misclassification of lakes: The role of uncertainty and index combination rules

The European Water Framework Directive (WFD) requires that the ecological status of waterbodies is assessed using multiple biological quality elements (BQEs) that are combined into a single status class. The recommended combination rule (the “one-out, all-out” rule; OOAO) has been criticised for being unreasonably conservative and for being sensitive to uncertainty. In this study, the objective was to compare the sensitivity to uncertainty of four different combination rules: (1) OOAO, (2) OOAO with exclusion of one element, (3) average and (4) weighted average. Index values for 5 BQEs (phytoplankton, phytobenthos, macrophytes, macroinvertebrates and fish) sampled from 10 lakes in the Wel River catchment in Poland were used to classify the lakes according to the OOAO and the three alternative combination rules. Based on the mean and (where possible) standard deviation of these index values, we modelled the risk of misclassification by simulating 10,000 resamples for each BQEs in each lake, classifying each resample and calculating the proportion of misclassified resamples under each combination rule. For individual BQEs, the risk of misclassification increased both with higher uncertainty (standard deviation) and with the proximity of the index value to a class boundary. Under the OOAO rule, the risk of misclassification was more biased towards worse status (“underclassification”) than towards better status. Furthermore, risk of underclassification was more affected by uncertainty under the OOAO rule compared with the alternative combination rules. This analysis has demonstrated the weaknesses associated with the OOAO rule for integration of BQEs for lake classification. However, the alternative combination rules are associated with other shortcomings, such as the need for subjective judgement, and involve a higher risk of not protecting the most sensitive BQE and thus the whole ecosystem. We recommend that future versions of instructions for WFD implementation consider alternatives to the OOAO combination rule, and provide guidelines for weighting of individual BQEs.     

Biological indicators track differential responses of pelagic and littoral areas to nutrient load reductions in German lakes

External nutrient loading was reduced over the past decades as a measure for improving the water quality of eutrophic lakes in western Europe, and has since been accelerated by the adoption of the European Water Framework Directive (WFD) in 2000 (EC, 2000). A variety of eutrophication-related metrics have indicated that the response of biological communities to this decreased nutrient loading has been diverse. Phytoplankton, a major component of the pelagic community, often responded rapidly, whereas a significant delay was observed for submerged macrophytes colonizing littoral areas. In this study we tested whether assessment methods developed for phytoplankton and macrophytes in lakes during Germany's implementation of the WFD reflect this differential response. An assessment of 263 German lakes confirmed that a lower ecological state was recorded when based on the biological quality element (BQE) for macrophytes than the BQE for phytoplankton during the investigated period (2003–2012). On average, lakes had a moderate ecological status for both phytoplankton and macrophyte BQEs, but differences of up to three classes were observed in single cases. Long-term data were available for five lowland lakes subject to strong reductions in phosphorus loading. Their phytoplankton-based assessments indicated a constant improvement of the ecological status in parallel to decreasing water phosphorus concentrations. In contrast, macrophyte-based assessments indicated a 10–20 year delay in their ecological recovery following nutrient load reduction. This delay was confirmed by detailed data on the temporal development of macrophyte species diversity and maximum colonization depths of two lakes after nutrient load reduction. We conclude that the available WFD assessment methods for phytoplankton and macrophyte BQEs are suitable to track the differential response of pelagic and littoral areas to nutrient load reductions in German lakes.     

Long-term monitoring studies as a powerful tool in marine ecosystem research

The EU-funded research project WISER (“Water bodies in Europe: Integrative Systems to assess Ecological status and Recovery”) developed new assessment methods required by the EU Water Framework Directive (WFD) for lakes, coastal and transitional waters. WISER also addressed the recovery of biotic assemblages from degradation. The results are summarised in five key messages, supported by papers in this special issue and by WISER results published elsewhere: (1) Response to stress differs between organism groups, water types and stressors; a conceptual model is proposed summarising how the individual organism groups respond to different types of degradation in rivers, lakes, transitional and coastal waters. (2) The sources of uncertainty differ between BQEs and water types, leading to methodological suggestions on how to design WFD sampling programmes. (3) Results from about 300 current assessment methods indicate geographical variations in metrics but assessments are comparable at an aggregated level (“ecological status”). (4) Scale and time matter; restoration requires action at (sub)-basin levels and recovery may require decades. (5) Long-term trends require consideration; the effects of both degradation and restoration at the water body or river basin scales is increasingly superimposed by multiple stressors acting at large scales, in particular by climate change.     

Estimating confidence of European WFD ecological status class and WISER Bioassessment Uncertainty Guidance Software (WISERBUGS)

The Water Framework Directive (WFD) requires estimates of the confidence and precision associated with any scheme for assessing and monitoring the ecological status class of any European rivers, lakes, transitional or coastal waters. This is a complex important issue, especially for waterbody assessments based on multiple metrics and/or two or more taxonomic groups. This paper aims to contribute towards improving understanding and providing practical approaches to assessing confidence of class by (i) discussing the various sources and causes of uncertainty, (ii) using UK rivers macroinvertebrate datasets to illustrate the estimation of replicate, temporal and spatial variance components and the implications for water body metric precision, confidence of class and optimal sampling design, (iii) introducing new freely available general software WISER Bioassessment Uncertainty Guidance Software (WISERBUGS) which uses prior sampling uncertainty estimates with user-specified metrics, class limits and metric combination rules to simulate the joint sampling uncertainty in metric EQR values and provide estimates of confidence of class based on individual metrics, (optionally weighted) multi-metric indices and/or multi-metric classification rules (worst case, mean or median class) based on one or more WFD biological quality elements.     

Transitional and coastal waters ecological status assessment: advances and challenges resulting from implementing the European Water Framework Directive

The derivation, performance, sensitivity and inherent uncertainty of ecological quality indicators have become major topics in developing tools for the management of marine, transitional and coastal waters. In reviewing the advances in these waters, related to an ecological status assessment, we show the future challenges to be addressed within the European Water Framework Directive (WFD). Using new analyses carried out under the research project ‘Water Bodies in Europe: Integrative Systems to Assess Ecological status and Recovery’, we provide a complete set of assessments for the biological quality elements (BQEs) (phytoplankton, macroalgae-seagrasses, macroinvertebrates and fish) to be assessed, as well as the validation of existing indicators and multimetric indices and, in some cases, the development of new assessment indices. We show that these indices respond differently to different human pressures and they each have challenges in defining reference conditions against which future changes are judged. In investigating good ecological potential, as the response to heavily modified water bodies, we show that there are flaws in the Directive, not least in its definitions. Our analyses have also focussed on uncertainty in using the indices and we emphasise the problems of defining ecological class boundaries based on indices which themselves may be combined indices (multimetrics). The analysis shows that some of those multimetrics are redundant and/or are inter-correlated and thus may reduce the sensitivity in defining ecological class boundaries. If this is related to the drivers-pressures-state change-impacts-response approach then there are lessons for management measures aimed at achieving good ecological status and even the potential for legal challenges to decisions based on uncertain indices under the WFD. Hence, we conclude the continued need for advances in assessing pressures and gradients, and defining reference conditions for state change, index development, impact assessment and the validation of indices for each BQE.     

Ecological assessment of running waters: Do macrophytes,macroinvertebrates, diatoms and fish show similar responses to human pressures?

This study aimed to compare the intensity and the sensitivity of the responses of four river biological quality elements (BQEs) – macrophytes, fish, diatoms and macroinvertebrates – to human pressures excluding natural variations in stream ecosystem functioning. Biological, water quality and hydro-morphological data were compiled for 290 French river sites.Out of the 93 metrics tested, 51 covering the four BQEs responded significantly to global degradation. The responses to specific pressures were consistent with the BQEs’ ecological and biological characteristics. For the four BQEs, metrics responded strongly to water quality degradations. Like fish, macroinvertebrate metrics were very sensitive to morphological degradations such as the presence of an impoundment, while diatom and macrophyte metrics did not show strong responses to these changes. Among the four BQEs’ metrics, fish metrics responded the strongest to hydrological perturbations. Although a high proportion of the metrics responded only to high levels of human-induced degradations, trait-based metrics seemed the most sensitive and responded to lower levels of pressure. Global and water quality degradations of the river appear to be better detected by BQE metrics than channel morphological and hydrological degradations. Our results highlight the different impacts of human-induced pressures on the four BQE metrics and the challenging task of assessing the effect of single pressures when most of sites are multi-impacted.     

A generic aggregation approach to account for statistical uncertainty when combining multiple assessment results

The one-out-all-out approach (OOAO) for aggregating the assessments of single elements (e.g. species or ecosystem components) has found application in environmental polices such as the European Water Framework Directive or the Marine Strategy Framework Directive (MSFD). However, the OOAO has been challenged as being too pessimistic by making positive assessment results virtually impossible along the increasing number of aggregated elements. This study presents a generic approach, the probabilistic One-out-all-out approach, (pOOAO), to account for this issue and thereby reconciling the OOAO with probabilistic aggregation methods The pOOAO allows to determine the minimum number of elements (K_GES), which should meet their assessment benchmarks and thus should achieve a good status. By pre-setting a generic confidence level for each single assessment (e.g. 0.95) the binomial distribution can be used to obtain K_GES for any number of assessed elements. The pOOAO can also accommodate for the integration of assessments from multiple indicators within an element by adjusting the confidence level in relation to the number of integrated indicators. Depending on the generic confidence level as well as on the number of and integrated indicators and aggregated elements, the pOOAO is either consistent with the OOAO or allows for a certain number of negative assessment results, which are attributed to statistical uncertainty and error propagation. The pOOAO is consistent with the OOAO if the desired confidence level in the single assessment results is very high (>0.99) and/or the number of aggregated elements and integrated indicators is low. Through this flexibility the pOOAO could find wide application within integrated ecosystem assessments frameworks such as the MSFD, but would require to estimate the confidence level for each single assessment.     

Exploring the robustness of macrophyte-based classification methods to assess the ecological status of coastal and transitional ecosystems under the Water Framework Directive

Identifying and quantifying the factors that contribute to the potential misclassification of the ecological status of water bodies is a major challenge of the Water Framework Directive (WFD). The present study compiles extensive biomonitoring data from a range of macrophyte-based classification methods developed by several European countries. The data reflect spatial and temporal variation as well as inter-observer variation. Uncertainty analysis identified that factors related to the spatial scale of sampling generally contributed most to the uncertainty in classifying water bodies to their ecological status, reflecting the high horizontal and depth-related heterogeneity displayed by macrophyte communities. In contrast, the uncertainty associated with temporal variation was low. In addition, inter-observer variation, where assessed, did not contribute much to overall uncertainty, indicating that these methods are easily transferable and insensitive to observer error. The study, therefore, suggests that macrophyte-based sampling schemes should prioritize large spatial replication over temporal replication to maximize the effectiveness and reliability of water body classification within the WFD. We encourage conducting similar uncertainty analyses for new/additional ecological indicators to optimize sampling schemes and improve the reliability of classification of ecological status.     

Coherence of phytoplankton and attached diatom-based ecological status assessment in Lake Balaton

Coherence between ecological status assessment by phytoplankton and attached diatoms was analyzed in the littoral zone of Lake Balaton. Sampling of periphytic diatoms, phytoplankton, and water were carried out at ten different littoral sites in the northern and southern shores of the lake for a year. Phytoplankton species were sorted into functional groups and ecological status was assessed by means of the phytoplankton assemblage Q index. The index TDIL was calculated using quantitative attached diatom data. Significant differences were found between the ecological assessments based on phytoplankton and phytobenthos metrics, both seasonally and spatially. The Q index indicated ecological states varying from bad to good, while the average of diatom indices varied from moderate to high conditions. The Q index provided more realistic ecological status of Lake Balaton, compared with trophic status based on TP values, especially in the summer period. Differences in the response-time indication of phytoplankton and attached diatoms suggest that lack of coherence should also be expected between the responses of other BQEs.     

Measurements of uncertainty in macrophyte metrics used to assess European lake water quality

Uncertainty is an important factor in ecological assessment, and has important implications for the ecological classification and management of lakes. However, our knowledge of the effects of uncertainty in the assessment of different ecological indicators is limited. Here, we used data from a standardized campaign of aquatic plant surveys, in 28 lakes from 10 European countries, to assess variation in macrophyte metrics across a set of nested spatial scales: countries, lakes, sampling stations, replicate transects, and replicate samples at two depth-zones. Metrics investigated in each transect included taxa richness, maximum depth of colonisation and two indicators of trophic status: Ellenberg’s N and a metric based on phosphorus trophic status. Metrics were found to have a slightly stronger relationship to pressures when they were calculated on abundance data compared to presence/absence data. Eutrophication metrics based on helophytes were found not to be useful in assessing the effects of nutrient pressure. These metrics were also found to vary with the depth of sampling, with shallower taxa representing higher trophic status. This study demonstrates the complex spatial variability in macrophyte communities, the effect of this variability on the metrics, and the implications to water managers, especially in relation to survey design.     

A synthesis of the biological quality elements for the implementation of the European Water Framework Directive in the Mediterranean ecoregion: The case of Saronikos Gulf

Benthic macroinvertebrates, macroalgae and phytoplankton constitute the biological quality elements proposed in the Water Framework Directive (WFD, 2000/60/EC) to be used for the classification of the ecological status of a water body. In the context of the preparation for the implementation of WFD, classification schemes for all three elements have been developed and tested for all the European ecoregions. In the present work, the classification schemes, with the corresponding metric tools and the interpretations of the class boundaries according to the normative definitions of WFD, are presented for each biological element in Saronikos Gulf, as case study in the Mediterranean ecoregion. The combination of the three biological elements into an integrated classification for coastal water bodies has been one of the major issues addressed in the context of WFD guidelines. Results are interpreted and validated, through an ecological viewpoint, on the basis of relevant environmental data. Moreover, this work presents a way to combine the EUNIS system to the typology of water bodies in the Mediterranean ecoregion.     

Evaluating the relative contributions of hydroperiod and soil fertility on growth of south Florida mangroves

The requirements of the European Water Framework Directive (WFD), aimed at an integrative assessment methodology for evaluating the ecological status of water bodies are frequently being achieved through multimetric techniques, i.e. by combining several indices, which address different stressors or different components of the biocoenosis. This document suggests a normative methodology for the development and application of Multimetric Indices as a tool with which to evaluate the ecological status of running waters. The methodology has been derived from and tested on a European scale within the framework of the AQEM and STAR research projects, and projects on the implementation of the WFD in Austria and Germany. We suggest a procedure for the development of Multimetric Indices, which is composed of the following steps: (1) selection of the most suitable form of a Multimetric Index; (2) metric selection, broken down into metric calculation, exclusion of numerically unsuitable metrics, definition of a stressor gradient, correlation of stressor gradients and metrics, selection of candidate metrics, selection of core metrics, distribution of metrics within the metric types, definition of upper and lower anchors and scaling; (3) generation of a Multimetric Index (general or stressor-specific approach); (4) setting class boundaries; (5) interpretation of results. Each step is described by examples.     

The development of an ecological quality assessment and classification system for Greek running waters based on benthic macroinvertebrates

A first approach towards an integrated, macroinvertebrate based, river type specific system for the assessment and classification of the ecological quality in Greek rivers affected by nutrient enrichment is presented. This system, which was developed within the framework of the AQEM project application in Greece, is regionally adapted and integrated, since it contains a multitude of biotic and abiotic factors affecting taxa presences and abundances. The methodology applied is very promising for the application of the Water Framework Directive (WFD) in Greece. According to the AQEM project, a seasonal `multi habitat sampling' procedure, at 45 sampling sites, scattered over three river types, was applied. In addition, a large set of hydrological, morphological, geological, biological, hydrochemical parameters and anthropogenic factors were considered. The system is based on a new, river type specific biotic metric, adapted for Greek conditions (BMG) and a new nutrient pollution metric. The developed metrics showed high correlations with each other. Hence, BMG seems to be suitable for the ecological quality assessment of Greek running waters affected by nutrients. Moreover, by applying BMG, a final classification of the rivers into five quality classes, according to the demands of the WFD, was achieved.     

Bringing European river quality into line: an exercise to intercalibrate macro-invertebrate classification methods

The Water Framework Directive (WFD) requires intercalibration to be performed to ensure that ecological status, as defined by the boundary values of national biological assessment systems, is consistent with the definitions outlined in the WFD and comparable between Member States (MS). This article describes an intercalibration of 17 national river macro-invertebrate assessment methods from the Central and Baltic regions of Europe. We explore the hypothesis that intercalibration should be successful if ratios of the observed biota to that expected in reference condition are used to compare assessments of different national assessment systems. National boundaries expressed as ecological quality ratios (EQRs) were converted to values of a common multi-metric for the purpose of comparison. Twelve MS for the High/Good boundary and nine MS for the Good/Moderate boundary (and four MS who subsequently harmonised their boundaries) were within ±0.05 EQR units of the intercalibration boundaries and were deemed to be of comparable ecological standard. The use of a reference-based approach was deemed to be successful given that all the critical pre-requisites for intercalibration were satisfied. The boundaries derived from this intercalibration represent the first common interpretation of the ecological status of rivers based on macro-invertebrate assessment methods across Europe.     

Quantifying uncertainties in biologically-based water quality assessment: A pan-European analysis of lake phytoplankton community metrics

Lake phytoplankton are adopted world-wide as a sensitive indicator of water quality. European environmental legislation, the EU Water Framework Directive (WFD), formalises this, requiring the use of phytoplankton to assess the ecological status of lakes and coastal waters. Here we provide a rigorous assessment of a number of proposed phytoplankton metrics for assessing the ecological quality of European lakes, specifically in response to nutrient enrichment, or eutrophication, the most widespread pressure affecting lakes. To be useful indicators, metrics must have a small measurement error relative to the eutrophication signal we want them to represent among lakes of different nutrient status. An understanding of variability in metric scores among different locations around a lake, or due to sampling and analytical variability can also identify how best this measurement error is minimised.To quantify metric variability, we analyse data from a multi-scale field campaign of 32 European lakes, resolving the extent to which seven phytoplankton metrics (including chlorophyll a, the most widely used metric of lake quality) vary among lakes, among sampling locations within a lake and through sample replication and processing. We also relate these metrics to environmental variables, including total phosphorus concentration as an indicator of eutrophication.For all seven metrics, 65–96% of the variance in metric scores was among lakes, much higher than variability occurring due to sampling/sample processing. Using multi-model inference, there was strong support for relationships between among-lake variation in three metrics and differences in total phosphorus concentrations. Three of the metrics were also related to mean lake depth. Variability among locations within a lake was minimal (<4%), with sub-samples and analysts accounting for much of the within-lake metric variance. This indicates that a single sampling location is representative and suggests that sub-sample replication and standardisation of analyst procedures should result in increased precision of ecological assessments based upon these metrics.For three phytoplankton metrics being used in the WFD: chlorophyll a concentration, the Phytoplankton Trophic Index (PTI) and cyanobacterial biovolume, >85% of the variance in metric scores was among-lakes and total phosphorus concentration was well supported as a predictor of this variation. Based upon this study, we can recommend that these three proposed metrics can be considered sufficiently robust for the ecological status assessment of European lakes in WFD monitoring schemes.     

Addressing a gap in the Water Framework Directive implementation: Rocky shores assessment based on benthic macroinvertebrates

A gap in the European Water Framework Directive (WFD) is addressed, aiming for the development of an ecological quality status assessment tool based solely on the Biological Quality Element benthic macroinvertebrates from intertidal rocky shores. The proposed Rocky shore Macroinvertebrates Assessment Tool (RMAT) was tested and validated along disturbance gradients (organic enrichment). During the whole process, the response of widely used metrics (e.g. Hurlbert index, Shannon-Wiener index, AZTI’s Marine Biotic Index; Bentix biotic index) and models (i.e., metrics combined) was compared to results provided by the Marine Macroalgae Assessment Tool to the same sampling sites.The RMAT is a multimetric index compliant with the WFD based on the benthic macroinvertebrates community, combining ‘abundance’ (Hurlbert index) and ‘taxonomic composition’ (Bentix index using density and biomass data) metrics. It performed well along anthropogenic disturbance gradients, showing ecological quality increasing from close to far away from the disturbance.The RMAT is a promising tool for rocky shore ecological assessment in the scope of the WFD or other monitoring activities worldwide.     

Errors and uncertainty in bioassessment methods – major results and conclusions from the STAR project and their application using STARBUGS

The STAR project’s extensive replicated sampling programmes have provided the first ever quantitative comparative studies of the susceptibility of a wide range of national macroinvertebrate sampling methods and taxonomic metrics to uncertainty resulting from the effects of field sampling variability and subsequent sub-sampling and laboratory (or bank-side) procedures and protocols. We summarise six STAR project papers examining various aspects of the potential sources of uncertainty in the observed fauna and observed metric values. The use of new simulation software STARBUGS (STAR Bioassessment Uncertainty Software System) to incorporate the effects of these potential errors into quantitative assessments of the uncertainty in assigning water bodies to WFD ecological status classes is discussed.     

Detection of ecological change using multiple organism groups: metrics and uncertainty

A number of biological approaches are commonly used to assess the ecological integrity of stream ecosystems. Recently, it is becoming increasingly common to use multiple organism groups in bioassessment. Advocates of the multiple organism approach argue that the use of different organism groups should strengthen inference-based models and ultimately result in lower assessment error, while opponents argue that organism groups often respond similarly to stress implying a high degree of redundancy. Using fish, macroinvertebrate, macrophyte and benthic diatom data, site-specific parameters (e.g., water chemistry and substratum) and catchment variables from European mountain (n = 77) and lowland (n = 85) streams we evaluated the discriminatory power and uncertainty associated with the use of a number of biological metrics commonly used in stream assessment. The primary environmental gradient for both streams types was land use and nutrient enrichment. Secondary and tertiary gradients were related to habitat quality or alterations in hydromorphology. Benthic diatom and macroinvertebrate metrics showed high discriminatory power (R² values often >0.50) and low error (<30%) with the primary (nutrient) gradient, while both fish and macrophyte metrics performed relatively poorly. Conversely, both fish and macrophyte metrics showed higher response (high coefficients of determination) than either benthic diatom or macroinvertebrate metrics to the second (e.g., alteration in habitat/hydromorphology) gradient. However, the discriminatory power and error associated with individual metrics varied markedly, indicating that caution should be exercised when selecting the ‘best’ organism group or metric to monitor stress.     

Development and validation of a fish-based index for the assessment of 'river health' in France

SUMMARY 1. In Europe, water policy is currently undergoing considerable change as emphasised by the recent European Water Framework Directive (WFD), which requires the restoration and maintenance of 'healthy' aquatic ecosystems by the assessment of their hydromorphological, chemical and biological characteristics. If the requirements of the WFD are to be met, effective biological tools are needed to measure the 'health' of rivers at scales large enough to be useful for management. These tools need to be ecologically based, efficient, rapid and applicable in different ecological regions. Among potential indicators, fish assemblages are of particular interest because of their ability to integrate environmental variability at different spatial scales. To meet the goals of the WFD, the French Water Agencies and the Ministry of the Environment initiated a research programme to develop a fish-based index that would be applicable nationwide.
2. A variety of metrics based on occurrence and abundance data and reflecting different aspects of the fish assemblage structure and function were selected from available literature and for their potential to indicate degradation.
3. Logistic and multiple linear regression procedures were applied, using an initial data set of 650 reference sites fairly evenly distributed across French rivers and defined by some easily measured regional and local characteristics, to elaborate the simplest possible response model that adequately explained the observed patterns of each metric for a given site.
4. Models obtained for each metric were validated using two independent data sets of 88 reference sites and 88 disturbed sites. These procedures allowed us to select the most effective metrics in discriminating between reference and disturbed sites.