Observer error in vegetation surveys: a review

Aims Vegetation sampling employing observers is prone to both inter-observer and intra-observer error. Three types of errors are common: (i) overlooking error (i.e. not observing species actually present), (ii) misidentification error (i.e. not correctly identifying species) and (iii) estimation error (i.e. not accurately estimating abundance). I conducted a literature review of 59 articles that provided quantitative estimates or statistical inferences regarding observer error in vegetation studies.Important findings Almost all studies (92%) that tested for a statistically significant effect of observer error found at least one significant comparison. In surveys of species composition, mean pseudoturnover (the percentage of species overlooked by one observer but not another) was 10–30%. Species misidentification rates were on the order of 5–10%. The mean coefficient of variation (CV) among observers in surveys of vegetation cover was often several hundred % for species with low cover, although CVs of 25–50% were more representative of species with mean covers of>50%. A variety of metrics and indices (including commonly used diversity indices) and multivariate data analysis techniques (including ordinations and classifications) were found to be sensitive to observer error. Sources of error commonly include both characteristics of the vegetation (e.g. small size of populations, rarity, morphology, phenology) and attributes of the observers (e.g. mental fatigue, personal biases, differences in experience, physical stress). The use of multiple observers, additional training including active feedback approaches, and continual evaluation and calibration among observers are recommended as strategies to reduce observer error in vegetation surveys.     

Interobserver error in macroscopic methods of estimating age at death from the human skeleton

This research evaluates the interobserver error when the macroscopic methods recommended by American and European anthropologists to estimate age at death of a skeleton, were applied to a sample of the Terry Collection (Smithsonian Institution, Washington D.C.). Although no statistical differences among observers were found for any of these methods, small dissimilarities suggest that techniques using a narrower scale of categories produce greater agreement among researchers. The present study is within a wider research project designed to evaluate the accuracy of these methods, when applied to an identified (age known) sample of 963 skeleton from the Terry Collection.     

An examination of observer bias as a source of error in surveys of seagrass shoots

Abstract Mensurative experiments investigated the effects of different observers on estimates of the density of shoots of two species of seagrass: Posidonia australis Hook and Zostera capricorni Aschers. Balanced programmes of sampling were used to examine variation in counts of seagrass shoots attributable to different observers, sizes of quadrats, depths and locations within large beds of each species of seagrass. A separate experiment examined differences between novice observers and a more experienced observer, when an ‘optimal’ size of sampling unit was used. Estimated densities of Zostera shoots varied inconsistently among observers, quadrats, depths and locations. Differences between observers were not affected by the size of quadrat used to count Posidonia shoots, but varied between locations in the seagrass bed. Experience had only a minor impact on biases. Only two of 12 novices produced counts that were different from the experienced observer. These results emphasize the importance of considering both accuracy and precision in the design of field studies of seagrasses.     

Plant cover estimation based on the beta distribution in grassland vegetation

Cover is the most frequently used measure of abundance in vegetation surveys of grasslands, and various qualitative and semi-quantitative methods have been developed for visual estimation of this metric. Field survey is usually made with a point-grid plate. The frequency distributions of cover derived from point-grid counts follow a beta distribution. Combining point-grid counts from a field survey and the beta distribution for a statistical analysis, we developed an effort-saving cover-measurement method. Cover is measured with a transparent plastic plate on which, for example, 10 × 10 = 100 points are arranged in a lattice with 1-cm grid spacing (thus, one point count represents 1 cm² of cover). N quadrats are set out at randomly dispersed sites in a grassland, and, in each, the plastic plate is used for making counts. The number of grid points located above a given species is counted in every quadrat until the number of counted points reaches a given value c, which is determined in advance. If the number of counted points reaches c in a quadrat, the count is stopped and the quadrat is classified in the category “>c”. In quadrats where c is not attained, full point counts above the species bodies are made. Let g be the number of observed quadrats whose cover is ≤c. Using these g cover measurements and the number of quadrats (N − g) with cover >c, we can quantitatively estimate cover for each species and the spatial pattern index value based on the maximum likelihood method. In trial counts using this method, the time savings varied between 5% and 41%, depending on the shape of the cover frequency distribution. The mean cover value estimates agreed well with conventional measures without a stopping point (i.e., based on full counts of all points in each quadrat).     

Observer error in sampling a rare plant population

Background: Estimation of abundance in vegetation sampling involving observers is almost always characterised by observer error, although such error is rarely reported.

Aims: To quantify observer error in population estimation of the rare plant species Physaria filiformis in Missouri, USA.

Methods: The abundance of P. filiformis was estimated within 25-m² plots by six trained observers with varying experience levels over 10 years. Observers assigned plots to six predefined density classes. A total of 477 plots were estimated annually, and actual counts were conducted on ca. 10% of the plots to assess per cent agreement of estimates with counts.

Results: Over a third of the estimates of plant abundance evaluated for accuracy (36.4%) deviated from exhaustive counts. The majority of the misestimates were underestimates by one density class (29.4%). The number and type of misestimates varied systematically with density class.

Conclusions: Observer error could be explained to some degree by variation in population density, but not by experience. It appears that inherent differences exist among observers that cannot be entirely compensated for by experience or training. Observer error in this system represents a systematic bias, and can be compensated for by use of correction factors, which would ideally be both density class-dependent and observer-specific.     

Assessment of sources of uncertainty in macrophyte surveys and the consequences for river classification

The application of macrophytes in freshwater monitoring is still relatively limited and studies on their intercalibration and sources of variation are required. Therefore, the aim of the study was to compare selected indices and metrics based on macrophytes and to quantify their variability. During the STAR project, several aspects influencing uncertainty in estimation of the ecological quality of river were assessed. Results showed that several metrics based on the indicative value of plant species can be used in evaluation of the ecological status of rivers. Among estimated sources of variance in metric values the inter-surveyor differences had the lowest effect and slightly stronger were the influences of temporal variation (years and seasons) and shading. The impact of habitat modification was the most important factor. Analysis showed that some of macrophyte-based metrics (notably MTR and IBMR) are of sufficient precision in terms of sampling uncertainty, that they could be useful for estimating the ecological status of rivers in accordance with the aims of the Water Framework Directive.     

Distance sampling surveys: using components of detection and total error to select among approaches

Wildlife population estimators often require formal adjustment for imperfect detection of individuals during surveys. Conventional distance sampling (CDS) and its extensions (mark-recapture distance sampling [MRDS], temporary emigration distance sampling [TEDS]) are popular approaches for producing unbiased estimators of wildlife abundance. However, despite extensive discussion and development of distance sampling theory in the literature, deciding which of these alternatives is most appropriate for a particular scenario can be confusing. Some of this confusion may stem from an incomplete understanding of how each approach addresses the components of the detection process. Here we describe the proper application of CDS, MRDS, and TEDS approaches and use applied examples to help clarify their differing assumptions with respect to the components of the detection process. To further aid the practitioner, we summarize the differences in a decision tree that can be used to identify cases where a more complex alternative (e.g., MRDS or TEDS) may be appropriate for a given survey application. Although the more complex approaches can account for additional sources of bias, in practical applications one also must consider estimator precision. Therefore, we also review the concept of total estimator error in the context of comparing competing methods for a given application to aid in the selection of the most appropriate distance sampling approach. Finally, we detail how the use of more advanced techniques (i.e., informed priors, open-population distance sampling models, and integrated modeling approaches) can further reduce total estimator error by leveraging information from existing and ongoing data collection. By synthesizing the existing literature on CDS, MRDS, TEDS and their extensions, in conjunction with the concepts of total estimator error and the components of the detection process, we provide a comprehensive guide that can be used by the practitioner to more efficiently, effectively, and appropriately apply distance sampling in a variety of settings.     

Observer variation in field assessments of vegetation condition: Implications for biodiversity conservation

Summary   Uncertainty in assessments of vegetation condition that are used to inform land management and planning decisions for biodiversity conservation in Australia may lead to unexpected outcomes, including loss of biodiversity. This study investigates observer error in field estimates of vegetation attributes, one component of uncertainty in assessments of vegetation condition. Ten observers conducted vegetation condition assessments using two assessment protocols (BioMetric and Habitat Hectares) on 20 sites in a grassy woodland community. Observers' estimates varied substantially across multiple scoring categories for all vegetation attributes on almost all sites. Across all sites, the average coefficient of variation in total vegetation condition scores was 15–18% for both protocols, with a maximum of 60%. The primary cause of variation in total vegetation condition scores was random error in raw estimates of vegetation attributes, although sensitivity of some highly weighted attributes to error exacerbated variation in some cases. Observers generally agreed on the total scores and ranks of highly degraded (pasture) sites, but were less consistent on other sites. Rank correlations between pairs of observers were stronger for Habitat Hectares, suggesting BioMetric may be slightly more sensitive to observer error. It is recommended that: (i) research is undertaken into methods for reducing observer error; (ii) review is made of the sensitivity of index scoring structures to observer error; (iii) field observers estimate uncertainty around point estimates of vegetation condition; and, (iv) decision-makers explicitly incorporate uncertainty into the decision-making processes and aim for outcomes that are robust to this uncertainty.     

Observer error,dental wear,and the inference of new world sundadonty

Dental morphology provides important information on human evolution and interpopulation relationships. Dental wear is one of the major limitations of morphological data analysis. Wear figures heavily in existing debates about patterns of New World dental variation with some scholars finding evidence for a more generalized dentition in early New World populations (Powell: Doctoral Dissertation, Texas A&M University, TX (1995)) and others questioning these findings based on the probable effects of dental wear on trait scores (Turner, The First Americans: the Pleistocene Colonization of the New World. San Francisco: California Academy of Sciences (2002) 123–158; Turner: Am J Phys Anthropol 130 (2006) 455–461; Turner and Scott, Handbook of paleoanthropology, Vol. III: Phylogeny of Hominids. New York: Springer (2007) 1901–1941). Here we evaluate these competing claims using data from the Early Archaic Windover sample. Results confirm the dental distinctiveness of Windover with respect to other Old World Asian (i.e., sinodont/sundadont) populations. However, comparison of our results to those of Powell (1995) also highlights significant interobserver error. Statistical analysis of matched wear and morphology scores suggests trait downgrading for some traits. Patterns of missing data present a more challenging (and potentially serious) problem. Use of Little's MCAR test for missing data mechanisms indicates a complex process of data collection in which incidental and opportunistic recording of both highly worn and unerupted teeth introduce a “missing not at random” mechanism into our dataset that biases dental trait frequencies. We conclude that patterns of missingness and formal research designs for “planned missingness” are needed to help mitigate this bias. Am J Phys Anthropol 156:349–362, 2015. © 2014 Wiley Periodicals, Inc.     

Detection probability in aerial surveys of feral horses

Observation bias pervades data collected during aerial surveys of large animals, and although some sources can be mitigated with informed planning, others must be addressed using valid sampling techniques that carefully model detection probability. Nonetheless, aerial surveys are frequently employed to count large mammals without applying such methods to account for heterogeneity in visibility of animal groups on the landscape. This often leaves managers and interest groups at odds over decisions that are not adequately informed. I analyzed detection of feral horse (Equus caballus) groups by dual independent observers from 24 fixed-wing and 16 helicopter flights using mixed-effect logistic regression models to investigate potential sources of observation bias. I accounted for observer skill, population location, and aircraft type in the model structure and analyzed the effects of group size, sun effect (position related to observer), vegetation type, topography, cloud cover, percent snow cover, and observer fatigue on detection of horse groups. The most important model-averaged effects for both fixed-wing and helicopter surveys included group size (fixed-wing: odds ratio = 0.891, 95% CI = 0.850–0.935; helicopter: odds ratio = 0.640, 95% CI = 0.587–0.698) and sun effect (fixed-wing: odds ratio = 0.632, 95% CI = 0.350–1.141; helicopter: odds ratio = 0.194, 95% CI = 0.080–0.470). Observer fatigue was also an important effect in the best model for helicopter surveys, with detection probability declining after 3 hr of survey time (odds ratio = 0.278, 95% CI = 0.144–0.537). Biases arising from sun effect and observer fatigue can be mitigated by pre-flight survey design. Other sources of bias, such as those arising from group size, topography, and vegetation can only be addressed by employing valid sampling techniques such as double sampling, mark–resight (batch-marked animals), mark–recapture (uniquely marked and identifiable animals), sightability bias correction models, and line transect distance sampling; however, some of these techniques may still only partially correct for negative observation biases. © 2011 The Wildlife Society.     

Relationships between plant diversity,vegetation cover,and site conditions: implications for grassland conservation in the Greater Caucasus

Overgrazing, land use abandonment and increasing recreational activities have altered the vegetation of high-montane and subalpine grassland of the Caucasus. The failure of previous restoration efforts with unsuitable and exotic plant species indicates the need for information on the present vegetation and in which way it might change. Within the Greater Caucasus, we have described and quantified the mountain grassland which develops under characteristic overgrazed and eroded site conditions. Further, we have proposed potential native plant species for revegetation to restore and conserve valuable mountain grassland habitats. We used non-metric dimensional scaling ordination and cluster comparison of functional plant groups to describe a gradient of grassland vegetation cover. For our study region, we identified four major vegetation types with increasing occurrence of ruderal pasture weeds and tall herb vegetation on abandoned hay meadows within the subalpine zone. Within high-montane grassland a decline of plant diversity can be observed on sites of reduced vegetation cover. Due to a low potential of the grassland ecosystem to balance further vegetation cover damage, the long-term loss of diverse habitats can be expected. We conclude with management recommendations to prevent erosion and habitat loss of precious mountain grasslands.     

Assessing the assessors: Quantifying observer variation in vegetation and habitat assessment

Vegetation assessment protocols used in research, monitoring and statutory planning often rely on the consistent application of methods by multiple assessors. Consequently, a study was undertaken to evaluate the application of a vegetation condition assessment protocol by a range of assessors. The aim of the study was to quantify variation among assessors in their measurement of field‐based vegetation attributes using Queensland’s BioCondition protocol, and to determine the effect of variability on the final condition score. The study consisted of 77 assessors, ranging from nil to 25 years experience in vegetation assessment, who each undertook an assessment at one site following training. Six of the 20 attributes used to derive the final condition score were not assessed consistently, this being because of a small number of assessors who had misinterpreted the protocol and had inappropriately assessed some attributes. Despite inclusion of outliers, 82% of assessors were within 10 points of the median condition score for the site. Based on the results, the definition and measurement of problematic attributes have been either clarified or removed from the revised assessment procedure. The study demonstrated that with training prior to use, assessors were able to consistently assess condition at the study site.     

Micro-patterns in grassland vegetation created and sustained by sheep-grazing

An initially uniform Holcus lanatus-dominated sward came partly under hay-making and partly under sheep-grazing. Preferential grazing by sheep resulted in grazing at different intensities giving rise to a macro-pattern of various plant communities. Besides this macro-pattern a micro-pattern developed in the grazed area, which was absent under hay-making. In the micro-pattern short, heavily grazed areas alternated with taller, lightly grazed patches, both having the same species composition. The heavily grazed area was characterized by equal amounts of monocots and dicots. The lightly grazed patches were dominated by Agrostis tenuis, and had a large amount of litter which probably causes the absence of mosses. The protein percentage of green material is higher in the heavily grazed areas than in the lightly grazed patches.Sequential charting indicated that the micro-pattern was more or less stable. An interaction between the vegetation micro-pattern and grazing patterns is suggested. Heavy grazing results in forage with a high protein content and hence attracts animals. Light grazing results in forage with a relatively low protein content, animals avoid the area and litter accumulates.Nomenclature follows Heukels & van Ooststroom (1977) Flora van Nederland.Mrs J. O'Brien corrected the English text     

Interobserver reliability of skinfold measurements in infants and young children

Interobserver reliabilities were determined for the triceps, biceps, subscapular, suprailiac, and abdominal skinfolds in 77 children, 9-24 months of age. Technical errors of measurement (replicate variances) and coefficients of variation were compared to data on 12-17-year-olds from the U.S. Health Examination Survey (HES) to 2.5-7-year-old Guatamalan children. Of the five skinfolds, the between-observer variation was not significantly different from zero in four; in the case of the biceps fold, F-ratio was significant at p less than .01. Errors of measurement are less for these data than for the HES or Guatemalan studies. This difference is attributed to the larger means of the older children and youth, as well as to the greater error of measurement shown to exist for larger skinfolds.     

Revising how the computer program CERVUS accommodates genotyping error increases success in paternity assignment

Genotypes are frequently used to identify parentage. Such analysis is notoriously vulnerable to genotyping error, and there is ongoing debate regarding how to solve this problem. Many scientists have used the computer program cervus to estimate parentage, and have taken advantage of its option to allow for genotyping error. In this study, we show that the likelihood equations used by versions 1.0 and 2.0 of cervus to accommodate genotyping error miscalculate the probability of observing an erroneous genotype. Computer simulation and reanalysis of paternity in Rum red deer show that correcting this error increases success in paternity assignment, and that there is a clear benefit to accommodating genotyping errors when errors are present. A new version of cervus (3.0) implementing the corrected likelihood equations is available at http://www.fieldgenetics.com .     

Intra- and interobserver error in craniometry: A cautionary tale

This study investigates intra- and interobserver measurement error in craniometry. Data consists of 72 craniometric measurements taken on a series of 28 Sadlermuit Eskimo crania. Utermohle measured the series twice; Zegura measured it once. Statistical procedures used to demonstrate measurement imprecision include the mean difference, the method error statistic, two-way anova without replication, the t-test for paired comparisons, Fisher's distribution-free sign test, and the t-test for independent samples. The results indicate less intraobserver repeatability than expected as well as an alarming lack of interobserver reproducibility for many of these craniometric measurements. We hope these results will serve as a caution against the widespread belief that craniometric measurements are always produced with a high degree of precision by experienced craniometrists. In addition, these results suggest that investigators employing craniometric measurements to study population affinities, functional morphology, forensics, fossil primates, and human microevolution might profit from conducting a measurement error analysis as an important baseline for the interpretation of the biological significance of their results.     

Similarity between seed bank and vegetation in Mediterranean grassland: a predictive model

Abstract. The similarity in species composition between seed bank and vegetation was analysed in Mediterranean grasslands in relation to altitude, topography and grazing. Soil samples were collected in permanent plots in autumn at the end of the summer drought period and in spring, before the new seed fall and after the natural winter seed stratification. The seed bank composition was determined by greenhouse germination over a nine-month period. Presence/absence of species in the standing vegetation throughout the complete annual cycle, and the percentage area of bare ground in October, were recorded in the same plots. The species composition of the standing vegetation is clearly determined by altitude, topography and grazing, while the floristic composition of the seed banks is only related to altitude and topography in the case of autumn seed bank and with any of the three factors in the spring seed bank. Relative abundances of grasses, legumes and forbs also show different patterns in vegetation and seed bank data. Sørensen similarity between the autumn seed bank and the vegetation declines as altitude rises, but there are no significant differences for topography and grazing. This similarity decreases in the case of the spring seed bank and does not show any significant relationship with any of the factors. The perennial/ annual ratio and the proportion of bare soil in October are proposed as explanatory variables in a predictive model of similarity between the seed composition of the seed bank and vegetation.