Experimental Investigation of Observation Error in Anuran Call Surveys

ABSTRACT Occupancy models that account for imperfect detection are often used to monitor anuran and songbird species occurrence. However, presence—absence data arising from auditory detections may be more prone to observation error (e.g., false-positive detections) than are sampling approaches utilizing physical captures or sightings of individuals. We conducted realistic, replicated field experiments using a remote broadcasting system to simulate simple anuran call surveys and to investigate potential factors affecting observation error in these studies. Distance, time, ambient noise, and observer abilities were the most important factors explaining false-negative detections. Distance and observer ability were the best overall predictors of false-positive errors, but ambient noise and competing species also affected error rates for some species. False-positive errors made up 5% of all positive detections, with individual observers exhibiting false-positive rates between 0.5% and 14%. Previous research suggests false-positive errors of these magnitudes would induce substantial positive biases in standard estimators of species occurrence, and we recommend practices to mitigate for false positives when developing occupancy monitoring protocols that rely on auditory detections. These recommendations include additional observer training, limiting the number of target species, and establishing distance and ambient noise thresholds during surveys.     

Quantitative assessment of hit detection and confirmation in single and duplicate high-throughput screenings

The process of identifying active targets (hits) in high-throughput screening (HTS) usually involves 2 steps: first, removing or adjusting for systematic variation in the measurement process so that extreme values represent strong biological activity instead of systematic biases such as plate effect or edge effect and, second, choosing a meaningful cutoff on the calculated statistic to declare positive compounds. Both false-positive and false-negative errors are inevitable in this process. Common control or estimation of error rates is often based on an assumption of normal distribution of the noise. The error rates in hit detection, especially false-negative rates, are hard to verify because in most assays, only compounds selected in primary screening are followed up in confirmation experiments. In this article, the authors take advantage of a quantitative HTS experiment in which all compounds are tested 42 times over a wide range of 14 concentrations so true positives can be found through a dose-response curve. Using the activity status defined by dose curve, the authors analyzed the effect of various data-processing procedures on the sensitivity and specificity of hit detection, the control of error rate, and hit confirmation. A new summary score is proposed and demonstrated to perform well in hit detection and useful in confirmation rate estimation. In general, adjusting for positional effects is beneficial, but a robust test can prevent overadjustment. Error rates estimated based on normal assumption do not agree with actual error rates, for the tails of noise distribution deviate from normal distribution. However, false discovery rate based on empirically estimated null distribution is very close to observed false discovery proportion.     

Diagnosis of benign and malignant cervical specimens by multifiberoptic microspectrophometry and by flow cytometry

Two techniques for the automation of mass screening for cervical cancer were studied. Microspectrophotometry was tried first, using a novel multifiberoptic scanning system that measured the nuclear size and DNA content of cells in routine smears restained by the Feulgen technique. Specimen diagnoses were based on the percentages of cell types present, as determined by thresholds set for the two parameters. While this method gave good results in the automated detection of severe dysplasias and carcinomas, with only 3 of 72 cases misdiagnosed as negative (4.2%), it had a 22.9% false-positive rate (misdiagnosing 24 of 105 "benign" cases) and a 30.3% false-negative rate for adenocarcinomas (10 of 33 cases misclassified). The second approach involved flow cytometric measurements of specimens that were double stained for the assessment of both the DNA and RNA content, with the results analyzed by preset windows in a two-dimensional plane. This technique gave a 6.1% false-negative rate in 49 positive specimens and a 32.3% false-positive rate in 102 benign specimens, with an overall correct classification rate of 76.2%, including adenocarcinomas.     

Estimating environmental effects on population dynamics: consequences of observation error

Within the paradigm of population dynamics a central task is to identify environmental factors affecting population change and to estimate the strength of these effects. We here investigate the impact of observation errors in measurements of population densities on estimates of environmental effects. Adding observation errors may change the autocorrelation of a population time series with potential consequences for estimates of effects of autocorrelated environmental covariates. Using Monte Carlo simulations, we compare the performance of maximum likelihood estimates from three stochastic versions of the Gompertz model (log–linear first order autoregressive model), assuming 1) process error only, 2) observation error only, and 3) both process and observation error (the linear state–space model on log‐scale). We also simulated population dynamics using the Ricker model, and evaluated the corresponding maximum likelihood estimates for process error models. When there is observation error in the data and the considered environmental variable is strongly autocorrelated, its estimated effect is likely to be biased when using process error models. The environmental effect is overestimated when the sign of the autocorrelations of the intrinsic dynamics and the environment are the same and underestimated when the signs differ. With non‐autocorrelated environmental covariates, process error models produce fairly exact point estimates as well as reliable confidence intervals for environmental effects. In all scenarios, observation error models produce unbiased estimates with reasonable precision, but confidence intervals derived from the likelihood profiles are far too optimistic if there is process error present. The safest approach is to use state–space models in presence of observation error. These are factors worthwhile to consider when interpreting earlier empirical results on population time series, and in future studies, we recommend choosing carefully the modelling approach with respect to intrinsic population dynamics and covariate autocorrelation.     

Field test and experimental use of CYBEST model 2 for practical gynecologic mass screening

CYBEST is an automated cytologic screening system for uterine cancer utilizing a pattern-recognition image-analysis system. The prototype was developed in 1972 following fundamental studies of feature extraction, feature evaluation using ambiguity differential functions and segmentation of cell and nuclear images. Model 2 was developed in 1974 with an improved mechanism and function. The parameters employed are nuclear size, nuclear optical density, N/C ratio and nuclear shape. The data of field tests using 220 samples containing three cases of dysplasia, 110 cases of carcinoma and 107 nonmalignant cases were as follows: two false-negative cases (1.8%), 13 false positives (12.1%) and one reject (0.9%). This system was experimentally tested for practical mass population screening with 1,829 cases including 17 atypical cases (four epidermoid carcinomas). The data were as follows: no false-negative cases and 581 false-positive cases (32.1%). Of the latter, 311 cases (17.2%) were pathologic samples, such as severe cervicitis, senile colpitis, Trichomonas infestation, etc., and the remaining 270 cases (14.9%) were within physiologic limits, corresponding to true false-positive samples.     

Ambient noise increases missed detections in nestling birds

Ambient noise can mask acoustic cues, making their detection and discrimination difficult for receivers. This can result in two types of error: missed detections, when receivers fail to respond to the appropriate cues, and false alarms, when they respond to inappropriate cues. Nestling birds are error-prone, sometimes failing to beg when parents arrive with food (committing missed detections) or begging in response to stimuli other than a parent's arrival (committing false alarms). Here, we ask whether the frequency of these errors by nestling tree swallows (Tachycineta bicolor) increases in the presence of noise. We found that nestlings exposed to noise had more missed detections than their unexposed counterparts. We also found that false alarms remained low overall and did not differ significantly between noise and quiet treatments. Our results suggest that nestlings living in noisy environments may be less responsive to their parents than nestlings in quieter environments.     

Biodiversity extinction thresholds are modulated by matrix type

Biodiversity extinction thresholds are abrupt declines in biological diversity that occur with habitat loss, associated with a decline in habitat connectivity. Matrix quality should influence the location of thresholds along habitat loss gradients through its effects on connectivity; however these relationships have seldom been explored empirically. Using field data from 23 independent 1254 ha landscapes in the Brazilian Atlantic Forest, we evaluated how tropical avian biodiversity responds to native forest loss within habitat patches embedded either in homogeneous pasture matrix context (with a high proportion of cattle pastures), and heterogeneous coffee matrix context (with high abundance of sun coffee plantations). We considered taxonomic, functional, and phylogenetic diversity, and tested if matrix type and choice of diversity metric influenced the location of biodiversity thresholds along the forest cover gradient. We found that matrix type postponed the abrupt loss of taxonomic diversity, from a threshold of 35% of forest cover in homogeneous pasture matrix to 19% in heterogeneous coffee matrix. Phylogenetic diversity responded similarly, with thresholds at 30 and 24% in homogeneous‐pasture and heterogeneous‐coffee matrices, respectively, but no relationship with forest cover was detected when corrected for richness correlation. Despite the absence of a threshold for functional diversity in either matrix types, a strong decline below 20% of habitat amount was detected. Finally, below 20% native habitat loss, all diversity indices demonstrated abrupt declines, indicating that even higher‐quality matrices cannot postpone diversity loss below this critical threshold. These results highlight that taxonomic diversity is a more sensitive index of biodiversity loss in fragmented landscapes, which may be used as a benchmark to prevent subsequent functional and phylogenetic losses. Furthermore, increasing matrix quality appears an efficient conservation strategy to maintain higher biodiversity levels in fragmented landscapes over a larger range of habitat loss.     

Where have all the interactions gone? Estimating the coverage of two-hybrid protein interaction maps

Yeast two-hybrid screens are an important method for mapping pairwise physical interactions between proteins. The fraction of interactions detected in independent screens can be very small, and an outstanding challenge is to determine the reason for the low overlap. Low overlap can arise from either a high false-discovery rate (interaction sets have low overlap because each set is contaminated by a large number of stochastic false-positive interactions) or a high false-negative rate (interaction sets have low overlap because each misses many true interactions). We extend capture-recapture theory to provide the first unified model for false-positive and false-negative rates for two-hybrid screens. Analysis of yeast, worm, and fly data indicates that 25% to 45% of the reported interactions are likely false positives. Membrane proteins have higher false-discovery rates on average, and signal transduction proteins have lower rates. The overall false-negative rate ranges from 75% for worm to 90% for fly, which arises from a roughly 50% false-negative rate due to statistical undersampling and a 55% to 85% false-negative rate due to proteins that appear to be systematically lost from the assays. Finally, statistical model selection conclusively rejects the Erd?s-Rényi network model in favor of the power law model for yeast and the truncated power law for worm and fly degree distributions. Much as genome sequencing coverage estimates were essential for planning the human genome sequencing project, the coverage estimates developed here will be valuable for guiding future proteomic screens. All software and datasets are available in and , -, and -, and are also available from our Web site, http://www.baderzone.org.     

Automated reporting from gel-based proteomics experiments using the open source Proteios database application

The assembly of data from different parts of proteomics workflow is often a major bottleneck in proteomics. Furthermore, there is an increasing demand for the publication of details about protein identifications due to the problems with false-positive and false-negative identifications. In this report, we describe how the open-source Proteios software has been expanded to automate the assembly of the different parts of a gel-based proteomics workflow. In Proteios it is possible to generate protein identification reports that contain all the information currently required by proteomics journals. It is also possible for the user to specify maximum allowed false positive ratios, and reports are automatically generated with the corresponding score cut-offs calculated. When protein identification is conducted using multiple search engines, the score thresholds that correlate to the predetermined error rate are also explicitly calculated for proteins that appear on the result lists of more than one search engine.     

Postcatastrophe Population Dynamics and Density Dependence of an Endemic Island Duck

ABSTRACT Laysan ducks (Anas laysanensis) are restricted to approximately 9 km² in the Northwestern Hawaiian Islands, USA. To evaluate the importance of density dependence for Laysan ducks, we conducted a Bayesian analysis to estimate the parameters of a Gompertz model and the magnitude of process variation and observation error based on the fluctuations in Laysan duck abundance on Laysan Island from 1994 to 2007. This model described a stationary distribution for the population at carrying capacity that fluctuates around a long-term mean of 456 ducks and is between 316 to 636 ducks 95% of the time. This range of expected variability can be used to identify changes in population size that warn of catastrophic events. Density-dependent population dynamics may explain the recovery of Laysan duck from catastrophic declines and allow managers to identify population monitoring thresholds.     

Evaluation of a Semiautomated System for Direct Fluorescent Antibody Detection of Salmonellae

A semi-automatic system under development by Aerojet Medical and Biological Systems for the direct fluorescent antibody detection of salmonellae was evaluated with various food, feed, and environmental samples. All samples were simultaneously examined by Automated Bioassay System (ABS), manual direct fluorescent antibody procedures and cultural procedures. The ABS gave satisfactory results with the processed samples. It detected all of the culturally positive powdered egg and candy samples with no false negative results and gave only 6.6 and 5.3% false positive rates, respectively. With meatmeal samples the ABS failed to detect one culturally positive specimen that was also positive by manual fluorescent antibody and gave one (1.1%) false-positive result. A high rate of false-negative results was obtained by ABS on unprocessed samples of creek water, poultry, and sausage. Adding another enrichment step to the protocol reduced the false-negative rate considerably but severely increased the false-positive rate. The instruments worked reasonably well, but research is needed to improve enrichment procedures for samples to be processed by the system.     

Peptide-to-protein distribution versus a competition for significance to estimate error rate in blood protein identification

The simplest model—that authentic tandem mass spectrometry (MS/MS) spectra are no different from noise, random spectra, or false-positive results—may be directly examined by chi-square comparison of the peptide-to-protein distribution. The peptide-to-protein distribution of a set of 4151 redundant blood proteins identified by X!TANDEM indicated that there is a low probability that the authentic data were the same as noise, random spectra, or false-positive correlations (P < 0.0001). In contrast, a competition for significance failed to distinguish approximately 90% of authentic blood proteins from those of noise, random spectra, or false-positive results (P < 0.01) and apparently incurred a large type II error (false negative). The chi-square test of peptide-to-protein frequency distributions was found to be an efficient means to distinguish authentic data from false-positive results. Frequency-based statistics unambiguously demonstrated that proteins can be identified by liquid chromatography–electrospray ionization-MS/MS from human blood with acceptable confidence. Thus, the chi-square fit of the peptide-to-protein distribution could distinguish authentic data from random or false-positive data, but the score distribution method could not separate real results from false results.     

False alarms and information transmission in grouping animals

A key benefit of grouping in prey species is access to social information, including information about the presence of predators. Larger groups of prey animals respond both sooner and at greater distances from predators, increasing the likelihood that group members will successfully avoid capture. However, identifying predators in complex environments is a difficult task, and false alarms (alarm behaviours without genuine threat) appear surprisingly frequent across a range of taxa including insects, amphibians, fish, mammals, and birds. In some bird flocks, false alarms have been recorded to substantially outnumber true alarms. False alarms can be costly in terms of both the energetic costs of producing alarm behaviours as well as lost opportunity costs (e.g. abandoning a feeding patch which was in fact safe, losing sleep if an animal is resting/roosting, or losing mating opportunities). Models have shown that false alarms may be a substantial but underappreciated cost of group living, introducing an inherent risk to using social information and a vulnerability to the propagation of false information. This review will focus on false alarms, introducing a two-stage framework to categorise the different factors hypothesised to influence the propensity of animal groups to produce false alarms. A number of factors may affect false alarm rate, and this new framework splits these factors into two core processing stages: (i) individual perception and response; and (ii) group processing of predator information. In the first stage, individuals in the group monitor the environment for predator cues and respond. The factors highlighted in this stage influence the likelihood that an individual will misclassify stimuli and produce a false alarm (e.g. lower light levels can make predator identification more difficult and false alarms more common). In the second stage, alarm information from individuals is processed by the group. The factors highlighted in this stage influence the likelihood of alarm information being copied by group members and propagated through the group (e.g. some animals implement group processing mechanisms that regulate the spread of behavioural responses such as consensus decision making through the quorum response). This review follows the structure of this new framework, focussing on the causes of false alarms, factors that influence false alarm rate, the transmission of alarm information through animal groups, mechanisms to mitigate the spread of false alarms, and the consequences of false alarms.     

Filtering intrusion detection alarms

A Network Intrusion Detection System (NIDS) is an alarm system for networks. NIDS monitors all network actions and generates alarms when it detects suspicious or malicious attempts. A false positive alarm is generated when the NIDS misclassifies a normal action in the network as an attack. We present a data mining technique to assist network administrators to analyze and reduce false positive alarms that are produced by a NIDS. Our data mining technique is based on a Growing Hierarchical Self-Organizing Map (GHSOM) that adjusts its architecture during an unsupervised training process according to the characteristics of the input alarm data. GHSOM clusters these alarms in a way that supports network administrators in making decisions about true and false alarms. Our empirical results show that our technique is effective for real-world intrusion data.     

Improving mass spectrometry peak detection using multiple peak alignment results

Mass spectrometry data are often corrupted by noise. It is very difficult to simultaneously detect low-abundance peaks and reduce false-positive peak detection caused by noise. In this paper, we propose to improve peak detection using an additional constraint: the consistent appearance of similar true peaks across multiple spectra. We observe that false -positive peaks in general do not repeat themselves well across multiple spectra. When we align all the identified peaks (including false-positive ones) from multiple spectra together, those false-positive peaks are not as consistent as true peaks. Thus, we propose to use information from other spectra in order to reduce false-positive peaks. The new method improves the detection of peaks over the traditional single spectrum based peak detection methods. Consequently, the discovery of cancer biomarkers also benefits from this improvement. Source code and additional data are available at: http://www.ece.ust.hk/ approximately eeyu/mspeak.htm.     

A predictive model for identifying proteins by a single peptide match

MOTIVATION: Tandem mass-spectrometry of trypsin digests, followed by database searching, is one of the most popular approaches in high-throughput proteomics studies. Peptides are considered identified if they pass certain scoring thresholds. To avoid false positive protein identification, > or = 2 unique peptides identified within a single protein are generally recommended. Still, in a typical high-throughput experiment, hundreds of proteins are identified only by a single peptide. We introduce here a method for distinguishing between true and false identifications among single-hit proteins. The approach is based on randomized database searching and usage of logistic regression models with cross-validation. This approach is implemented to analyze three bacterial samples enabling recovery 68-98% of the correct single-hit proteins with an error rate of < 2%. This results in a 22-65% increase in number of identified proteins. Identifying true single-hit proteins will lead to discovering many crucial regulators, biomarkers and other low abundance proteins. SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.     

Predicting how populations decline to extinction

Global species extinction typically represents the endpoint in a long sequence of population declines and local extinctions. In comparative studies of extinction risk of contemporary mammalian species, there appear to be some universal traits that may predispose taxa to an elevated risk of extinction. In local population-level studies, there are limited insights into the process of population decline and extinction. Moreover, there is still little appreciation of how local processes scale up to global patterns. Advancing the understanding of factors which predispose populations to rapid declines will benefit proactive conservation and may allow us to target at-risk populations as well as at-risk species. Here, we take mammalian population trend data from the largest repository of population abundance trends, and combine it with the PanTHERIA database on mammal traits to answer the question: what factors can be used to predict decline in mammalian abundance? We find in general that environmental variables are better determinants of cross-species population-level decline than intrinsic biological traits. For effective conservation, we must not only describe which species are at risk and why, but also prescribe ways to counteract this.     

LESSONS FROM MONITORING TRENDS IN ABUNDANCE OF MARINE MAMMALS

We assessed scientists' ability to detect declines of marine mammal stocks based on recent levels of survey effort, when the actual decline is precipitous. We defined a precipitous decline as a 50% decrease in abundance in 15 yr, at which point a stock could be legally classified as "depleted" under the U.S. Marine Mammal Protection Act. We assessed stocks for three categories of cetaceans: large whales ( n = 23, most of which are listed as endangered), beaked whales ( n = 11, potentially vulnerable to anthropogenic noise), and small whales/dolphins/porpoises ( n = 69, bycatch in fisheries and important abundant predators), for two categories of pinnipeds with substantially different survey precision: counted on land ( n = 13) and surveyed on ice ( n = 5), and for a category containing polar bear and sea otter stocks ( n = 6). The percentage of precipitous declines that would not be detected as declines was 72% for large whales, 90% for beaked whales, and 78% for dolphins/porpoises, 5% for pinnipeds on land, 100% for pinnipeds on ice, and 55% for polar bears/sea otters (based on a one-tailed t -test, α= 0.05), given the frequency and precision of recent monitoring effort. We recommend alternatives to improve performance.     

An expert system to estimate the pesticide contamination of small streams using benthic macroinvertebrates as bioindicators II. The knowledge base of LIMPACT

The development and the evaluation of a biological indicator system for pesticide pollution in streams are presented. For small headwater streams with an agricultural catchment area, the expert system LIMPACT estimates the pesticide contamination according to the four classes: Not Detected (ND), Low (L), Moderate (M) and High (H) contamination without any specification of the chemical agents. The input parameters are the abundance data of benthic macroinvertebrate taxa within four time frames in a year (March/April, May/June, July/August, September/October) and nine basic water-quality and morphological parameters. The heuristic knowledge base was developed with the shell-kit D3 and contains 921 diagnostic rules with scores either to establish or to de-establish a diagnosis. The 418 rules had less than three symptoms, and only 47 rules had more than four symptoms in their rule condition. We differentiate between positive indicator (PI) taxa, which indicate contamination by high abundance values and positive abundance dynamics, and negative indicator (NI) taxa, a high abundance of which rules out contamination and indicates an uncontaminated site. We analysed 39 taxa and found 13 positive and 24 negative indicators. The database comprises 157 investigations per stream and year with rainfall event-controlled pesticide sampling and repeated benthic sampling as described in Part 1 [Ecol. Indicators, this issue]. For the evaluation of LIMPACT, we used the same cases. The correct class for the 157 investigations per stream and year is established by LIMPACT in 66.7–85.5% of the cases, with better results for uncontaminated sites. The overall alpha error probability (false positive) is 9.6% while the beta error probability (false negative) varied between 0 and 8% depending on the contamination class. If each stream is considered only once in the system (n=104), the correct diagnosis is established by LIMPACT in 51.9–88.6% of the cases. In most of the remaining cases no diagnosis is established instead of a wrong one.     

Impacts of climate change on national biodiversity population trends

Climate change has had well‐documented impacts on the distribution and phenology of species across many taxa, but impacts on species’ abundance, which relates closely to extinction risk and ecosystem function, have not been assessed across taxa. In the most comprehensive multi‐taxa comparison to date, we modelled variation in national population indices of 501 mammal, bird, aphid, butterfly and moth species as a function of annual variation in weather variables, which through time allowed us to identify a component of species’ population growth that can be associated with post‐1970s climate trends. We found evidence that these climate trends have significantly affected population trends of 15.8% of species, including eight with extreme (> 30% decline per decade) negative trends consistent with detrimental impacts of climate change. The modelled effect of climate change could explain 48% of the significant across‐species population decline in moths and 63% of the population increase in winged aphids. The other taxa did not have significant across‐species population trends or consistent climate change responses. Population declines in species of conservation concern were linked to both climatic and non‐climatic factors respectively accounting for 42 and 58% of the decline. Evident differential impacts of climate change between trophic levels may signal the potential for future ecosystem disruption. Climate change has therefore already driven large‐scale population changes of some species, had significant impacts on the overall abundance of some key invertebrate groups and may already have altered biological communities and ecosystems in Great Britain.