Avoiding erroneously high levels of detection in combinations of semi-independent tests

A randomization procedure is proposed which allows statistical tests to be combined into a single test to maintain specified and acceptable levels of false detection. This method was applied to the problem of detecting density dependence in 135 unpublished time-series (of 10 generations) from insect populations, and to simulated density-dependent and density-independent data, so that the correctness of observed levels of detection from the published data could be verified. To allow the application of the randomization procedure to Bulmer's (1975) tests and Varley and Gradwell's (1960) test, these were recast as randomization tests. The randomization procedure was tested with 39 combinations of tests for density dependence (and limitation/attraction); it generally producedcombined tests with levels of detection that were intermediate between detection levels of the constituent tests (and hence was limite by these). The specified rate of false detection (5%) was never exceeded (by more than 1%) when combined tests were applied to time-series from a random-walk model. Two different combinations of tests produced levels of detection from the published time-series which were slightly greater than their constituent tests when they were combined into single tests. These were the randomized form of Bulmer's (1975) first test with the tests of Pollard et al. (1987) and Reddingius and den Boer (1989) with the randomized form of Bulmer's second test. The combination of Bulmer's first and Pollard et al.'s test produced a greater level of detection (21.5%) than any other single test or combination of tests. These results were confirmed by the analysis of modelled density dependent data. Although the increase in power of combinations of tests over single tests is small with the data we used, the combined tests (listed above) had rates of detection that were less influenced by the form of data (of two forms of density-dependent data) than were their constituent tests. Hence, it appears that the combined tests are of greater generality than single test statistics. The method presented here for combining several statistical tests into a single randomization test is applicable in many other areas of ecology where we wish to apply several tests and take the most probable result of these; and if the tests being conducted are, or can be expressed as, randomization tests.     

Tests for density dependence revisited

We have examined a number of statistical issues associated with methods for evaluating different tests of density dependence. The lack of definitive standards and benchmarks for conducting simulation studies makes it difficult to assess the performance of various tests. The biological researcher has a bewildering choice of statistical tests for testing density dependence and the list is growing. The most recent additions have been based on computationally intensive methods such as permutation tests and boot-strapping. We believe the computational effort and time involved will preclude their widespread adoption until: (1) these methods have been fully explored under a wide range of conditions and shown to be demonstrably superior than other, simpler methods, and (2) general purpose software is made available for performing the calculations. We have advocated the use of Bulmer's (first) test as a de facto standard for comparative studies on the grounds of its simplicity, applicability, and satisfactory performance under a variety of conditions. We show that, in terms of power, Bulmer's test is robust to certain departures from normality although, as noted by other authors, it is affected by temporal trends in the data. We are not convinced that the reported differences in power between Bulmer's test and the randomisation test of Pollard et al. (1987) justifies the adoption of the latter. Nor do we believe a compelling case has been established for the parametric bootstrap likelihood ratio test of Dennis and Taper (1994). Bulmer's test is essentially a test of the serial correlation in the (log) abundance data and is affected by the presence of autocorrelated errors. In such cases the test cannot distinguish between the autoregressive effect in the errors and a true density dependent effect in the time series data. We suspect other tests may be similarly affected, although this is an area for further research. We have also noted that in the presence of autocorrelation, the type I error rates can be substantially different from the assumed level of significance, implying that in such cases the test is based on a faulty significance region. We have indicated both qualitatively and quantitatively how autoregressive error terms can affect the power of Bulmer's test, although we suggest that more work is required in this area. These apparent inadequacies of Bulmer's test should not be interpreted as a failure of the statistical procedure since the test was not intended to be used with autocorrelated error terms.     

Testing for density-dependent effects in sequential censuses

Summary In response to Gaston and Lawton (1987), we evaluated the ability of four statistical procedures to detect density dependence. We used data from the same 16 populations as Gaston and Lawton (1987). In each population, density dependence had been previously established with techniques that use more extensive data. The major axis test (Slade 1977) was rarely (3 populations of 16) capable of detecting density dependence. The autocorrelation test (Bulmer 1975) detected density dependence in 5 of 16 species (14 of 59 tests overall). The randomization procedure (Pollard et al. 1987) detected density dependence in 7 of the 16 data sets (10 of 59 tests overall). The simulation procedure (Vickery and Nudds 1984) detected density dependence in 5 of the 16 data sets (11 of 59 tests overall). We suggest that not all annual census data taken from populations subject to density-dependent effects will actually show evidence of such effects. We conclude that Pollard et al. 's (1987) randomization procedure is the best test for detecting density dependence in sequential census data but it is not as powerful as more elaborate techniques (k-factor analysis, experimentation, etc.), nor is it meant to replace more extensive analyses.     

A test of statistical techniques for detecting density dependence in sequential censuses of animal populations

Summary Principal and reduced major axes, and Bulmer's (1975) tests have been suggested as methods for detecting the presence of density dependence in a series of population censuses that are unsuitable for analysis by alternative means e.g. by k-factor analysis. These alternative methods are tested using census data, some of which are previously unpublished, from natural populations known from independent evidence to be subject to density dependent processes. All the methods fail to detect density dependence reliably, irrespective of sample size and the dynamics of the population. We conclude that none of the methods tested is sufficiently reliable to be useful as a test of density dependence in sequential censues of animal populations.     

On the stabilization of animal numbers. Problems of testing

Summary When testing for regulation of population numbers, rather than using Bulmer's second test in cases where population numbers are estimated instead of measured, we prefer to correct Bulmer's first test for estimation error. A correction method is expounded, discussed, and applied to two series of census data: the pine looper of Klomp and the garden chafer of Milne. In neither case the tentative conclusion from using the uncorrected test was changed after correction. Therefore, in practice Bulmer's first test without correction can be used well as a first orientation. Twelve long series (more than 10 years) of census data of both univoltine and semelparous (a necessary condition) insects were tested for significant density dependence in the fluctuations of numbers with the randomization test of Pollard et al. None of the series, all we could find to meet the necessary condition as well as being longer than 10 years, showed significant density dependence at the 0.05 level, though the pine looper of Klomp did so at the 0.1 level. Next, the same series were tested for regulation in the sense of keeping density within limits with both the first test of Bulmer and the permutation test of Reddingius and Den Boer. Onky Klomp's pine looper population at Hoge Veluwe scored significantly. In a following paper this population will be considered more closely, in order to enable understanding of this test result.Communication No. 362 of the Biological Station, Wijster     

Tests for density dependence

Several statistical tests for density dependence have been proposed in the literature, and so in any practical case the question poses itself which one of these tests to choose. This paper offers a few remarks additional to those made by Fox and Ridsill-Smith (1995) and others. Parametric statistical tested are based on a fully specified mathematical model. Examples of such tests are Bulmer's (1975) first test, and the test of Dennis and Taper (1994). Distribution-free tests are based on far less stringent assumptions. An example of such a test is the one proposed by Pollard et al. (1987). The choice between parametric tests can best be made by considering which one of the underlying mathematical models ist most plausible. If all models are almost equally plausible, considerations of computational requirement and ease of application may be important. Strong doubts concerning the plausibility of mathematical models may lead one to prefer a distribution-free test. An important feature of any test is its power, i.e. the probability of its rejecting the null hypothesis when this hypothesis is not true. Other things being equal, tests are preferable when they have superior powers. But power of a test depends on the true state of nature, and the only way to study power quantitatively is by assuming some mathematical model as approximately representing this true state. As any mathematical model can at best only be an approximation to the situation in nature, a mathematical model and the statistical tests based on it should be robust against small deviations from model assumptions. Solow (1990) showed that Bulmer's test is not robust with respect to the assumption that the residuals in the underlying autoregression model be stochastically independent. Contrary to what was suggested by Fox and Ridsill-Smith (1995), who misinterpreted some statements in Reddingius (1990), the present author thinks this is a serious shortcoming of this test since an ecologist cannot assume a priori that important density-independent ecological factors are not somehow serially correlated. Moreover, he is rather sceptical about the usefulness of statistical tests for density dependence. They have contributed more to misunderstandings than to a significant increase in ecological insight. In any case, statistical tests are designed to test hypotheses that are stated before data are collected, and the question which test to use also has to be answered before the data have been collected. Designing and using statistical tests a posteriori to detect things in data mainly leads to confusion and controversy.     

Comment arising from a paper by Wolda and Dennis: using and interpreting the results of tests for density dependence

We argue that tests for density dependence are useful in analyses of population dynamics and suggest guide lines for their use and interpretation of results which avoid many of the problems discussed by Wolda and Dennis (1993). Processes other than density dependence per se can cause statistical tests to indicate the presence of density dependence (Wolda and Dennis 1993 and unpublished simulations). Tests for density dependence cannot reveal the mechanism of regulation, but they do indicate the nature of long-term population dynamics. Tests for density dependence give misleading results if sampling is not at generation intervals; however, this problem is avoided if we only use tests on data collected in each generation (Holyoak 1993a). Similarly, species should be semelparous. Non-delayed density dependence should not be considered without looking for delayed density dependence, since the presence of delayed density dependence can lead to over-detection of non-delayed density dependence (Woiwod and Hanski 1992; Holyoak 1993b). The partial autocorrelation function and knowledge of life-history are more useful than tests for density dependence for indicating whether any density dependence is delayed or not (Royama 1992; Holyoak 1993b). Estimation error with a constant upper size limit causes tests for density dependence to overestimate the frequency of delayed density dependence; however we do not know whether estimation error is bounded in real populations. Work in progress suggests that 20–40 generations (depending on the nature of population dynamics) gives a moderate level of accuracy with tests for density dependence, and >40 generations are necessary for tests to be accurate in their assessment of the strength of density dependence. We conclude that tests are useful indicators of whether density dependence, or other feedback mechanisms are likely to be acting.     

Models for testing

Summary This is a comment on a note by Solow (1990). It is shown that Solow's simulation results indicate that Bulmer's test for density dependence is non-robust to a particular kind of second-order Markovity that might well be overlooked by an ecologist. It is suggested that Solow's claim that Bulmer's test is insensitive is not wholly justified. Some scepticism concerning the applicability of statistical testing theory to animal population data is expressed.Communication no. 410 of the Biological Station, Wijster     

Competition between herbivourous fishes and urchins on Caribbean reefs

Summary When the common sea urchin Diadema antillarum was removed from a 50 m strip of reef in St. Thomas, US Virgin Islands, cover of upright algae and the grazing rates and densities of herbivorous parrotfish and surgeonfish increased significantly within 11–16 weeks when compared to immediately adjacent control areas. Sixteen months after removal, Diadema had recovered to 70% of original density, abundance of upright algae no longer differed between removal and control areas, and the abundance and grazing activity of herbivorous fish in the removal was approaching equivalence with control areas. On a patch reef in St. Croix that had been cleared of Diadema 10–11 years earlier (Ogden et al. 1973b), urchins had recovered to only 50–60% of original density. This reef still showed significantly higher rates of grazing by fish and a significantly greater density of parrotfish and surgeonfish than a nearby control reef where Diadema densities had not been altered. These results indicate that high Diadema densities (7–12/m² for this study) may suppress the densities of herbivorous fish on Caribbean reefs.     

Density dependence in cereal aphid populations

Long sequences of data on the incidence of cereal aphids from five European countries were analysed for evidence of density dependent processes occurring between years. Using a randomisation test (Pollard, Lakhani & Rotheray, 1987), density dependence was revealed in all (16) population censuses of Metopolo-phium dirhodum , 60% of Rhopalosiphum padi censuses (10 of 17) but only 25% of Sitobion avenae population censuses (4 of 16). Correcting for density independent effects of weather revealed the existence of significant direct density dependence in some populations censuses where it was previously undetected. The implications of density dependence in cereal aphid populations are considered.     

Testing for density dependence allowing for weather effects

A test for density dependence in time-series data allowing for weather effects is presented. The test is based on a discrete time autoregressive model for changes in population density with a covariate for the effects of weather. The distribution of the test statistic on the null hypothesis of density independence is obtained by parametric bootstrapping. A computer simulation exercise is used to demonstrate the gain in statistical power by allowing for weather effects. Application of the method to time-series data on three species of butterflies and two species of songbirds showed stronger evidence of density dependence than two standard tests. Received: 4 October 1996 / Accepted: 4 August 1997     

Cycles in insect populations: delayed density dependence or exogenous driving variables?

Summary Delayed density dependence, and the cycles in insect populations that it can generate, are often investigated using time-series analysis. Recently, several authors have raised concerns about the validity of using time-series analysis to detect density dependence. One particular concern is the suggestion that exogenous driving variables, such as cyclic weather patterns, can lead to the spurious detection of density dependence in natural populations.
Using non-biological data (the electricity bills of one of the authors), we show how easy it is to be misled by the results of time-series analysis. We then present 16 years' data on the gall-forming sawfly, Euura lasiolepis (Hymenoptera: Tenthredinidae), and show that cycles in weather, specifically winter precipitation, lead to the spurious detection of density dependence in time-series analysis. We conclude that time-series analysis cannot stand alone as a method for inferring the action of density dependence, and urge further investigation of the effects of apparent cycles in abiotic forces on insect populations.     

Latitudinal decrease in folivory within Nothofagus pumilio forests: dual effect of climate on insect density and leaf traits?

Aim The strength of consumer–plant interactions may decrease with latitude. Our objectives were to assess the spatial variation in folivory on Nothofagus pumilio and understand the influence of climate on folivory patterns as mediated by changes in folivore density and leaf traits. Location Nothofagus pumilio forests, between 38 and 55°S (Argentina). Methods We studied the correlation of leaf damage with latitude on data from 47 sampling sites, and evaluated spatial patterns of autocorrelation on latitudinally detrended data with a principal coordinates of neighbour matrices method. Path analysis was used to test the association of temperature and precipitation with leaf damage, mediated by folivore density and leaf traits. We evaluated the adequacy of this ecological model by examining the spatial pattern of autocorrelation in the residuals, and combined spatial and environmental predictors of leaf damage into partial regression. Results Leaf damage decreased with latitude, which was the only significant spatial predictor. The latitudinal decrease in temperature and precipitation was correlated with a decrease in the density of folivores and leaf size, and diminished leaf damage. Our ecological model adequately explained the spatial autocorrelation in the data: 44% of the variation in leaf damage was explained by the latitudinally structured component of the environment, whereas local environmental effects accounted for another 22%. Main conclusions We conclude that N. pumilio forests show consistent latitudinal patterns of variation in folivory, folivore density and leaf traits. Our study suggests that the latitudinal variation in folivory rates is partly driven by the influence of climate on both plants and herbivores. This warns us about the potential susceptibility of folivory rates to climate warming. We emphasize the value of large‐scale analyses as complementary to local experimental approaches to understanding the regulation of herbivory.     

Identifying Predator–Prey Processes from Time-Series

The functional response is a key element in predator–prey models as well as in food chains and food webs. Classical models consider it as a function of prey abundance only. However, many mechanisms can lead to predator dependence, and there is increasing evidence for the importance of this dependence. Identification of the mathematical form of the functional response from real data is therefore a challenging task. In this paper we apply model-fitting to test if typical ecological predator–prey time series data, which contain both observation error and process error, can give some information about the form of the functional response. Working with artificial data (for which the functional response is known) we will show that with moderate noise levels, identification of the model that generated the data is possible. However, the noise levels prevailing in real ecological time-series can give rise to wrong identifications. We will also discuss the quality of parameter estimation by fitting differential equations to such time-series.     

Effects of acorn abundance on density dependence in a Japanese wood mouse (Apodemus speciosus) population

We analysed the effects of Quercus crispula acorn abundance on the density dependence of the large Japanese wood mouse Apodemus speciosus using time series data (1992–2007). The data were obtained in a forest in northern Hokkaido, Japan, by live-trapping rodents and directly counting acorns on the ground. Acorn abundance in one year clearly influenced the abundance of wood mice in the following year in all models examined based on the Gompertz and Ricker model; in addition, the abundance of wood mice had effects on the population. Acorn abundance influenced the strength of density dependence (intraspecific competition) of the wood mouse population. When the abundance of acorns was high, density dependence was relaxed, and as a result the equilibrium density at which the population growth rate decreased to zero became higher. Those effects of acorn abundance were regarded as a nonlinear perturbation effect (sensu Royama 1992). The nonlinearity of density dependence was also detected; higher densities had stronger effects on population growth rates. Electronic supplementary material The online version of this article (doi:) contains supplementary material, which is available to authorized users.     

Are patterns of density dependence in the Global Population Dynamics Database driven by uncertainty about population abundance?

Density dependence in population growth rates is of immense importance to ecological theory and application, but is difficult to estimate. The Global Population Dynamics Database (GPDD), one of the largest collections of population time series available, has been extensively used to study cross-taxa patterns in density dependence. A major difficulty with assessing density dependence from time series is that uncertainty in population abundance estimates can cause strong bias in both tests and estimates of strength. We analyse 627 data sets in the GPDD using Gompertz population models and account for uncertainty via the Kalman filter. Results suggest that at least 45% of the time series display density dependence, but that it is weak and difficult to detect for a large fraction. When uncertainty is ignored, magnitude of and evidence for density dependence is strong, illustrating that uncertainty in abundance estimates qualitatively changes conclusions about density dependence drawn from the GPDD.     

Identifying environmental signals from population abundance data using multivariate time-series analysis

Individual organisms are affected by various natural and anthropogenic environmental factors throughout their life history. This is reflected in the way population abundance fluctuates. Consequently, observed population dynamics are often produced by the superimposition of multiple environmental signals. This complicates the analysis of population time-series. Here, a multivariate time-series method called maximum autocorrelation factor analysis (MAFA) was used to extract underlying signals from multiple population time series data. The extracted signals were compared with environmental variables that were suspected to affect the populations. Finally, a simple multiple regression analysis was applied to the same data set, and the results from the regression analysis were compared with those from MAFA. The extracted signals with MAFA were strongly associated with the environmental variables, suggesting that they represent environmental factors. On the other hand, with the multiple regression analysis, one of the important signals was not identifiable, revealing the shortcoming of the conventional approach. MAFA summarizes data based on their lag-one autocorrelation. This allows the identification of underlying signals with a small effect size on population abundance during the observation. It also uses multiple time series collected in parallel; this enables us to effectively analyze short time series. In this study, annual spawning adult counts of Chinook salmon at various locations within the Klamath Basin, California, were analyzed.     

Density dependence,boundedness, and attraction: detecting stability in stochastic systems

Summary By analogy with deterministic stability, the stability of stochastic ecological systems can be viewed as a tendency for population densities to avoid dynamic boundaries (i.e. boundedness) or to approach a dynamic attractor (i.e. attraction). At the population level, these two views generate predictions consistent with density dependence. I therefore devised two new statistical tests of attraction, the random-walk attraction test and the randomized attraction test; I then used them successfully, along with randomization techniques that detect boundedness and two autocorrelation methods, to test for density dependence in published sequences of population densities. The attraction tests identify the apparent attractor, the band of densities toward which density tends to shift between generations. Locating the apparent attractor can generate a prediction of the next direction of density change; for data from a dragonfly assemblage, about 80% of these predictions were correct. From the single-population tests, I also developed two multispecies tests of attraction (the multispecies random-walk and randomized attraction tests) and two multispecies tests of boundedness (the multispecies permutation and randomization tests). These detected attraction and boundedness in the dragonfly assemblage and attraction in a collection of laboratory fruitfly populations. An evaluation of the statistical power of the new density attraction tests indicates a strong dependence on the sequence length n and on the number of populations m: power increases with n and particularly with m. Nevertheless, detecting attraction becomes likely even in populations with strong linear density-dependence only with n>30 or for shorter sequences in multispecies assemblages.     

Testing the predictive performance of distribution models

Distribution models are used to predict the likelihood of occurrence or abundance of a species at locations where census data are not available. An integral part of modelling is the testing of model performance. We compared different schemes and measures for testing model performance using 79 species from the North American Breeding Bird Survey. The four testing schemes we compared featured increasing independence between test and training data: resubstitution, random data hold‐out and two spatially segregated data hold‐out designs. The different testing measures also addressed different levels of information content in the dependent variable: regression R² for absolute abundance, squared correlation coefficient r² for relative abundance and AUC/Somer’s D for presence/absence. We found that higher levels of independence between test and training data lead to lower assessments of prediction accuracy. Even for data collected independently, spatial autocorrelation leads to dependence between random hold‐out test data and training data, and thus to inflated measures of model performance. While there is a general awareness of the importance of autocorrelation to model building and hypothesis testing, its consequences via violation of independence between training and testing data have not been addressed systematically and comprehensively before. Furthermore, increasing information content (from correctly classifying presence/absence, to predicting relative abundance, to predicting absolute abundance) leads to decreasing predictive performance. The current tests for presence/absence distribution models are typically overly optimistic because a) the test and training data are not independent and b) the correct classification of presence/absence has a relatively low information content and thus capability to address ecological and conservation questions compared to a prediction of abundance. Meaningful evaluation of model performance requires testing on spatially independent data, if the intended application of the model is to predict into new geographic or climatic space, which arguably is the case for most applications of distribution models.     

Do persistent rare species experience stronger negative frequency dependence than common species?

Understanding why so many species are rare yet persistent remains a significant challenge for both theoretical and empirical ecologists. Yenni et al. (2012, Ecology, 93, 456–461) proposed that strong negative frequency dependence causes species to be rare while simultaneously buffering them against extinction. This hypothesis predicts that, on average, rare species should experience stronger negative frequency dependence than common species. However, it is unknown if ecological communities generally show this theoretical pattern. We discuss the implications of this phenomenon for community dynamics, and develop a method to test for a non‐random relationship between negative frequency dependence and relative abundance using species abundance data from 90 communities across a broad range of environments and taxonomic groups. To account for biases introduced by measurement error, we compared the observed correlation between species relative abundance and the strength of frequency dependence against expectations from a randomization procedure. In approximately half of the analysed communities, we found increasingly strong negative frequency dependence with decreasing relative abundance: rare species experienced stronger frequency dependence than common species. The randomization test never detected stronger negative frequency dependence in more common species. Our results suggest that strong negative frequency dependence is a signature of persistent, rare species in many communities.