Validation of null model tests using Neyman–Pearson hypothesis testing theory

A long-standing question in ecology is whether interspecific competition affects co-occurrence patterns of species. Null model tests of presence–absence data (NMTPAs) constitute an important approach to address the question, but different tests often give conflicting results when applied to the same data. Neyman–Pearson hypothesis testing theory provides a rigorous and well accepted framework for assessing the validity and optimality of statistical tests. Here, I treat NMTPAs within this framework, and measure the robustness and bias of 72 representative tests. My results indicate that, when restrictive assumptions are met, existing NMTPAs are adequate, but for general testing situations, the use of all existing NMTPAs is unjustified — the tests are nonrobust or biased. For many current applications of NMTPAs, restrictive assumptions appear unmet, so these results illustrate an area in which existing NMTPAs can be improved. In addition to highlighting useful improvements to existing NMTPAs, the results here provide a rigorous framework for developing improved methods.     

Generating and testing predictions about community structure: Which theory is relevant and can it be tested with observational data?

Three issues are discussed relevant to the controversy over using null models and observational data on guild structure to test community-level predictions based on limiting similarity theory. First, I argue that most limiting similarity theory is not based on reasonable assumptions for plants and that the theory that is relevant does not generate any predictions about expected guild proportionality on a small spatial scale. Therefore, regardless of adequacy of the statistical methods, the predictions being tested by the body of literature using null models to test for niche limitation are unlikely to be relevant in most plant comunities. Second, assuming that the predictions are after all worth being tested, I argue that most tests using the guild approach do not provide adequate explanations of how the defined guilds could lead to greater competition within vs. between guilds. If this is not true for the particular guilds used, the predictions of guild proportionality or size constancy will not be valid. Third, I address the controversy over whether field experiments can provide more solid evidence than observational data about the role of competition in determining community structure by (1) suggesting methods of dealing with potential drawbacks of field experiments, and (2) suggesting alternative experimental approaches for directly addressing issues about community structure.     

Random sampling does not exclude spatial dependence: The importance of neutral models for ecological hypothesis testing

Lájer (2007) notes that, to investigate phytosociological and ecological relationships, many authors apply traditional inferential tests to sets of relevés obtained by non-random methods. Unfortunately, this procedure does not provide reliable support for hypothesis testing because non-random sampling violates the assumptions of independence required by many parametric inferential tests. Instead, a random sampling scheme is recommended. Nonetheless, random sampling will not eliminate spatial autocorrelation. For instance, a classical law of geography holds that everything in a piece of (biotic) space is interrelated, but near objects are more related than distant ones. Because most ecological processes that shape community structure and species coexistence are spatially explicit, spatial autocorrelation is a vital part of almost all ecological data. This means that, independently from the underlying sampling design, ecological data are generally spatially autocorrelated, violating the assumption of independence that is generally required by traditional inferential tests. To overcome this drawback, randomization tests may be used. Such tests evaluate statistical significance based on empirical distributions generated from the sample and do not necessarily require data independence. However, as concerns hypothesis testing, randomization tests are not the universal remedy for ecologists, because the choice of inadequate null models can have significant effects on the ecological hypotheses tested. In this paper, I emphasize the need of developing null models for which the statistical assumptions match the underlying biological mechanisms.     

A general approach for sample size and power calculations based on the Haseman-Elston method

Risch and Zhang (1995; Science 268: 1584-9) reported a simple sample size and power calculation approach for the Haseman-Elston method and based their computations on the null hypothesis of no genetic effect. We argue that the more reasonable null hypothesis is that of no recombination. For this null hypothesis, we provide a general approach for sample size and power calculations within the Haseman-Elston framework. We demonstrate the validity of our approach in a Monte-Carlo simulation study and illustrate the differences using data from published segregation analyses on body weight and heritability estimates on carotid artery artherosclerotic lesions.     

Exact sample size determination for a single Poisson random sample

Classical power analysis for sample size determination is typically performed in clinical trials. A “hybrid” classical Bayesian or a “fully Bayesian” approach can be alternatively used in order to add flexibility to the design assumptions needed at the planning stage of the study and to explicitly incorporate prior information in the procedure. In this paper, we exploit and compare these approaches to obtain the optimal sample size of a single-arm trial based on Poisson data. We adopt exact methods to establish the rejection of the null hypothesis within a frequentist or a Bayesian perspective and suggest the use of a conservative criterion for sample size determination that accounts for the not strictly monotonic behavior of the power function in the presence of discrete data. A Shiny web app in R has been developed to provide a user-friendly interface to easily compute the optimal sample size according to the proposed criteria and to assure the reproducibility of the results.     

Nonparametric comparison of two survival-time distributions in the presence of dependent censoring

When testing the null hypothesis that treatment arm-specific survival-time distributions are equal, the log-rank test is asymptotically valid when the distribution of time to censoring is conditionally independent of randomized treatment group given survival time. We introduce a test of the null hypothesis for use when the distribution of time to censoring depends on treatment group and survival time. This test does not make any assumptions regarding independence of censoring time and survival time. Asymptotic validity of this test only requires a consistent estimate of the conditional probability that the survival event is observed given both treatment group and that the survival event occurred before the time of analysis. However, by not making unverifiable assumptions about the data-generating mechanism, there exists a set of possible values of corresponding sample-mean estimates of these probabilities that are consistent with the observed data. Over this subset of the unit square, the proposed test can be calculated and a rejection region identified. A decision on the null that considers uncertainty because of censoring that may depend on treatment group and survival time can then be directly made. We also present a generalized log-rank test that enables us to provide conditions under which the ordinary log-rank test is asymptotically valid. This generalized test can also be used for testing the null hypothesis when the distribution of censoring depends on treatment group and survival time. However, use of this test requires semiparametric modeling assumptions. A simulation study and an example using a recent AIDS clinical trial are provided.     

Simulation of community patterns

Programs are described for the simulation of coenoplanes-mathematically defined, two-dimensional community patterns characterized by scattered and broadly overlapping Gaussian reponse surfaces for species populations. Field data have been studied to derive a set of assumptions regarding realistic properties of coenoplanes and ways in which some of these properties should be variable. Additional assumptions define properties of samples and sets of samples taken from coenoplanes. The programs thus provide a basis for tests of different ordination techniques and effects of characteristics of sample sets on these.     

From diversity indices to community assembly processes: a test with simulated data

Ecological theory suggests that spatial distribution of biodiversity is strongly driven by community assembly processes. Thus the study of diversity patterns combined with null model testing has become increasingly common to infer assembly processes from observed distributions of diversity indices. However, results in both empirical and simulation studies are inconsistent. The aim of our study is to determine with simulated data which facets of biodiversity, if any, may unravel the processes driving its spatial patterns, and to provide practical considerations about the combination of diversity indices that would produce significant and congruent signals when using null models. The study is based on simulated species’ assemblages that emerge under various landscape structures in a spatially explicit individual‐based model with contrasting, predefined assembly processes. We focus on four assembly processes (species‐sorting, mass effect, neutral dynamics and competition colonization trade‐off) and investigate the emerging species’ distributions with varied diversity indices (alpha, beta and gamma) measured at different spatial scales and for different diversity facets (taxonomic, functional and phylogenetic). We find that 1) the four assembly processes result in distinct spatial distributions of species under any landscape structure, 2) a broad range of diversity indices allows distinguishing between communities driven by different assembly processes, 3) null models provide congruent results only for a small fraction of diversity indices and 4) only a combination of these diversity indices allows identifying the correct assembly processes. Our study supports the inference of assembly processes from patterns of diversity only when different types of indices are combined. It highlights the need to combine phylogenetic, functional and taxonomic diversity indices at multiple spatial scales to effectively infer underlying assembly processes from diversity patterns by illustrating how combination of different indices might help disentangling the complex question of coexistence.     

Incorporating intraspecific variation in tests of trait-based community assembly

Environmental filtering and niche differentiation are processes proposed to drive community assembly, generating nonrandom patterns in community trait distributions. Despite the substantial intraspecific trait variation present in plant communities, most previous studies of trait-based community assembly have used species mean trait values and therefore not accounted for intraspecific variation. Using a null model approach, I tested for environmental filtering and niche differentiation acting on three key functional traits??vegetative height, specific leaf area (SLA), and leaf dry matter content (LDMC)??in old-field plant communities. I also examined how accounting for intraspecific variation at the among-plot and individual levels affected the detection of nonrandom assembly patterns. Tests using fixed species mean trait values provided evidence of environmental filtering acting on height and SLA and niche differentiation acting on SLA. Including plot-level intraspecific variation increased the strength of these patterns, indicating an important role of intraspecific variation in community assembly. Tests using individual trait data indicated strong environmental filtering acting on all traits, but provided no evidence of niche differentiation, although these signals may have been obscured by the effects of dispersal limitation and spatial aggregation of conspecific individuals. There was also strong evidence of nonrandom assembly of individuals within single species, with the strength of environmental filtering varying among species. This study demonstrates that, while analyses using fixed species mean trait values can provide insights into community assembly processes, accounting for intraspecific variation provides a more complete view of communities and the processes driving their assembly.     

Trait evolution, community assembly, and the phylogenetic structure of ecological communities

Taxa co-occurring in communities often represent a nonrandom sample, in phenotypic or phylogenetic terms, of the regional species pool. While heuristic arguments have identified processes that create community phylogenetic patterns, further progress hinges on a more comprehensive understanding of the interactions between underlying ecological and evolutionary processes. We created a simulation framework to model trait evolution, assemble communities (via competition, habitat filtering, or neutral assembly), and test the phylogenetic pattern of the resulting communities. We found that phylogenetic community structure is greatest when traits are highly conserved and when multiple traits influence species membership in communities. Habitat filtering produces stronger phylogenetic structure when taxa with derived (as opposed to ancestral) traits are favored in the community. Nearest-relative tests have greater power to detect patterns due to competition, while total community relatedness tests perform better with habitat filtering. The size of the local community relative to the regional pool strongly influences statistical power; in general, power increases with larger pool sizes for communities created by filtering but decreases for communities created by competition. Our results deepen our understanding of processes that contribute to phylogenetic community structure and provide guidance for the design and interpretation of empirical research.     

Disentangling niche and neutral influences on community assembly: assessing the performance of community phylogenetic structure tests

Patterns of phylogenetic relatedness within communities have been widely used to infer the importance of different ecological and evolutionary processes during community assembly, but little is known about the relative ability of community phylogenetics methods and null models to detect the signature of processes such as dispersal, competition and filtering under different models of trait evolution. Using a metacommunity simulation incorporating quantitative models of trait evolution and community assembly, I assessed the performance of different tests that have been used to measure community phylogenetic structure. All tests were sensitive to the relative phylogenetic signal in species metacommunity abundances and traits; methods that were most sensitive to the effects of niche-based processes on community structure were also more likely to find non-random patterns of community phylogenetic structure under dispersal assembly. When used with a null model that maintained species occurrence frequency in random communities, several metrics could detect niche-based assembly when there was strong phylogenetic signal in species traits, when multiple traits were involved in community assembly, and in the presence of environmental heterogeneity. Interpretations of the causes of community phylogenetic structure should be modified to account for the influence of dispersal.     

The continuing misuse of null hypothesis significance testing in biological anthropology

There is over 60 years of discussion in the statistical literature concerning the misuse and limitations of null hypothesis significance tests (NHST). Based on the prevalence of NHST in biological anthropology research, it appears that the discipline generally is unaware of these concerns. The p values used in NHST usually are interpreted incorrectly. A p value indicates the probability of the data given the null hypothesis. It should not be interpreted as the probability that the null hypothesis is true or as evidence for or against any specific alternative to the null hypothesis. P values are a function of both the sample size and the effect size, and therefore do not indicate whether the effect observed in the study is important, large, or small. P values have poor replicability in repeated experiments. The distribution of p values is continuous and varies from 0 to 1.0. The use of a cut‐off, generally p ≤ 0.05, to separate significant from nonsignificant results, is an arbitrary dichotomization of continuous variation. In 2016, the American Statistical Association issued a statement of principles regarding the misinterpretation of NHST, the first time it has done so regarding a specific statistical procedure in its 180‐year history. Effect sizes and confidence intervals, which can be calculated for any data used to calculate p values, provide more and better information about tested hypotheses than p values and NHST.     

What Can Cross-Cultural Correlations Teach Us about Human Nature?

Many recent evolutionary psychology and human behavioral ecology studies have tested hypotheses by examining correlations between variables measured at a group level (e.g., state, country, continent). In such analyses, variables collected for each aggregation are often taken to be representative of the individuals present within them, and relationships between such variables are presumed to reflect individual-level processes. There are multiple reasons to exercise caution when doing so, including: (1) the ecological fallacy, whereby relationships observed at the aggregate level do not accurately represent individual-level processes; (2) non-independence of data points, which violates assumptions of the inferential techniques used in null hypothesis testing; and (3) cross-cultural non-equivalence of measurement (differences in construct validity between groups). We provide examples of how each of these gives rise to problems in the context of testing evolutionary hypotheses about human behavior, and we offer some suggestions for future research.     

初始投标值数量和样本容量对双边界二分式CVM的影响

双边界二分式条件价值评估法是当前广泛应用于评估生态环境和自然资源价值的陈述偏好法。为了研究双边界二分式CVM中,不同分布中初始投标值数量及样本容量对WTP的影响,应用蒙特卡洛模拟和支付意愿函数模型相结合的方法,用概率分位数设计投标值,分别选择数据来源不同的Weibull分布和对数Logistic分布,动态模拟不同初始投标值数量和样本容量对WTP期望值产生的影响。模拟结果表明,对不同的WTP概率分布,初始投标值数量和样本容量对WTP的影响趋势是一致的,当初始投标值数量和样本容量分别大于5和500时,初始投标值数量和样本容量的增加,对WTP估计值的影响程度越来越小。研究结果表明,在二分式CVM研究中,当以WTP期望为代表值时,其初始投标值数量和样本容量至少为5和500。研究结论为二分式CVM问卷设计中投标值数量及样本容量的设定提供参考依据。     

A Likelihood Ratio Test for the Nonparametric Behrens‐Fisher Problem

The nonparametric Behrens‐Fisher hypothesis is the most appropriate null hypothesis for the two‐sample comparison when one does not wish to make restrictive assumptions about possible distributions. In this paper, a numerical approach is described by which the likelihood ratio test can be calculated for the nonparametric Behrens‐Fisher problem. The approach taken here effectively reduces the number of parameters in the score equations to one by using a recursive formula for the remaining parameters. The resulting single dimensional problem can be solved numerically. The power of the likelihood ratio test is compared by simulation to that of a generalized Wilcoxon test of Brunner and Munzel. The tests have similar power for all alternatives considered when a simulated null distribution is used to generate cutoff values for the tests. The methods are illustrated on data on shoulder pain from a clinical trial.     

Worldwide patterns in species richness of Falconiformes: analytical null models, geometric constraints, and the mid-domain effect.

Recently, the hypothesis that the geographic distribution of species could be influenced by the shape of the domain edges, the so-called Mid-Domain Effect (MDE), has been included as one of the five credible hypotheses for explaining spatial gradients in species richness, despite all the unsuccessful current attempts to prove empirically the validity of MDE. We used data on spatial worldwide distributions of Falconiformes to evaluate the validity of MDE assumptions, incorporated into two different sorts of null models at a global level and separately across five domains/landmasses. Species richness values predicted by the null models of the MDE and those values predicted by Net Primary Productivity, a surrogate variable expressing the effect of available energy, were compared in order to evaluate which hypothesis better predicts the observed values. Our tests showed that MDE continues to lack empirical support, regardless of its current acceptability, and so, does not deserve to be classified as one possible explanation of species richness gradients.     

Assessing Equivalence Tests with Respect to their Expected p‐Value

Monte‐Carlo simulation methods are commonly used for assessing the performance of statistical tests under finite sample scenarios. They help us ascertain the nominal level for tests with approximate level, e.g. asymptotic tests. Additionally, a simulation can assess the quality of a test on the alternative. The latter can be used to compare new tests and established tests under certain assumptions in order to determinate a preferable test given characteristics of the data. The key problem for such investigations is the choice of a goodness criterion. We expand the expected p‐value as considered by Sackrowitz and Samuel‐Cahn (1999) to the context of univariate equivalence tests. This presents an effective tool to evaluate new purposes for equivalence testing because of its independence of the distribution of the test statistic under null‐hypothesis. It helps to avoid the often tedious search for the distribution under null‐hypothesis for test statistics which have no considerable advantage over yet available methods. To demonstrate the usefulness in biometry a comparison of established equivalence tests with a nonparametric approach is conducted in a simulation study for three distributional assumptions.     

Testing the correlation for clustered categorical and censored discrete time-to-event data when covariates are measured without/with errors

In the analysis of clustered categorical data, it is of common interest to test for the correlation within clusters, and the heterogeneity across different clusters. We address this problem by proposing a class of score tests for the null hypothesis that the variance components are zero in random effects models, for clustered nominal and ordinal categorical responses. We extend the results to accommodate clustered censored discrete time-to-event data. We next consider such tests in the situation where covariates are measured with errors. We propose using the SIMEX method to construct the score tests for the null hypothesis that the variance components are zero. Key advantages of the proposed score tests are that they can be easily implemented by fitting standard polytomous regression models and discrete failure time models, and that they are robust in the sense that no assumptions need to be made regarding the distributions of the random effects and the unobserved covariates. The asymptotic properties of the proposed tests are studied. We illustrate these tests by analyzing two data sets and evaluate their performance with simulations.     

Body size determines soil community assembly in a tropical forest

Tropical forests shelter an unparalleled biological diversity. The relative influence of environmental selection (i.e., abiotic conditions, biotic interactions) and stochastic–distance‐dependent neutral processes (i.e., demography, dispersal) in shaping communities has been extensively studied for various organisms, but has rarely been explored across a large range of body sizes, in particular in soil environments. We built a detailed census of the whole soil biota in a 12‐ha tropical forest plot using soil DNA metabarcoding. We show that the distribution of 19 taxonomic groups (ranging from microbes to mesofauna) is primarily stochastic, suggesting that neutral processes are prominent drivers of the assembly of these communities at this scale. We also identify aluminium, topography and plant species identity as weak, yet significant drivers of soil richness and community composition of bacteria, protists and to a lesser extent fungi. Finally, we show that body size, which determines the scale at which an organism perceives its environment, predicted the community assembly across taxonomic groups, with soil mesofauna assemblages being more stochastic than microbial ones. These results suggest that the relative contribution of neutral processes and environmental selection to community assembly directly depends on body size. Body size is hence an important determinant of community assembly rules at the scale of the ecological community in tropical soils and should be accounted for in spatial models of tropical soil food webs.