Modelling niche and neutral dynamics: on the ecological interpretation of variation partitioning results

Many studies have attempted to disentangle the effects of neutral and niche‐mediated processes on community composition using partial Mantel tests and variance partitioning on dissimilarity matrices. Recently, doubts about the reliability of these methods have emerged. Here we explore how the results are affected by three confounding factors that may affect ecological data to different degrees: spatial autocorrelation of the environmental variables, length of the compositional gradient, and sampling noise. We document that the statistical hypotheses tested in these methods may or may not correspond to the ecological hypotheses of interest. A major discrepancy emerges if a large proportion of sampling units in the analysed dataset share no species, in which case compositional dissimilarities saturate to a fixed maximum value although explanatory dissimilarities do not. With increasing dissimilarity saturation, the explanatory power of regression models decrease, which may lead to the erroneous conclusion that the ecological processes represented by the explanatory variables are not operating. A survey of recent literature suggests that there is a general lack of awareness of this problem, although it appears to affect > 10% of relevant studies. Our simulations show that if dissimilarity saturation is due to a long ecological gradient, extended dissimilarities essentially solve the problem for any degree of saturation. Using distances from a hybrid multidimensional scaling alleviates the saturation problem when the degree of saturation is < 60%. However, neither correction method can provide a solution to problems caused by insufficient sampling. How the presence of multiple explanatory gradients in combination with sampling noise affects overall analysis performance remains to be clarified.