Flexible species distribution modelling methods perform well on spatially separated testing data

Aim

To assess whether flexible species distribution models that perform well at nearby testing locations still perform strongly when evaluated on spatially separated testing data.

Location

Australian Wet Tropics (AWT), Ontario, Canada (CAN), north-east New South Wales, Australia (NSW), New Zealand (NZ), five countries of South America (SA), and Switzerland (SWI).

Time period

Most species data were collected between 1950 and 2000.

Major taxa studied

Birds, mammals, plants and reptiles.

Methods

We compared 10 species distribution modelling methods with varying flexibility in terms of the allowed complexity of their fitted functions boosted regression trees (BRT), generalized additive model (GAM), multivariate adaptive regression splines (MARS), maximum entropy (MaxEnt), support vector machine (SVM), variants of generalized linear model (GLM) and random forest (RF), and an Ensemble model]. We used established practices for model selection to avoid overfitting, including parameter tuning in learning methods. Models were trained on presence–background data for 171 species and tested on presence–absence data. Training and testing data were separated using both random and spatial partitioning, the latter based on 75-km blocks. We calculated the average performance and mean rank of the methods (focussing on the area under the receiver operating characteristic and precision-recall gain curves, and correlation) and assessed the statistical significance of the differences between them.

Results

The ranking of methods did not change when evaluated on spatially separated testing data. Methods with the strongest predictive performance were nonparametric methods known to be flexible. An ensemble formed by averaging predictions of five pre-selected modelling methods was the best model in both random and spatial partitioning, followed by MaxEnt and a variant of random forest.

Main conclusions

Whilst some modellers expect methods limited to simple smooth functions to predict better spatially separated data, we found no evidence of that using blocks of 75 km. We conclude that flexible models that are tuned well enough to avoid overfitting are effective at predicting to spatially distinct areas.