Testing prediction accuracy in short-term ecological studies |
| |
| Institution: | 1. Institute for Global Food Security, School of Biological Sciences, Queen''s University Belfast, Belfast BT9 5DL, Northern Ireland;2. DSI/NRF Research Chair in Inland Fisheries and Freshwater Ecology, South African Institute for Aquatic Biodiversity, Makhanda 6140, South Africa;3. Ecology and Evolutionary Biology, School of Biological Sciences, University of Reading, Harborne Building, Reading RG6 6AS, England;4. Department of Ecology and Resource Management, University of Venda, Thohoyandou 0950, South Africa;5. South African Institute for Aquatic Biodiversity, Makhanda 6140, South Africa;6. Department of Biological Sciences and Biotechnology, Botswana International University of Science and Technology, Palapye, Botswana;7. Department of Zoology and Entomology, Rhodes University, Makhanda 6140, South Africa;1. Departamento de Ecología, Genética y Evolución, Instituto IEGEBA (CONICET-UBA), Facultad de Ciencias Exactas y Naturales, Universidad de Buenos Aires, Ciudad Universitaria, Ciudad Autónoma de Buenos Aires, Intendente Güiraldes 2160, C1428EGA, Argentina;2. División de Mastozoología, Museo Argentino de Ciencias Naturales “Bernardino Rivadavia,” Avenida Ángel Gallardo 470, C1405DJR Buenos Aires, Argentina.;1. Swiss Federal Institute for Forest, Snow and Landscape Research WSL, Zürcherstrasse 111, CH-8093 Birmensdorf, Switzerland;2. Department of Evolutionary Biology and Environmental Studies, University of Zurich, Winterthurerstrasse 190, 8057 Zürich, Switzerland;3. info fauna karch, UniMail, Bâtiment G, Bellevaux 51, 2000 Neuchâtel, Switzerland |
| |
| Abstract: | Applied ecology is based on an assumption that a management action will result in a predicted outcome. Testing the prediction accuracy of ecological models is the most powerful way of evaluating the knowledge implicit in this cause-effect relationship, however, the prevalence of predictive modeling and prediction testing are spreading slowly in ecology. The challenge of prediction testing is particularly acute for small-scale studies, because withholding data for prediction testing (e.g., via k-fold cross validation) can reduce model precision. However, by necessity small-scale studies are common. We use one such study that explored small mammal abundance along an elevational gradient to test prediction accuracy of models with varying degrees of information content. For each of three small mammal species, we conducted 5000 iterations of the following process: (1) randomly selected 75 % of the data to develop generalized linear models of species abundance that used detailed site measurements as covariates, (2) used an information theoretic approach to compare the top model with detailed covariates to habitat type-only and null models constructed with the same data, (3) tested those models’ ability to predict the 25 % of the randomly withheld data, and (4) evaluated prediction accuracy with a quadratic loss function. Detailed models fit the model-evaluation data best but had greater expected prediction error when predicting out-of-sample data relative to the habitat type models. Relationships between species and detailed site variables may be evident only within the framework of explicitly hierarchical analyses. We show that even with a small but relatively typical dataset (n = 28 sampling locations across 125 km over two years), researchers can effectively compare models with different information content and measure models’ predictive power, thus evaluating their own ecological understanding and defining the limits of their inferences. Identifying the appropriate scope of inference through prediction testing is ecologically valuable and is attainable even with small datasets. |
| |
| Keywords: | Elevational gradient Expected prediction error Model validation Scale dependency Small mammals |
| 本文献已被 ScienceDirect 等数据库收录！ |
|
