The devil lies in details: reply to Stuart Hurlbert

As pointed out in Stuart Hurlbert's recent article, ecologists still at times design their experiments sloppily, creating a situation where various forms of ‘non‐demonic intrusion’ could account for the documented contrasts between treatments and controls. If such contrasts are nevertheless presented to the reader as if they were statistically demonstrated treatment effects, then pseudoreplication is not a pseudoissue and the use of a stigmatizing label of is entirely warranted, as pointed out by Hurlbert. The problems with Hurlbert's concepts start in the context of studies, where the scope of the experiment is to provoke a chain of dramatic and a priori extremely unlikely events, which a given conjecture predicts to happen as a consequence of a given manipulation. As the essence of these experiments is to trigger large dynamical responses in a biological system, they often require much space and/or special conditions, allowing for efficient isolation of the experimental system from potential sources of contamination. These constraints can be incompatible with standard designs (randomization, replication and treatment‐control interspersion). In the context of experiments, where it has been necessary to sacrifice randomization, replication or treatment‐control interspersion, the logic of inferring treatment effects is the same as used when interpreting causes of spontaneous events or events triggered by manipulations with practical purposes. The observed contrasts can be reasonably interpreted as effects of the treatment if and only if their magnitudes and the timing of their emergence makes alternative explanations utterly implausible (which is up to the reader to judge). If the logic of inference is clearly explained and no claim of statistically demonstrated treatment effect is made, the use of stigmatizing labels like ‘pseudoreplication’ is unwarranted. However, it might clarify the literature if such imperfectly designed experiments are referred to as experimental events, to be distinguished from perfectly designed experiments, where mechanical interpretation of contrasts between treatments and controls as treatment effects can be regarded as socially acceptable.     

Terrestrial ecologists ignore aquatic literature: Asymmetry in citation breadth in ecological publications and implications for generality and progress in ecology

The search for generality in ecology should include assessing the influence of studies done in one system on those done in other systems. Assuming generality is reflected in citation patterns, we analyzed frequencies of terrestrial, marine, and freshwater citations in papers categorized as terrestrial, marine and freshwater in high-impact “general” ecological journals. Citation frequencies were strikingly asymmetric. Aquatic researchers cited terrestrial papers ~ 10 times more often than the reverse, implying uneven cross-fertilization of information between aquatic and terrestrial ecologists. Comparisons between citation frequencies in the early 1980s and the early 2000s for two of the seven journals yielded similar results. Summing across all journals, 60% of all research papers (n = 5824) published in these journals in 2002–2006 were terrestrial vs. 9% freshwater and 8% marine. Since total numbers of terrestrial and aquatic ecologists are more similar than these proportions suggest, the representation of publications by habitat in “general” ecological journals appears disproportional and unrepresentative of the ecological science community at large. Such asymmetries are a concern because (1) aquatic and terrestrial systems can be tightly integrated, (2) pressure for across-system understanding to meet the challenge of climate change is increasing, (3) citation asymmetry implies barriers to among-system flow of understanding, thus (4) impeding scientific and societal progress. Changing this imbalance likely depends on a bottom-up approach originating from the ecological community, through pressure on societies, journals, editors and reviewers.     

Disproportional Representation of Primates in the Ecological Literature

We address the question why papers dealing with the ecology of primates are so sparsely represented in the general ecological literature. A literature analyses based on entries in Web of Science and PrimateLit reveals that despite a large number of papers published on primates in general and on the ecology of primates, only a very small fraction of these papers is published in high-ranking international ecological journals. We discuss a number of potential reasons for the disproportion and highlight the problems associated with experimental research on wild primates and constraints on sample size as major issues.     

In the deep end: pooling data and other statistical challenges of zoo and aquarium research

Zoo and aquarium research presents many logistic challenges, including extremely small sample sizes and lack of independent data points, which lend themselves to the misuse of statistics. Pseudoreplication and pooling of data are two statistical problems common in research in the biological sciences. Although the prevalence of these and other statistical miscues have been documented in other fields, little attention has been paid to the practice of statistics in the field of zoo biology. A review of articles published in the journal Zoo Biology between 1999–2004 showed that approximately 40% of the 146 articles utilizing inferential statistics during that span contained some evidence of pseudoreplication or pooling of data. Nearly 75% of studies did not provide degrees of freedom for all statistics and approximately 20% did not report test statistic values. Although the level of pseudoreplication in this dataset is not outside the levels found in other branches of biology, it does indicate the challenges of dealing with appropriate data analysis in zoo and aquarium studies. The standardization of statistical techniques to deal with the methodological challenges of zoo and aquarium populations can help advance zoo research by guiding the production and analysis of applied studies. This study recommends techniques for dealing with these issues, including complete disclosure of data manipulation and reporting of statistical values, checking and control for institutional effects in statistical models, and avoidance of pseudoreplicated observations. Additionally, zoo biologists should seek out other models such as hierarchical or factorial models or randomization tests to supplement their repertoire of t‐tests and ANOVA. These suggestions are intended to stimulate conversation and examination of the current use of statistics in zoo biology in an effort to develop more consistent requirements for publication. Zoo Biol 0:1–14, 2006. © 2006 Wiley‐Liss, Inc.     

生态学试验设计与解释中的常见问题

一个"好"的试验,统计检验的显著差异可证明处理效应存在;而一个"坏"的试验,统计检验的显著差异本身并不能证明处理效应的真实存在.试验单位的不独立就可能使干扰因素偏倚地影响试验结果,是多种形式的伪重复的根本原因.如果不统筹地考虑试验方法和数据分析方法,环境要素的时空格局就可能被错误地当成处理效应.以案例分析的形式探讨了中国生态学试验设计与解释中常见的3个问题:(1)简单伪重复问题,这是Hurlbert[2]早已指出的伪重复情形中的一种,通常是把取样的重复当成了处理的重复;(2)把反复测量结果当成重复的问题,即对同一个对象的反应变量前后进行多次观测,却把这些观测值视为重复而进行统计检验所造成的问题;(3)混淆时空效应与处理效应的问题,由于取样方法(破坏性取样)或者研究对象(例如流动的水体)性质的特殊性等原因,数据中所体现出来的格局有可能由于时空效应而并非处理效应所造成.在这3种情况下,数据的产生方式与所用统计方法的前提相违背.     

Pseudoreplication, chatter, and the international nature of science: a response to D.V. Tatarnikov

The commentary by Tatarnikov (2005) on the design and analysis of manipulative experiments in ecology represents an obvious danger to readers with poor knowledge of modern statistics due to its erroneous interpretation of pseudoreplication and statistical independence. Here we offer clarification of those concepts--and related ones such as experimental unit and evaluation unit--by reference to studies cited by Tatarnikov (2005). We stress the necessity of learning from the accumulated experience of the international scientific community in order not to repeat the errors found in earlier publications that have already been analyzed and widely written about. (An Englisch translation of the full article is available as a pdf-file from either or the authors.)     

Pseudofactorialism,response structures and collective responsibility

Pseudofactorialism is defined as ‘the invalid statistical analysis that results from the misidentification of two or more response variables as representing different levels of an experimental variable or treatment factor. Most often the invalid analysis consists of use of an (n + 1)‐way anova in a situation where two or more n‐way anova s would be the appropriate approach’. I and my students examined a total of 1362 papers published from the 1960s to 2009 reporting manipulative experiments, primarily in the field of ecology. The error was present in 7% of these, including 9% of 80 experimental papers examined in 2009 issues of Ecology and the Journal of Animal Ecology. Key features of 60 cases of pseudofactorialism are tabulated as a basis for discussion of the varied ways and circumstances in which the error can occur. As co‐authors, colleagues, editors and anonymous referees and editors who approved them for publication, a total of 459 persons other than the senior authors shared responsibility for these 60 papers. Pseudofactorialism may sometimes be motivated by a desire to test whether different response variables respond in the same way to treatment factors. Proper procedures for doing that are briefly reviewed. A major cause of pseudofactorialism is the widespread failure in statistics texts, primary literature and documentation for statistics software packages to distinguish the three major components of experimental design – treatment structure, design structure, response structure – and clearly define key terms such as experimental unit, evaluation unit, split unit, factorial and repeated measures. A quick way to check for the possible presence of the pseudofactorialism is to determine whether the number of valid experimental units in a study is smaller than (i) the error degrees of freedom in a multi‐way anova ; or (ii) the total number of tallies (N) in a multi‐way contingency table. Such situations also can indicate the commission of pseudoreplication, however.     

论文引用率影响因素——中外生态学期刊比较

本文选择8种有代表性中外生态学期刊,对其一定发表周期内的论文引用率进行分析,探讨生态学论文引用率的影响因素及中外生态学期刊的差异.结果表明：4种英文期刊的年均被引次数均远大于4种中文期刊;英文期刊1位作者的论文数量百分比相对较高;所有期刊的合著论文比例均较高,体现了合作性在现代生态学研究中的重要性;论文作者数量与引用率之间有一定的正相关关系,但不显著;英文期刊论文的长度显著高于中文论文;随着论文长度的增加,年均被引次数增多.对中外期刊论文的引用率变化动态进行分析表明,英文期刊中总被引次数高的论文其增长速率也较快,表明其持续影响力强于中文生态学论文.我们希望这些结果会对生态学者以及相关期刊工作者有所裨益.     

Problems of kin recognition

Behavioural ecologists have long assumed that animals discriminate between their kin and non-kin, but paid little attention to how animals recognize their relatives. Although the first papers on kin recognition mechanisms appeared barely 10 years ago, studies now appear frequently in journals of animal behaviour. Initial findings reveal that kin recognition abilities are surprisingly well-distributed throughout the animal kingdom. Yet an understanding of the evolutionary and ecological significance of these abilities demands further analyses of the components of kin recognition mechanisms and the social contexts in which they are expressed. Many controversies and unresolved issues remain, and experimental approaches to these problems promise to continue making kin recognition an important, rapidly moving discipline within behavioural ecology.     

Logic of experiments in ecology: is pseudoreplication a pseudoissue?

Hurlbert divides experimental ecologist into ‘those who do not see any need for dispersion (of replicated treatments and controls), and those who do recognize its importance and take whatever measures are necessary to achieve a good dose of it’. Experimental ecologists could also be divided into those who do not see any problems with sacrificing spatial and temporal scales in order to obtain replication, and those who understand that appropriate scale must always have priority over replication. If an experiment is conducted in a spatial or temporal scale, where the predictions of contesting hypotheses are convergent or ambiguous, no amount of technical impeccability can make the work instructive. Conversely, replication can always be obtained afterwards, by conducting more experiments with basically similar design in different areas and by using meta‐analysis. This approach even reduces the sampling bias obtained if resources are allocated to a small number of well‐replicated experiments. For a strict advocate of the hypothetico‐deductive method, replication is unnecessary even as a matter of principle, unless the predicted response is so weak that random background noise is a plausible excuse for a discrepancy between predictions and results. By definition, a prediction is an ‘all‐statement’, referring to all systems within a well‐defined category. What applies to all must apply to any. Hence, choosing two systems and assigning them randomly to a treatment and a control is normally an adequate design for a deductive experiment. The strength of such experiments depends on the firmness of the predictions and their a priori probability of corroboration. Replication is but one of many ways of reducing this probability. Whether the experiment is replicated or not, inferential statistics should always be used, to enable the reader to judge how well the apparent patterns in samples reflect real patterns in statistical populations. The concept ‘pseudoreplication’ amounts to entirely unwarranted stigmatization of a reasonable way to test predictions referring to large‐scale systems.     

Mark-mark scatterplots improve pattern analysis in spatial plant ecology

Point process statistics provides valuable tools for many ecological studies, where ‘points’ are commonly determined to represent the locations of plants or animals and ‘marks’ are additional items such as species or size. In the statistical analysis of marked point patterns, various correlation functions are used such as the mark variogram or the mark correlation function. Often the interpretation of these functions is not easy and the non-spatial ecologist is in need of support. In order to make the analysis of spatial point patterns more accessible to ecologists, we introduced and tested a new graphical method, the mark-mark scatterplot. This plot visualises the marks of point pairs of inter-point distances r smaller than some small distance r_max. We tested the application of the mark-mark scatterplot by reconsidering three quite different tree patterns: a pattern of longleaf pine trees from the southern US which was strongly influenced by fires, a tropical tree pattern of the species Shorea congestiflora from Sri Lanka and a Scots pine pattern from Siberia (Russia). The new method yielded previously undetected cause-effect information on mark behaviour at short inter-point distances and thus improved the analysis with mark correlation functions as well as complemented the information they provided. We discovered important new correlations in clusters of trees at close proximity. The application of the mark-mark scatterplot will facilitate the interpretation of point process summary statistics and will make point process analysis more accessible to ecologists not specialized in point process statistics.     

Studying invasion: have we missed the boat?

Invasive species, and the ensuing homogenization of the world's biota, form a global problem with consequences ranging from the decline and extirpation of native species to threats to human health. The magnitude of this issue demands a thorough understanding of the invasion process, which consists of three main stages: initial dispersal, establishment of self‐sustaining populations, and spread. To assess the relative distribution of research effort among these stages, we conducted a literature review using 873 articles published in 23 major journals over the past 10 years. Of the 873 papers, only 96 (11.0%) studied initial dispersal, and only half of these (6.2% of the total) were empirical. As the first stage in a contingent process, we argue that initial dispersal is the best stage during which to direct management efforts. In addition, initial dispersal has direct relevance for fundamental ecological questions regarding community assembly and metacommunity dynamics. In so far that answering these questions and preventing invasion are goals of ecologists, the disparity in research effort noted here suggests that ecologists need to expand their efforts to include more research on initial dispersal.     

Trends and prospects for local knowledge in ecological and conservation research and monitoring

Local ecological knowledge (LEK) of those who earn their livelihoods from natural environments has long been recognized as providing far-reaching insights into ecological processes. It is being increasingly used by ecologists to address diverse questions that often focus on applied conservation issues and may incorporate local knowledge with biological data from more conventional research and monitoring. We characterize how LEK has been used in the ecological and conservation literature over the last 25 years by broadly examining 360 journals and by evaluating 12 prominent ecological and conservation journals in greater detail. Over this period, the use of LEK has increased considerably, although only 0.01% of papers in the broad and 0.42% of those in the more detailed evaluation incorporated LEK. Despite this increase, LEK-based publications remain nearly absent from the more established theoretical literature and are largely restricted to more recent and arguably less prestigious applied and interdisciplinary journals. Most LEK studies used interviews, but generally failed to actively include community members in the research process. Changes to the research and publishing process that include local people and address these shortcomings and the broader issues of power and influence in the sciences are critical to the successful utilization of LEK. These changes are necessary for the appropriate depiction of these knowledge systems and to ensure that local knowledge holders will continue participating in ecological research aimed at conservation.     

Symposium introduction: The importance of scale in ecology

Abstract Conceptual issues about scale, both spatial and temporal, have had considerable influence on the way in which ecologists view populations, communities and ecosystems. Scale includes two aspects: the extent over which a community or ecosystem is studied, and the resolution or ‘grain’ at which measurements or experiments are conducted. We illustrate the influence of extent and grain on perceptions of ecological patterns and processes, derived from fundamental measurements, field experiments and theory and modelling. These concepts provide background for a series of subsequent papers that were presented at a symposium on spatial and temporal scaling in freshwater systems. These papers conclude that multi-scale measurements and experiments plus novel methodologies for analysing large-scale surveys and manipulations should be priorities for future research in freshwater systems.     

MIQE précis: Practical implementation of minimum standard guidelines for fluorescence-based quantitative real-time PCR experiments

The conclusions of thousands of peer-reviewed publications rely on data obtained using fluorescence-based quantitative real-time PCR technology. However, the inadequate reporting of experimental detail, combined with the frequent use of flawed protocols is leading to the publication of papers that may not be technically appropriate. We take the view that this problem requires the delineation of a more transparent and comprehensive reporting policy from scientific journals. This editorial aims to provide practical guidance for the incorporation of absolute minimum standards encompassing the key assay parameters for accurate design, documentation and reporting of qPCR experiments (MIQE précis) and guidance on the publication of pure 'reference gene' articles.