Negative changes in the scientific publication process in ecology: potential causes and consequences

1. Increasingly viewed to have societal impact and value, science is affected by complex changes such as globalisation and the increasing dominance of commercial interest. As a result, technical advancements, financial concerns, institutional prestige and journal proliferation have created challenges for ecological and other scientific journals and affected the perception of both researchers and the public about the science that they publish. 2. Journals are now used for more than dissemination of scientific research. Institutions use journal rankings for a variety of purposes and often require a pre‐established number of articles in hiring and budgetary decisions. Consequently, journal impact factors have achieved greater importance, and the splitting of articles into smaller parcels of information (‘salami‐slicing’) to increase numbers of publications has become more frequent. 3. Journals may prescribe upper limits to article length, even though the average length of articles for several ecological journals examined has increased over time. There are clear signs, however, that journals without length limits for articles will become rarer. In contrast to ecological journals, taxonomic journals are not following this trend. 4. Two case histories demonstrate how splitting longer ecological articles into a series of shorter ones results in both redundancy of information and actually increases the journal space used overall. Furthermore, with current rejection rates of ecological journals (often ～80%), many thin salami‐sliced articles jam the peer‐review system much longer (through resubmission after rejection) than unsliced articles previously did (e.g. when rejection rates were ～50%). In our experience, the increased pressure to publish many articles in ‘high‐impact’ journals also may decrease the attractiveness of a future scientific career in ecology to young people. 5. ‘Gatekeeping’ of journal quality has shifted from editors to reviewers, and several recent trends are apparent including: bias about appropriate statistical methods; reviewers being more rigid overall; non‐native English writers being criticised for poor communications skills; and favourable reviews being signed more often than unfavourable ones. In terms of production, outsourcing of copy editing has increased the final error rate of published material. 6. We supplemented our perceptions with those of older colleagues (～100 experienced ecologists) that responded to an informal survey on this topic (response rate: 81%). In the opinion of almost 90% of our respondents, the overall review process has changed and for 20% among them the professional quality of reviews has declined. 7. We, and many older colleagues, are convinced there have been some negative changes in the scientific publication process. If younger colleagues share this concern, we can collectively counter this deteriorating situation, because we are the key to the publishing and evaluation process.     

Gender diversity of editorial boards and gender differences in the peer review process at six journals of ecology and evolution

Despite substantial progress for women in science, women remain underrepresented in many aspects of the scholarly publication process. We examined how the gender diversity of editors and reviewers changed over time for six journals in ecology and evolution (2003–2015 for four journals, 2007–2015 or 2009–2015 for the other two), and how several aspects of the peer review process differed between female and male editors and reviewers. We found that for five of the six journals, women were either absent or very poorly represented as handling editors at the beginning of our dataset. The representation of women increased gradually and consistently, with women making up 29% of the handling editors (averaged across journals) in 2015, similar to the representation of women as last authors on ecology papers (23% in 2015) but lower than the proportion of women among all authors (31%) and among members of the societies that own the journals (37%–40%). The proportion of women among reviewers has also gradually but consistently increased over time, reaching 27% by 2015. Female editors invited more female reviewers than did male editors, and this difference increased with age of the editor. Men and women who were invited to review did not differ in whether they responded to the review invitation, but, of those that responded, women were slightly more likely to agree to review. In contrast, women were less likely than men to accept invitations to serve on journal editorial boards. Our analyses indicate that there has been progress in the representation of women as reviewers and editors in ecology and evolutionary biology, but women are still underrepresented among the gatekeepers of scholarly publishing relative to their representation among researchers.     

The validity of peer review in a general medicine journal

All the opinions in this article are those of the authors and should not be construed to reflect, in any way, those of the Department of Veterans Affairs.

Background

Our study purpose was to assess the predictive validity of reviewer quality ratings and editorial decisions in a general medicine journal.

Methods

Submissions to the Journal of General Internal Medicine (JGIM) between July 2004 and June 2005 were included. We abstracted JGIM peer review quality ratings, verified the publication status of all articles and calculated an impact factor for published articles (Rw) by dividing the 3-year citation rate by the average for this group of papers; an Rw>1 indicates a greater than average impact.

Results

Of 507 submissions, 128 (25%) were published in JGIM, 331 rejected (128 with review) and 48 were either not resubmitted after revision was requested or were withdrawn by the author. Of 331 rejections, 243 were published elsewhere. Articles published in JGIM had a higher citation rate than those published elsewhere (Rw: 1.6 vs. 1.1, p = 0.002). Reviewer quality ratings of article quality had good internal consistency and reviewer recommendations markedly influenced publication decisions. There was no quality rating cutpoint that accurately distinguished high from low impact articles. There was a stepwise increase in Rw for articles rejected without review, rejected after review or accepted by JGIM (Rw 0.60 vs. 0.87 vs. 1.56, p<0.0005). However, there was low agreement between reviewers for quality ratings and publication recommendations. The editorial publication decision accurately discriminated high and low impact articles in 68% of submissions. We found evidence of better accuracy with a greater number of reviewers.

Conclusions

The peer review process largely succeeds in selecting high impact articles and dispatching lower impact ones, but the process is far from perfect. While the inter-rater reliability between individual reviewers is low, the accuracy of sorting is improved with a greater number of reviewers.     

Next‐generation field courses: Integrating Open Science and online learning

As Open Science practices become more commonplace, there is a need for the next generation of scientists to be well versed in these aspects of scientific research. Yet, many training opportunities for early career researchers (ECRs) could better emphasize or integrate Open Science elements. Field courses provide opportunities for ECRs to apply theoretical knowledge, practice new methodological approaches, and gain an appreciation for the challenges of real‐life research, and could provide an excellent platform for integrating training in Open Science practices. Our recent experience, as primarily ECRs engaged in a field course interrupted by COVID‐19, led us to reflect on the potential to enhance learning outcomes in field courses by integrating Open Science practices and online learning components. Specifically, we highlight the opportunity for field courses to align teaching activities with the recent developments and trends in how we conduct research, including training in: publishing registered reports, collecting data using standardized methods, adopting high‐quality data documentation, managing data through reproducible workflows, and sharing and publishing data through appropriate channels. We also discuss how field courses can use online tools to optimize time in the field, develop open access resources, and cultivate collaborations. By integrating these elements, we suggest that the next generation of field courses will offer excellent arenas for participants to adopt Open Science practices.     

Impact factor statistics and publication practice: What can we learn?

Peer review procedures and citation statistics are important yet often neglected components of the scientific publication process. Here I discuss fundamental consequences of such quality measures for the scientific community and propose three remedial actions: (1) use of a ‘‘Combined Impact Estimate’’ as a measure of citation statistics, (2) adoption of an open reviewing policy and (3) acceleration of the publication process in order to raise the reputation of the entire discipline (in our case: behavioural science). Authors, reviewers and editors are invited to contribute to the improvement of publication practice.     

Prion reviewer acknowledgments

Prion is grateful for the ongoing support of its peer reviewers, who ensure that the submissions accepted for publication in Prion continue to be of the highest standard. We very much appreciate their time and the thoughtful reviews they provide. We would like to thank the following peer reviewers for their assistance in 2013:     

Systematic reviews and meta-analyses of preclinical studies: publication bias in laboratory animal experiments

In 2006, Peters et al. identified 86 systematic reviews (SRs) of laboratory animal experiments (LAEs). They found 46 LAE meta-analyses (MAs), often of poor quality. Six of these 46 MAs tried to assess publication bias. Publication bias is the phenomenon of an experiment's results determining its likelihood of publication, often over-representing positive findings. As such, publication bias is the Achilles heel of any SR. Since researchers increasingly become aware of the fact that SRs directly support the 'three Rs', we expect the number of SRs of LAEs will sharply increase. Therefore, it is useful to see how publication bias is dealt with. Our objective was to identify all SRs and MAs of LAEs where the purpose was to inform human health published between July 2005 and 2010 with special attention to MAs' quality features and publication bias. We systematically searched Medline, Embase, Toxline and ScienceDirect from July 2005 to 2010, updating Peters' review. LAEs not directly informing human health or concerning fundamental biology were excluded. We found 2780 references of which 163 met the inclusion criteria: 158 SRs, of which 30 performed an MA, and five MAs without an SR. The number of SRs roughly doubled every three years since 1997. The number of MAs roughly doubled every five years since 1999. Compared with before July 2005, more MAs were preceded by SR and reported on (quality) features of included studies and heterogeneity. A statistically significant proportion of MAs considered publication bias (26/35) and tried to formally assess it (21/35).     

MrEnt: an editor for publication‐quality phylogenetic tree illustrations

We developed MrEnt, a Windows‐based, user‐friendly software that allows the production of complex, high‐resolution, publication‐quality phylogenetic trees in few steps, directly from the analysis output. The program recognizes the standard Nexus tree format and the annotated tree files produced by BEAST and MrBayes. MrEnt combines in a single software a large suite of tree manipulation functions (e.g. handling of multiple trees, tree rotation, character mapping, node collapsing, compression of large clades, handling of time scale and error bars for chronograms) with drawing tools typical of standard graphic editors, including handling of graphic elements and images. The tree illustration can be printed or exported in several standard formats suitable for journal publication, PowerPoint presentation or Web publication.     

Quantifying shifts in topic popularity over 44 years of Austral Ecology

The Ecological Society of Australia was founded in 1959, and the society’s journal was first published in 1976. To examine how research published in the society’s journal has changed over this time, we used text mining to quantify themes and trends in the body of work published by the Australian Journal of Ecology and Austral Ecology from 1976 to 2019. We used topic models to identify 30 ‘topics’ within 2778 full‐text articles in 246 issues of the journal, followed by mixed modelling to identify topics with above‐average or below‐average popularity in terms of the number of publications or citations that they contain. We found high inter‐decadal turnover in research topics, with an early emphasis on highly specific ecosystems or processes giving way to a modern emphasis on community, spatial and fire ecology, invasive species and statistical modelling. Despite an early focus on Australian research, papers discussing South American ecosystems are now among the fastest‐growing and most frequently cited topics in the journal. Topics that were growing fastest in publication rates were not always the same as those with high citation rates. Our results provide a systematic breakdown of the topics that Austral Ecology authors and editors have chosen to research, publish and cite through time, providing a valuable window into the historical and emerging foci of the journal.