Systematic Differences in Impact across Publication Tracks at PNAS

Background

Citation data can be used to evaluate the editorial policies and procedures of scientific journals. Here we investigate citation counts for the three different publication tracks of the Proceedings of the National Academy of Sciences of the United States of America (PNAS). This analysis explores the consequences of differences in editor and referee selection, while controlling for the prestige of the journal in which the papers appear.

Methodology/Principal Findings

We find that papers authored and “Contributed” by NAS members (Track III) are on average cited less often than papers that are “Communicated” for others by NAS members (Track I) or submitted directly via the standard peer review process (Track II). However, we also find that the variance in the citation count of Contributed papers, and to a lesser extent Communicated papers, is larger than for direct submissions. Therefore when examining the 10% most-cited papers from each track, Contributed papers receive the most citations, followed by Communicated papers, while Direct submissions receive the least citations.

Conclusion/Significance

Our findings suggest that PNAS “Contributed” papers, in which NAS–member authors select their own reviewers, balance an overall lower impact with an increased probability of publishing exceptional papers. This analysis demonstrates that different editorial procedures are associated with different levels of impact, even within the same prominent journal, and raises interesting questions about the most appropriate metrics for judging an editorial policy''s success.     

Saline Systems highlights for 2005

On the 4^th of July, 2005, the Saline Systems editorial group launched the new online open access journal, Saline Systems, with BioMed Central as the publisher. The scope of the journal includes both basic and applied research on halophilic organisms and saline environments, from gene systems to ecosystems. The stated goal of the journal is to meet publication needs for researchers working in coastal and inland saline environments and provide an interdisciplinary and readily accessible forum for scientists worldwide. The inaugural volume of the journal contains a significant number of high quality original research papers and reviews on a wide range of relevant topics. At the end of the launch period, from January 1, 2006 onwards, the journal will be introducing article-processing charges to cover the cost of publication. Charges will be partly or completely waived for authors from BioMed Central institutional subscribers and in cases of financial hardship.     

Jane goodall and the chimpanzees of gombe: An analysis of publications and their impact on teaching science

Five types of introductory university textbooks (N=37) were analyzed for references and citations pertaining to research on wild chimpanzees. Jane Goodall's publications were cited about three times as often as the publications from field sites other than Gombe and approximately five times more often than other Gombe researchers. Biological anthropology textbooks cited Goodall's work most often, followed by textbooks in general anthropology and cultural anthropology. Psychology and biology textbooks cited Goodall least often. Goodall's most comprehensive work, The Chimpanzees of Gombe: Patterns of Behavior (1986), was the most often cited publication about Gombe's apes, and tool-use was the most cited topic. The number of citations to wild chimpanzees tripled from publications in the 1960s to those in the 1980s, suggesting a growing recognition of primatology in the teaching of science.     

Our "Collegium Antropologicum" officially the most improved social science journal in the world for mid-2002

Thomson ISI's bimonthly web-product ISI Essential Science Indicators (ESI) is an in-depth analytical tool that regularly reports quantitative analyses of research performance and science trends, covering about 8,500 scientific journals from the entire world. In each issue ESI lists the scientists, institutions, countries and journals that are most improved from one update to the next, i.e. that show the largest percentage increase in total citations. In its edition of January 2003, it reported that our "Collegium Antropologicum" was the most improved journal in the field of Social Sciences during the period from July 2002 to September 2002. The field of Social Sciences is one of 22 categories of science regularly analyzed by ESI. It includes anthropology, public health, sociology, social work and policy, political science, law, education, communication, library and information sciences, environmental studies and rehabilitation. Due to journal's success, which is based on publications of predominantly Croatian scientists within the past seven post-war years, Croatia was also officially the most improved among more than 200 countries, and University of Zagreb was the most improved in the field of Social Science among thousands of other institutions. We hope that this is an early sign of revival of the scientific activity in our country after the War in Croatia (1991-1995).     

Citation Classics in Systematic Reviews and Meta-Analyses: Who Wrote the Top 100 Most Cited Articles?

Background

Systematic reviews of the literature occupy the highest position in currently proposed hierarchies of evidence. The aims of this study were to assess whether citation classics exist in published systematic review and meta-analysis (SRM), examine the characteristics of the most frequently cited SRM articles, and evaluate the contribution of different world regions.

Methods

The 100 most cited SRM were identified in October 2012 using the Science Citation Index database of the Institute for Scientific Information. Data were extracted by one author. Spearman’s correlation was used to assess the association between years since publication, numbers of authors, article length, journal impact factor, and average citations per year.

Results

Among the 100 citation classics, published between 1977 and 2008, the most cited article received 7308 citations and the least-cited 675 citations. The average citations per year ranged from 27.8 to 401.6. First authors from the USA produced the highest number of citation classics (n=46), followed by the UK (n=28) and Canada (n=15). The 100 articles were published in 42 journals led by the Journal of the American Medical Association (n=18), followed by the British Medical Journal (n=14) and The Lancet (n=13). There was a statistically significant positive correlation between number of authors (Spearman’s rho=0.320, p=0.001), journal impact factor (rho=0.240, p=0.016) and average citations per year. There was a statistically significant negative correlation between average citations per year and year since publication (rho = -0.636, p=0.0001). The most cited papers identified seminal contributions and originators of landmark methodological aspects of SRM and reflect major advances in the management of and predisposing factors for chronic diseases.

Conclusions

Since the late 1970s, the USA, UK, and Canada have taken leadership in the production of citation classic papers. No first author from low or middle-income countries (LMIC) led one of the most cited 100 SRM.     

科技学术期刊国际化发展的思考与初探

结合办刊实践,提出了如下措施和建议:(1)明确读者定位,开拓优质稿源新渠道。(2)通过策划选题,充分体现知识的创新性;设置特色栏目,栏目内容具有科学性和实用性;增加载文的信息量,提高信息密度等途径优化期刊内容,努力创办特色期刊。(3)通过强化政策导向,建立合理机制吸引优秀稿件;跟踪国际科技前沿课题,掌握学术团队的发展动态;重视学术会议组稿,加强引进和派出访问学者的联系等措施提高稿件学术质量,增强刊物核心竞争力。(4)建立国际交换关系,提升刊物自身价值。(5)加强与国际重要检索系统的联系,重视进入专业数据库。(6)加快信息化建设,加大期刊的对外宣传。以此提升期刊的国际影响力,促进期刊的国际化发展。     

Multiple Citation Indicators and Their Composite across Scientific Disciplines

Many fields face an increasing prevalence of multi-authorship, and this poses challenges in assessing citation metrics. Here, we explore multiple citation indicators that address total impact (number of citations, Hirsch H index [H]), co-authorship adjustment (Schreiber Hm index [Hm]), and author order (total citations to papers as single; single or first; or single, first, or last author). We demonstrate the correlation patterns between these indicators across 84,116 scientists (those among the top 30,000 for impact in a single year [2013] in at least one of these indicators) and separately across 12 scientific fields. Correlation patterns vary across these 12 fields. In physics, total citations are highly negatively correlated with indicators of co-authorship adjustment and of author order, while in other sciences the negative correlation is seen only for total citation impact and citations to papers as single author. We propose a composite score that sums standardized values of these six log-transformed indicators. Of the 1,000 top-ranked scientists with the composite score, only 322 are in the top 1,000 based on total citations. Many Nobel laureates and other extremely influential scientists rank among the top-1,000 with the composite indicator, but would rank much lower based on total citations. Conversely, many of the top 1,000 authors on total citations have had no single/first/last-authored cited paper. More Nobel laureates of 2011–2015 are among the top authors when authors are ranked by the composite score than by total citations, H index, or Hm index; 40/47 of these laureates are among the top 30,000 by at least one of the six indicators. We also explore the sensitivity of indicators to self-citation and alphabetic ordering of authors in papers across different scientific fields. Multiple indicators and their composite may give a more comprehensive picture of impact, although no citation indicator, single or composite, can be expected to select all the best scientists.     

I Publish in I Edit? - Do Editorial Board Members of Urologic Journals Preferentially Publish Their Own Scientific Work?

Scientists who are members of an editorial board have been accused of preferentially publishing their scientific work in the journal where they serve as editor. Reputation and academic standing do depend on an uninterrupted flow of published scientific work and the question does arise as to whether publication mainly occurs in the self-edited journal. This investigation was designed to determine whether editorial board members of five urological journals were more likely to publish their research reports in their own rather than in other journals. A retrospective analysis was conducted for all original reports published from 2001–2010 by 65 editorial board members nominated to the boards of five impact leading urologic journals in 2006. Publications before editorial board membership, 2001–2005, and publications within the period of time as an editorial board member, 2006–2010, were identified. The impact factors of the journals were also recorded over the time period 2001–2010 to see whether a change in impact factor correlated with publication locality. In the five journals as a whole, scientific work was not preferentially published in the journal in which the scientists served as editor. However, significant heterogeneity among the journals was evident. One journal showed a significant increase in the amount of published papers in the ‘own’ journal after assumption of editorship, three journals showed no change and one journal showed a highly significant decrease in publishing in the ‘own’ journal after assumption of editorship.     

Continuing problems with gray literature*

Frequent gatherings, such as those on coastal management, have resulted in increased production of gray literature like conference proceedings and institutional reports, which are published without adequate peer review. In developing countries like those in southeast Asia, manuals and other publications used in workshops and training programs seldom use peer-reviewed references. Among papers sampled, those in conference proceedings have lower percentages of citations to peer-reviewed journals, whether or not the proceedings are issued as books or journal supplements. From three proceedings and one institutional report with a total of 37 papers and an average of 22 cited references per paper, citations to gray literature averaged 92 percent of total citations. This poor quality of the reference lists decrease the credibility of a paper. Scientific conferences should be designed to reverse the production and use of gray literature by limiting the scope of the proceedings to invited reviews, with other presentations appearing only as abstracts to encourage their ultimate publication in peer-reviewed journals. A conference book of reviews by respected scientists will then support incorporation of scientific information into policy and management decisions for more effective coastal management.     

More topics from the tropics: additional thoughts to Mammides et al.

Most studies on tropical conservation questions are conducted by researchers of developed countries from the north. This geographic disconnection was recently criticised by Mammides et al. Here, we reflect on their findings and add further views from scientist’s and journal editor’s perspectives. We argue that journals are, a priori, most strongly interested in research questions and approaches that will likely increase their scientific impact and prestige. This is rarely compatible with publishing articles on questions with restricted global impact or based on single taxa. We question whether small changes in the editorial policy of international conservation journals will considerably improve the geographic diversity in key conservation publications. Rather, thematic scopes of the leading conservation journals should be modified, preferably in close collaboration with leading conservationists from the south. We are convinced that long-term investments in the tropics will create a stronger local scientific community, thus bolstering academic morale, and finally may lead to an increase in the submission and acceptance rate of articles written from scientists from these regions.     

Quantifying shifts in topic popularity over 44 years of Austral Ecology

The Ecological Society of Australia was founded in 1959, and the society’s journal was first published in 1976. To examine how research published in the society’s journal has changed over this time, we used text mining to quantify themes and trends in the body of work published by the Australian Journal of Ecology and Austral Ecology from 1976 to 2019. We used topic models to identify 30 ‘topics’ within 2778 full‐text articles in 246 issues of the journal, followed by mixed modelling to identify topics with above‐average or below‐average popularity in terms of the number of publications or citations that they contain. We found high inter‐decadal turnover in research topics, with an early emphasis on highly specific ecosystems or processes giving way to a modern emphasis on community, spatial and fire ecology, invasive species and statistical modelling. Despite an early focus on Australian research, papers discussing South American ecosystems are now among the fastest‐growing and most frequently cited topics in the journal. Topics that were growing fastest in publication rates were not always the same as those with high citation rates. Our results provide a systematic breakdown of the topics that Austral Ecology authors and editors have chosen to research, publish and cite through time, providing a valuable window into the historical and emerging foci of the journal.     

Content Factor: A Measure of a Journal's Contribution to Knowledge

Impact Factor, the pre-eminent performance metric for medical journals, has been criticized for failing to capture the true impact of articles; for favoring methodology papers; for being unduly influenced by statistical outliers; and for examining a period of time too short to capture an article's long-term importance. Also, in the era of search engines, where readers need not skim through journals to find information, Impact Factor's emphasis on citation efficiency may be misplaced. A better metric would consider the total number of citations to all papers published by the journal (not just the recent ones), and would not be decremented by the total number of papers published. We propose a metric embodying these principles, "Content Factor", and examine its performance among leading medical and orthopaedic surgery journals. To remedy Impact Factor's emphasis on recent citations, Content Factor considers the total number of citations, regardless of the year in which the cited paper was published. To correct for Impact Factor's emphasis on efficiency, no denominator is employed. Content Factor is thus the total number of citations in a given year to all of the papers previously published in the journal. We found that Content Factor and Impact Factor are poorly correlated. We further surveyed 75 experienced orthopaedic authors and measured their perceptions of the "importance" of various orthopaedic surgery journals. The correlation between the importance score and the Impact Factor was only 0.08; the correlation between the importance score and Content Factor was 0.56. Accordingly, Content Factor better reflects a journal's "importance". In sum, while Content Factor cannot be defended as the lone metric of merit, to the extent that performance data informs journal evaluations, Content Factor- an easily obtained and intuitively appealing metric of the journal's knowledge contribution, not subject to gaming- can be a useful adjunct.     

Measuring co-authorship and networking-adjusted scientific impact

Appraisal of the scientific impact of researchers, teams and institutions with productivity and citation metrics has major repercussions. Funding and promotion of individuals and survival of teams and institutions depend on publications and citations. In this competitive environment, the number of authors per paper is increasing and apparently some co-authors don't satisfy authorship criteria. Listing of individual contributions is still sporadic and also open to manipulation. Metrics are needed to measure the networking intensity for a single scientist or group of scientists accounting for patterns of co-authorship. Here, I define I(1) for a single scientist as the number of authors who appear in at least I(1) papers of the specific scientist. For a group of scientists or institution, I(n) is defined as the number of authors who appear in at least I(n) papers that bear the affiliation of the group or institution. I(1) depends on the number of papers authored N(p). The power exponent R of the relationship between I(1) and N(p) categorizes scientists as solitary (R>2.5), nuclear (R = 2.25-2.5), networked (R = 2-2.25), extensively networked (R = 1.75-2) or collaborators (R<1.75). R may be used to adjust for co-authorship networking the citation impact of a scientist. I(n) similarly provides a simple measure of the effective networking size to adjust the citation impact of groups or institutions. Empirical data are provided for single scientists and institutions for the proposed metrics. Cautious adoption of adjustments for co-authorship and networking in scientific appraisals may offer incentives for more accountable co-authorship behaviour in published articles.     

Nutritional studies in Croatia--a century of research

This paper is dedicated to the pioneers of nutritional research in Croatia: to Professor Edvin Ferber, Professor Hubert Maver and Professor Ratko Buzina, to whom we owe exceptional contribution in the development of science of nutrition, as well as for many scientific publications from the fifties to the eighties in the 20th century, leaving us great information about nutritional state in Croatian population. The paper brings a review of nuitritional research in Croatia with an emphasis on history of research and papers published in Collegium Antropologicum. Since first publications on the subject, a number of institutions and scholars participated in numerous research projects which resulted in a vast number of published papers, depicting a multidisciplinary approach to the subject. In addition, the results of 44 analyses that have been a part of doctoral (18) and master's research (26) are discussed.     

国际生物信息学研究的可视化分析

利用当前主流的信息可视化分析软件Cite Space对2005~2014年间SCI收录的生物信息学的5种高影响力外文期刊所刊载论文的题录数据进行统计和可视化分析,绘制该领域的关键词共现、膨胀词共现、经典文献共现、高被引文献共现和关键节点文献共现的网络可视化图谱,试图揭示生物信息学领域的研究热点、研究前沿以及知识基础,以期帮助研究人员了解该领域在国际范围内的研究态势。     

The Assessment of Science: The Relative Merits of Post-Publication Review,the Impact Factor,and the Number of Citations

The assessment of scientific publications is an integral part of the scientific process. Here we investigate three methods of assessing the merit of a scientific paper: subjective post-publication peer review, the number of citations gained by a paper, and the impact factor of the journal in which the article was published. We investigate these methods using two datasets in which subjective post-publication assessments of scientific publications have been made by experts. We find that there are moderate, but statistically significant, correlations between assessor scores, when two assessors have rated the same paper, and between assessor score and the number of citations a paper accrues. However, we show that assessor score depends strongly on the journal in which the paper is published, and that assessors tend to over-rate papers published in journals with high impact factors. If we control for this bias, we find that the correlation between assessor scores and between assessor score and the number of citations is weak, suggesting that scientists have little ability to judge either the intrinsic merit of a paper or its likely impact. We also show that the number of citations a paper receives is an extremely error-prone measure of scientific merit. Finally, we argue that the impact factor is likely to be a poor measure of merit, since it depends on subjective assessment. We conclude that the three measures of scientific merit considered here are poor; in particular subjective assessments are an error-prone, biased, and expensive method by which to assess merit. We argue that the impact factor may be the most satisfactory of the methods we have considered, since it is a form of pre-publication review. However, we emphasise that it is likely to be a very error-prone measure of merit that is qualitative, not quantitative.

Author summary

Subjective assessments of the merit and likely impact of scientific publications are routinely made by scientists during their own research, and as part of promotion, appointment, and government committees. Using two large datasets in which scientists have made qualitative assessments of scientific merit, we show that scientists are poor at judging scientific merit and the likely impact of a paper, and that their judgment is strongly influenced by the journal in which the paper is published. We also demonstrate that the number of citations a paper accumulates is a poor measure of merit and we argue that although it is likely to be poor, the impact factor, of the journal in which a paper is published, may be the best measure of scientific merit currently available.