Integrative Approach to Quality Assessment of Medical Journals Using Impact Factor,Eigenfactor, and Article Influence Scores

Background

Impact factor (IF) is a commonly used surrogate for assessing the scientific quality of journals and articles. There is growing discontent in the medical community with the use of this quality assessment tool because of its many inherent limitations. To help address such concerns, Eigenfactor (ES) and Article Influence scores (AIS) have been devised to assess scientific impact of journals. The principal aim was to compare the temporal trends in IF, ES, and AIS on the rank order of leading medical journals over time.

Methods

The 2001 to 2008 IF, ES, AIS, and number of citable items (CI) of 35 leading medical journals were collected from the Institute of Scientific Information (ISI) and the http://www.eigenfactor.org databases. The journals were ranked based on the published 2008 ES, AIS, and IF scores. Temporal score trends and variations were analyzed.

Results

In general, the AIS and IF values provided similar rank orders. Using ES values resulted in large changes in the rank orders with higher ranking being assigned to journals that publish a large volume of articles. Since 2001, the IF and AIS of most journals increased significantly; however the ES increased in only 51% of the journals in the analysis. Conversely, 26% of journals experienced a downward trend in their ES, while the rest experienced no significant changes (23%). This discordance between temporal trends in IF and ES was largely driven by temporal changes in the number of CI published by the journals.

Conclusion

The rank order of medical journals changes depending on whether IF, AIS or ES is used. All of these metrics are sensitive to the number of citable items published by journals. Consumers should thus consider all of these metrics rather than just IF alone in assessing the influence and importance of medical journals in their respective disciplines.     

Impact of Article Language in Multi-Language Medical Journals - a Bibliometric Analysis of Self-Citations and Impact Factor

Background

In times of globalization there is an increasing use of English in the medical literature. The aim of this study was to analyze the influence of English-language articles in multi-language medical journals on their international recognition – as measured by a lower rate of self-citations and higher impact factor (IF).

Methods and Findings

We analyzed publications in multi-language journals in 2008 and 2009 using the Web of Science (WoS) of Thomson Reuters (former Institute of Scientific Information) and PubMed as sources of information. The proportion of English-language articles during the period was compared with both the share of self-citations in the year 2010 and the IF with and without self-citations. Multivariable linear regression analysis was performed to analyze these factors as well as the influence of the journals‘ countries of origin and of the other language(s) used in publications besides English.We identified 168 multi-language journals that were listed in WoS as well as in PubMed and met our criteria. We found a significant positive correlation of the share of English articles in 2008 and 2009 with the IF calculated without self-citations (Pearson r=0.56, p = <0.0001), a correlation with the overall IF (Pearson r = 0.47, p = <0.0001) and with the cites to years of IF calculation (Pearson r = 0.34, p = <0.0001), and a weak negative correlation with the share of self-citations (Pearson r = -0.2, p = 0.009). The IF without self-citations also correlated with the journal‘s country of origin – North American journals had a higher IF compared to Middle and South American or European journals.

Conclusion

Our findings suggest that a larger share of English articles in multi-language medical journals is associated with greater international recognition. Fewer self-citations were found in multi-language journals with a greater share of original articles in English.     

Representation of cancer in the medical literature--a bibliometric analysis

Background

There exists a lack of knowledge regarding the quantity and quality of scientific yield in relation to individual cancer types. We aimed to measure the proportion, quality and relevance of oncology-related articles, and to relate this output to their associated disease burden. By incorporating the impact factor(IF) and Eigenfactor™(EF) into our analysis we also assessed the relationship between these indices and the output under study.

Methods

All publications in 2007 were retrieved for the 26 most common cancers. The top 20 journals ranked by IF and EF in general medicine and oncology, and the presence of each malignancy within these titles was analysed. Journals publishing most prolifically on each cancer were identified and their impact assessed.

Principal Findings

63260 (PubMed) and 126845 (WoS) entries were generated, respectively. 26 neoplasms accounted for 25% of total output from the top medical publications. 5 cancers dominated the first quartile of output in the top oncology journals; breast, prostate, lung, and intestinal cancer, and leukaemia. Journals associated with these cancers were associated with much higher IFs and EFs than those journals associated with the other cancer types under study, although these measures were not equivalent across all sub-specialties. In addition, yield on each cancer was related to its disease burden as measured by its incidence and prevalence.

Conclusions

Oncology enjoys disproportionate representation in the more prestigious medical journals. 5 cancers dominate yield, although this attention is justified given their associated disease burden. The commonly used IF and the recently introduced EF do not correlate in the assessment of the preeminent oncology journals, nor at the level of individual malignancies; there is a need to delineate between proxy measures of quality and the relevance of output when assessing its merit. These results raise significant questions regarding the best method of assessment of research and scientific output in the field of oncology.     

Comparative study of scientific publications in urology and nephrology journals originating from USA, china and Japan (2001-2010)

Background

In the past decade, scientific research has developed rapidly in China, but the growth seems to vary widely between different disciplines. In this study, we aimed to compare the quantity and quality of publications in urology and nephrology journals from USA, China and Japan.

Methods

Journals listed in the “Urology and Nephrology” category of Science Citation Index Expanded subject categories were included. Scientific papers in these journals written by researchers from USA, Japan and China were retrieved from the “PubMed” and “Web of Knowledge” online databases.

Results

The annual number of total scientific articles increased significantly from 2001 to 2010 in China, and has ranked second in the world since 2006. In the field of urology and nephrology, the annual number increased significantly from 2001 to 2010 in USA and China; but not in Japan. The share of articles increased significantly over time in China, decreased significantly in Japan, and remained unchanged in USA. In 2010, USA contributed 32.17% of the total world output in urology and nephrology field and ranked 1^st; Japan contributed 5.19% and ranked 5^th; China contributed 3.83% and ranked 9^th. Publications from USA had the highest accumulated IFs and the highest total citations of articles (USA>Japan>China, p<0.001). No significant difference was found in average IF among the three countries. USA published the most articles in the top 10 urology and nephrology journals (USA(35165)>Japan(6704)>China(2233), p<0.001). Researchers from USA published more clinical trials and randomized controlled trials than Japan and China (USA>Japan>China, p<0.001).

Conclusion

Although China has undergone significant increase in annual number and percentage of scientific publication in urology and nephrology journals in the past decade, it still lags far behind USA and Japan in the field of urology and nephrology in terms of quantity and quality.     

Articles by Latin American Authors in Prestigious Journals Have Fewer Citations

Background

The journal Impact factor (IF) is generally accepted to be a good measurement of the relevance/quality of articles that a journal publishes. In spite of an, apparently, homogenous peer-review process for a given journal, we hypothesize that the country affiliation of authors from developing Latin American (LA) countries affects the IF of a journal detrimentally.

Methodology/Principal Findings

Seven prestigious international journals, one multidisciplinary journal and six serving specific branches of science, were examined in terms of their IF in the Web of Science. Two subsets of each journal were then selected to evaluate the influence of author''s affiliation on the IF. They comprised contributions (i) with authorship from four Latin American (LA) countries (Argentina, Brazil, Chile and Mexico) and (ii) with authorship from five developed countries (England, France, Germany, Japan and USA). Both subsets were further subdivided into two groups: articles with authorship from one country only and collaborative articles with authorship from other countries. Articles from the five developed countries had IF close to the overall IF of the journals and the influence of collaboration on this value was minor. In the case of LA articles the effect of collaboration (virtually all with developed countries) was significant. The IFs for non-collaborative articles averaged 66% of the overall IF of the journals whereas the articles in collaboration raised the IFs to values close to the overall IF.

Conclusion/Significance

The study shows a significantly lower IF in the group of the subsets of non-collaborative LA articles and thus that country affiliation of authors from non-developed LA countries does affect the IF of a journal detrimentally. There are no data to indicate whether the lower IFs of LA articles were due to their inherent inferior quality/relevance or psycho-social trend towards under-citation of articles from these countries. However, further study is required since there are foreseeable consequences of this trend as it may stimulate strategies by editors to turn down articles that tend to be under-cited.     

Why Has the Number of Scientific Retractions Increased?

Background

The number of retracted scientific publications has risen sharply, but it is unclear whether this reflects an increase in publication of flawed articles or an increase in the rate at which flawed articles are withdrawn.

Methods and Findings

We examined the interval between publication and retraction for 2,047 retracted articles indexed in PubMed. Time-to-retraction (from publication of article to publication of retraction) averaged 32.91 months. Among 714 retracted articles published in or before 2002, retraction required 49.82 months; among 1,333 retracted articles published after 2002, retraction required 23.82 months (p<0.0001). This suggests that journals are retracting papers more quickly than in the past, although recent articles requiring retraction may not have been recognized yet. To test the hypothesis that time-to-retraction is shorter for articles that receive careful scrutiny, time-to-retraction was correlated with journal impact factor (IF). Time-to-retraction was significantly shorter for high-IF journals, but only ∼1% of the variance in time-to-retraction was explained by increased scrutiny. The first article retracted for plagiarism was published in 1979 and the first for duplicate publication in 1990, showing that articles are now retracted for reasons not cited in the past. The proportional impact of authors with multiple retractions was greater in 1972–1992 than in the current era (p<0.001). From 1972–1992, 46.0% of retracted papers were written by authors with a single retraction; from 1993 to 2012, 63.1% of retracted papers were written by single-retraction authors (p<0.001).

Conclusions

The increase in retracted articles appears to reflect changes in the behavior of both authors and institutions. Lower barriers to publication of flawed articles are seen in the increase in number and proportion of retractions by authors with a single retraction. Lower barriers to retraction are apparent in an increase in retraction for “new” offenses such as plagiarism and a decrease in the time-to-retraction of flawed work.     

Trial Registration Numbers Are Underreported in Biomedical Publications

Context

Since September 2005, the International Committee of Medical Journal Editors (ICMJE) has required that randomised controlled trials (RCTs) are prospectively registered in a publicly accessible database. After registration, a trial registration number (TRN) is assigned to each RCT, which should make it easier to identify future publications and cross-check published results with associated registry entries, as long as the unique identification number is reported in the article.

Objective

Our primary objective was to evaluate the reporting of trial registration numbers in biomedical publications. Secondary objectives were to evaluate how many published RCTs had been registered and how many registered RCTs had resulted in a publication, using a sample of trials from the Netherlands Trials Register (NTR).

Design, Setting

Two different samples of RCTs were examined: 1) RCTs published in November 2010 in core clinical journals identified in MEDLINE; 2) RCTs registered in the NTR with a latest expected end date of 31 August 2008.

Results

Fifty-five percent (166/302) of the reports of RCTs found in MEDLINE and 60% (186/312) of the published reports of RCTs from the NTR cohort contained a TRN. In both samples, reporting of a TRN was more likely in RCTs published in ICMJE member journals as compared to non-ICMJE member journals (MEDLINE 58% vs. 45%; NTR: 70% vs. 49%). Thirty-nine percent of published RCTs in the MEDLINE sample appear not to have been registered, and 48% of RCTs registered in the NTR seemed not to have been published at least two years after the expected date for study completion.

Conclusion

Our results show that further promotion and implementation of trial registration and accurate reporting of TRN is still needed. This might be helped by inclusion of the TRN as an item on the CONSORT checklist.     

Why Most Biomedical Findings Echoed by Newspapers Turn Out to be False: The Case of Attention Deficit Hyperactivity Disorder

Context

Because positive biomedical observations are more often published than those reporting no effect, initial observations are often refuted or attenuated by subsequent studies.

Objective

To determine whether newspapers preferentially report on initial findings and whether they also report on subsequent studies.

Methods

We focused on attention deficit hyperactivity disorder (ADHD). Using Factiva and PubMed databases, we identified 47 scientific publications on ADHD published in the 1990s and soon echoed by 347 newspapers articles. We selected the ten most echoed publications and collected all their relevant subsequent studies until 2011. We checked whether findings reported in each “top 10” publication were consistent with previous and subsequent observations. We also compared the newspaper coverage of the “top 10” publications to that of their related scientific studies.

Results

Seven of the “top 10” publications were initial studies and the conclusions in six of them were either refuted or strongly attenuated subsequently. The seventh was not confirmed or refuted, but its main conclusion appears unlikely. Among the three “top 10” that were not initial studies, two were confirmed subsequently and the third was attenuated. The newspaper coverage of the “top 10” publications (223 articles) was much larger than that of the 67 related studies (57 articles). Moreover, only one of the latter newspaper articles reported that the corresponding “top 10” finding had been attenuated. The average impact factor of the scientific journals publishing studies echoed by newspapers (17.1 n = 56) was higher (p<0.0001) than that corresponding to related publications that were not echoed (6.4 n = 56).

Conclusion

Because newspapers preferentially echo initial ADHD findings appearing in prominent journals, they report on uncertain findings that are often refuted or attenuated by subsequent studies. If this media reporting bias generalizes to health sciences, it represents a major cause of distortion in health science communication.     

Equal Contributions and Credit: An Emerging Trend in the Characterization of Authorship in Major Anaesthesia Journals during a 10-Yr Period

Background

The practice of giving certain authors equal credit in original research publications was increasingly common in some specialty. This study aimed to investigate the prevalence and characteristics of designating some authors with equally credited authors (ECAs) in major anaesthesia journals.

Methodology/Principal Findings

The practice of giving authors equal credit was searched and identified in the three major anaesthesia journals between January 1, 2002 and December 31, 2011. Papers with ECAs had a higher proportion of the total number of articles in 2011 versus published in 2002 (Anesthesiology, 8.8% vs. 0.9%; British Journal of Anaesthesia, 8.8% vs. 0%; Anesthesia & Analgesia, 3.4% vs. 0.3%; totally, 6.4% vs. 0.4%). A significant increasing trend in annual proportion of articles with ECA was found in the three journals. The first two authors listed in the byline had equal credit in most cases.

Conclusions/Significance

The practice of giving authors equal credit in original research papers is increasingly common in major anaesthesia journals. It may be warranted for the journals to guide the authors how to regard this practice.     

Representation of Women and Pregnant Women in HIV Research: A Limited Systematic Review

Background

HIV-related outcomes may be affected by biological sex and by pregnancy. Including women in general and pregnant women in particular in HIV-related research is important for generalizability of findings.

Objective

To characterize representation of pregnant and non-pregnant women in HIV-related research conducted in general populations.

Data Sources

All HIV-related articles published in fifteen journals from January to March of 2011. We selected the top five journals by 2010 impact factor, in internal medicine, infectious diseases, and HIV/AIDS.

Study Eligibility Criteria

HIV-related studies reporting original research on questions applicable to both men and women of reproductive age were considered; studies were excluded if they did not include individual-level patient data.

Study appraisal and synthesis methods.

Articles were doubly reviewed and abstracted; discrepancies were resolved through consensus. We recorded proportion of female study participants, whether pregnant women were included or excluded, and other key factors.

Results

In total, 2014 articles were published during this period. After screening, 259 articles were included as original HIV-related research reporting individual-level data; of these, 226 were determined to be articles relevant to both men and women of reproductive age. In these articles, women were adequately represented within geographic region. The vast majority of published articles, 183/226 (81%), did not mention pregnancy (or related issues); still fewer included pregnant women (n=33), reported numbers of pregnant women (n=19), or analyzed using pregnancy status (n=9).

Limitations

Data were missing for some key variables, including pregnancy. The time period over which published works were evaluated was relatively short.

Conclusions and implications of key findings.

The under-reporting and inattention to pregnancy in the HIV literature may reduce policy-makers’ ability to set evidence-based policy around HIV/AIDS care for pregnant women and women of child-bearing age.     

Comparative genomics of methicillin-resistant Staphylococcus aureus ST239: distinct geographical variants in Beijing and Hong Kong

Background

The ST239 lineage is a globally disseminated, multiply drug-resistant hospital-associated methicillin-resistant Staphylococcus aureus (HA-MRSA). We performed whole-genome sequencing of representative HA-MRSA isolates of the ST239 lineage from bacteremic patients in hospitals in Hong Kong (HK) and Beijing (BJ) and compared them with three published complete genomes of ST239, namely T0131, TW20 and JKD6008. Orthologous gene group (OGG) analyses of the Hong Kong and Beijing cluster strains were also undertaken.

Results

Homology analysis, based on highest-percentage nucleotide identity, indicated that HK isolates were closely related to TW20, whereas BJ isolates were more closely related to T0131 from Tianjin. Phylogenetic analysis, incorporating a total of 30 isolates from different continents, revealed that strains from HK clustered with TW20 into the ‘Asian clade’, whereas BJ isolates and T0131 clustered closely with strains of the ‘Turkish clade’ from Eastern Europe. HK isolates contained the typical φSPβ-like prophage with the SasX gene similar to TW20. In contrast, BJ isolates contained a unique 15 kb PT1028-like prophage but lacked φSPβ-like and φSA1 prophages. Besides distinct mobile genetic elements (MGE) in the two clusters, OGG analyses and whole-genome alignment of these clusters highlighted differences in genes located in the core genome, including the identification of single nucleotide deletions in several genes, resulting in frameshift mutations and the subsequent predicted truncation of encoded proteins involved in metabolism and antimicrobial resistance.

Conclusions

Comparative genomics, based on de novo assembly and deep sequencing of HK and BJ strains, revealed different origins of the ST239 lineage in northern and southern China and identified differences between the two clades at single nucleotide polymorphism (SNP), core gene and MGE levels. The results suggest that ST239 strains isolated in Hong Kong since the 1990s belong to the Asian clade, present mainly in southern Asia, whereas those that emerged in northern China were of a distinct origin, reflecting the complexity of dissemination and the dynamic evolution of this ST239 lineage.

Electronic supplementary material

The online version of this article (doi:10.1186/1471-2164-15-529) contains supplementary material, which is available to authorized users.     

Scientific publications on primary biliary cirrhosis from 2000 through 2010: an 11-year survey of the literature

Background

Primary biliary cirrhosis (PBC) is a chronic liver disease characterized by intrahepatic bile-duct destruction, cholestasis, and fibrosis. It can lead to cirrhosis and eventually liver failure. PBC also shows some regional differences with respect to incidence and prevalence that are becoming more pronounced each year. Recently, researchers have paid more attention to PBC. To evaluate the development of PBC research during the past 11 years, we determined the quantity and quality of articles on this subject. We also compared the contributions of scientists from the US, UK, Japan, Italy, Germany, and China.

Methods

The English-language papers covering PBC published in journals from 2000 through 2010 were retrieved from the PubMed database. We recorded the number of papers published each year, analyzed the publication type, and calculated the accumulated, average impact factors (IFs) and citations from every country. The quantity and quality of articles on PBC were compared by country. We also contrasted the level of PBC research in China and other countries.

Results

The total number of articles did not significantly increase during the past 11 years. The number of articles from the US exceeded those from any other country; the publications from the US also had the highest IFs and the most citations. Four other countries showed complex trends with respect to the quantity and quality of articles about PBC.

Conclusion

The researchers from the US have contributed the most to the development of PBC research. They currently represent the highest level of research. Some high-level studies, such as RCTs, meta-analyses, and in-depth basic studies should be launched. The gap between China and the advanced level is still enormous. Chinese investigators still have a long way to go.     

Temporal trends in the impact factor of European versus USA biomedical journals

Background

The impact factors of biomedical journals tend to rise over time. We sought to assess the trend in the impact factor, during the past decade, of journals published on behalf of United States (US) and European scientific societies, in four select biomedical subject categories (Biology, Cell Biology, Critical Care Medicine, and Infectious Diseases).

Methods

We identified all journals included in the above-mentioned subject categories of Thomson Reuters Journal Citation Reports® for the years 1999, 2002, 2005, and 2008. We selected those that were published on behalf of US or European scientific societies, as documented in journal websites.

Results

We included 167 journals (35 in the subject category of Biology, 79 in Cell Biology, 27 in Critical Care Medicine, and 26 in Infectious Diseases). Between 1999 and 2008, the percentage increase in the impact factor of the European journals was higher than for the US journals (73.7±110.0% compared with 39.7±70.0%, p = 0.049). Regarding specific subject categories, the percentage change in the factor of the European journals tended to be higher than the respective US journals for Cell Biology (61.7% versus 16.3%), Critical Care Medicine (212.4% versus 65.4%), Infectious Diseases (88.3% versus 48.7%), whereas the opposite was observed for journals in Biology (41.0% versus 62.5%).

Conclusion

Journals published on behalf of European scientific societies, in select biomedical fields, may tend to close the “gap” in impact factor compared with those of US societies.

What''s Already Known About This Topic?

The impact factors of biomedical journals tend to rise through years. The leading positions in productivity in biomedical research are held by developed countries, including those from North America and Western Europe.

What Does This Article Add?

The journals from European biomedical scientific societies tended, over the past decade, to increase their impact factor more than the respective US journals.     

A Comprehensive Survey of Retracted Articles from the Scholarly Literature

Background

The number of retracted scholarly articles has risen precipitously in recent years. Past surveys of the retracted literature each limited their scope to articles in PubMed, though many retracted articles are not indexed in PubMed. To understand the scope and characteristics of retracted articles across the full spectrum of scholarly disciplines, we surveyed 42 of the largest bibliographic databases for major scholarly fields and publisher websites to identify retracted articles. This study examines various trends among them.

Results

We found, 4,449 scholarly publications retracted from 1928–2011. Unlike Math, Physics, Engineering and Social Sciences, the percentages of retractions in Medicine, Life Science and Chemistry exceeded their percentages among Web of Science (WoS) records. Retractions due to alleged publishing misconduct (47%) outnumbered those due to alleged research misconduct (20%) or questionable data/interpretations (42%). This total exceeds 100% since multiple justifications were listed in some retraction notices. Retraction/WoS record ratios vary among author affiliation countries. Though widespread, only miniscule percentages of publications for individual years, countries, journals, or disciplines have been retracted. Fifteen prolific individuals accounted for more than half of all retractions due to alleged research misconduct, and strongly influenced all retraction characteristics. The number of articles retracted per year increased by a factor of 19.06 from 2001 to 2010, though excluding repeat offenders and adjusting for growth of the published literature decreases it to a factor of 11.36.

Conclusions

Retracted articles occur across the full spectrum of scholarly disciplines. Most retracted articles do not contain flawed data; and the authors of most retracted articles have not been accused of research misconduct. Despite recent increases, the proportion of published scholarly literature affected by retraction remains very small. Articles and editorials discussing retractions, or their relation to research integrity, should always consider individual cases in these broad contexts. However, better mechanisms are still needed for raising researchers’ awareness of the retracted literature in their field.     

Neglected Tropical Diseases: A Systematic Evaluation of Research Capacity in Nigeria

Background

Nigeria carries the highest burden and diversity of neglected tropical diseases (NTDs) in sub-Saharan Africa and is preparing to scale up its efforts to control/eliminate these diseases. To achieve this it will require a range of internal technical support and expertise for mapping, monitoring and evaluating, operational research and documenting its success. In order to begin to evaluate this potential in Nigeria, this study collated and analysed information for lymphatic filariasis (LF), onchocerciasis, schistosomiasis and soil-transmitted helminths (STH), which are currently being targeted with preventive chemotherapy through mass drug administration (MDA).

Methodology/Principal Findings

Information from 299 scientific articles published on the selected NTDs in 179 journals between January 2008 and September 2013 was extracted and systematically compiled into a geo-referenced database for analysis and mapping. The highest number of articles was from the southern geo-political zones of the country. The majority of articles focused on one specific disease, and schistosomiasis and STH were found to have the highest and most wide ranging research output. The main type of study was parasitological, and the least was biotechnological. Nigerian authors were mostly affiliated with universities, and there was a wide range of international co-authors from Africa and other regions, especially the USA and UK. The majority of articles were published in journals with no known impact factor.

Conclusions/Significance

The extensive database and series of maps on the research capacity within Nigeria produced in this study highlights the current potential that exists, and needs to be fully maximized for the control/elimination of NTDs in the country. This study provides an important model approach that can be applied to other low and middle income countries where NTDs are endemic, and NTD programmes require support from the expertise within their own country, as well as internationally, to help raise their profile and importance.     

All That Glitters Isn't Gold: A Survey on Acknowledgment of Limitations in Biomedical Studies

Background

Acknowledgment of all serious limitations to research evidence is important for patient care and scientific progress. Formal research on how biomedical authors acknowledge limitations is scarce.

Objectives

To assess the extent to which limitations are acknowledged in biomedical publications explicitly, and implicitly by investigating the use of phrases that express uncertainty, so-called hedges; to assess the association between industry support and the extent of hedging.

Design

We analyzed reporting of limitations and use of hedges in 300 biomedical publications published in 30 high and medium -ranked journals in 2007. Hedges were assessed using linguistic software that assigned weights between 1 and 5 to each expression of uncertainty.

Results

Twenty-seven percent of publications (81/300) did not mention any limitations, while 73% acknowledged a median of 3 (range 1–8) limitations. Five percent mentioned a limitation in the abstract. After controlling for confounders, publications on industry-supported studies used significantly fewer hedges than publications not so supported (p = 0.028).

Limitations

Detection and classification of limitations was – to some extent – subjective. The weighting scheme used by the hedging detection software has subjective elements.

Conclusions

Reporting of limitations in biomedical publications is probably very incomplete. Transparent reporting of limitations may protect clinicians and guideline committees against overly confident beliefs and decisions and support scientific progress through better design, conduct or analysis of new studies.     

A tale of four stories: soil ecology, theory, evolution and the publication system

Background

Soil ecology has produced a huge corpus of results on relations between soil organisms, ecosystem processes controlled by these organisms and links between belowground and aboveground processes. However, some soil scientists think that soil ecology is short of modelling and evolutionary approaches and has developed too independently from general ecology. We have tested quantitatively these hypotheses through a bibliographic study (about 23000 articles) comparing soil ecology journals, generalist ecology journals, evolutionary ecology journals and theoretical ecology journals.

Findings

We have shown that soil ecology is not well represented in generalist ecology journals and that soil ecologists poorly use modelling and evolutionary approaches. Moreover, the articles published by a typical soil ecology journal (Soil Biology and Biochemistry) are cited by and cite low percentages of articles published in generalist ecology journals, evolutionary ecology journals and theoretical ecology journals.

Conclusion

This confirms our hypotheses and suggests that soil ecology would benefit from an effort towards modelling and evolutionary approaches. This effort should promote the building of a general conceptual framework for soil ecology and bridges between soil ecology and general ecology. We give some historical reasons for the parsimonious use of modelling and evolutionary approaches by soil ecologists. We finally suggest that a publication system that classifies journals according to their Impact Factors and their level of generality is probably inadequate to integrate “particularity” (empirical observations) and “generality” (general theories), which is the goal of all natural sciences. Such a system might also be particularly detrimental to the development of a science such as ecology that is intrinsically multidisciplinary.