The Evolution of Dental Journals from 2003 to 2012: A Bibliometric Analysis

Bibliometrics are a set of methods, which can be used to analyze academic literature quantitatively and its changes over time. The objectives of this study were 1) to evaluate trends related to academic performance of dental journals from 2003 to 2012 using bibliometric indices, and 2) monitor the changes of the five dental journals with the highest and lowest impact factor (IF) published in 2003. Data for the subject category "Dentistry, Oral Surgery & Medicine" was retrieved from the Journal Citation Reports (JCR) published from 2003 to 2012. Linear regressions analysis was used to determine statistical trends over the years with each bibliometric indicator as the dependent variable and the JCR year as the predictor variable. Statistically significant rise in the total number of dental journals, the number of all articles with the steepest rise observed for research articles, the number of citations and the aggregate IF was observed from 2003 to 2012. The analysis of the five top and five bottom-tire dental journals revealed a rise in IF however, with a wide variation in relation to the magnitude of this rise. Although the IF of the top five journals remained relatively constant, the percentile ranks of the four lowest ranking journals in 2003 increased significantly with the sharpest rise being noted for the British Journal of Oral & Maxillofacial Surgery. This study revealed significant growth of dental literature in absolute terms, as well as upward trends for most of the citation-based bibliometric indices from 2003 to 2012.     

Species Distribution Models and Impact Factor Growth in Environmental Journals: Methodological Fashion or the Attraction of Global Change Science

In this work, I evaluate the impact of species distribution models (SDMs) on the current status of environmental and ecological journals by asking the question to which degree development of SDMs in the literature is related to recent changes in the impact factors of ecological journals. The hypothesis evaluated states that research fronts are likely to attract research attention and potentially drive citation patterns, with journals concentrating papers related to the research front receiving more attention and benefiting from faster increases in their impact on the ecological literature. My results indicate a positive relationship between the number of SDM related articles published in a journal and its impact factor (IF) growth during the period 2000–09. However, the percentage of SDM related papers in a journal was strongly and positively associated with the percentage of papers on climate change and statistical issues. The results support the hypothesis that global change science has been critical in the development of SDMs and that interest in climate change research in particular, rather than the usage of SDM per se, appears as an important factor behind journal IF increases in ecology and environmental sciences. Finally, our results on SDM application in global change science support the view that scientific interest rather than methodological fashion appears to be the major driver of research attraction in the scientific literature.     

Prediction of citation counts for clinical articles at two years using data available within three weeks of publication: retrospective cohort study

Objective To determine if citation counts at two years could be predicted for clinical articles that pass basic criteria for critical appraisal using data within three weeks of publication from external sources and an online article rating service.Design Retrospective cohort study.Setting Online rating service, Canada.Participants 1274 articles from 105 journals published from January to June 2005, randomly divided into a 60:40 split to provide derivation and validation datasets.Main outcome measures 20 article and journal features, including ratings of clinical relevance and newsworthiness, routinely collected by the McMaster online rating of evidence system, compared with citation counts at two years.Results The derivation analysis showed that the regression equation accounted for 60% of the variation (R²=0.60, 95% confidence interval 0.538 to 0.629). This model applied to the validation dataset gave a similar prediction (R²=0.56, 0.476 to 0.596, shrinkage 0.04; shrinkage measures how well the derived equation matches data from the validation dataset). Cited articles in the top half and top third were predicted with 83% and 61% sensitivity and 72% and 82% specificity. Higher citations were predicted by indexing in numerous databases; number of authors; abstraction in synoptic journals; clinical relevance scores; number of cited references; and original, multicentred, and therapy articles from journals with a greater proportion of articles abstracted.Conclusion Citation counts can be reliably predicted at two years using data within three weeks of publication.     

Content Factor: A Measure of a Journal's Contribution to Knowledge

Impact Factor, the pre-eminent performance metric for medical journals, has been criticized for failing to capture the true impact of articles; for favoring methodology papers; for being unduly influenced by statistical outliers; and for examining a period of time too short to capture an article's long-term importance. Also, in the era of search engines, where readers need not skim through journals to find information, Impact Factor's emphasis on citation efficiency may be misplaced. A better metric would consider the total number of citations to all papers published by the journal (not just the recent ones), and would not be decremented by the total number of papers published. We propose a metric embodying these principles, "Content Factor", and examine its performance among leading medical and orthopaedic surgery journals. To remedy Impact Factor's emphasis on recent citations, Content Factor considers the total number of citations, regardless of the year in which the cited paper was published. To correct for Impact Factor's emphasis on efficiency, no denominator is employed. Content Factor is thus the total number of citations in a given year to all of the papers previously published in the journal. We found that Content Factor and Impact Factor are poorly correlated. We further surveyed 75 experienced orthopaedic authors and measured their perceptions of the "importance" of various orthopaedic surgery journals. The correlation between the importance score and the Impact Factor was only 0.08; the correlation between the importance score and Content Factor was 0.56. Accordingly, Content Factor better reflects a journal's "importance". In sum, while Content Factor cannot be defended as the lone metric of merit, to the extent that performance data informs journal evaluations, Content Factor- an easily obtained and intuitively appealing metric of the journal's knowledge contribution, not subject to gaming- can be a useful adjunct.     

Articles by Latin American Authors in Prestigious Journals Have Fewer Citations

Background

The journal Impact factor (IF) is generally accepted to be a good measurement of the relevance/quality of articles that a journal publishes. In spite of an, apparently, homogenous peer-review process for a given journal, we hypothesize that the country affiliation of authors from developing Latin American (LA) countries affects the IF of a journal detrimentally.

Methodology/Principal Findings

Seven prestigious international journals, one multidisciplinary journal and six serving specific branches of science, were examined in terms of their IF in the Web of Science. Two subsets of each journal were then selected to evaluate the influence of author''s affiliation on the IF. They comprised contributions (i) with authorship from four Latin American (LA) countries (Argentina, Brazil, Chile and Mexico) and (ii) with authorship from five developed countries (England, France, Germany, Japan and USA). Both subsets were further subdivided into two groups: articles with authorship from one country only and collaborative articles with authorship from other countries. Articles from the five developed countries had IF close to the overall IF of the journals and the influence of collaboration on this value was minor. In the case of LA articles the effect of collaboration (virtually all with developed countries) was significant. The IFs for non-collaborative articles averaged 66% of the overall IF of the journals whereas the articles in collaboration raised the IFs to values close to the overall IF.

Conclusion/Significance

The study shows a significantly lower IF in the group of the subsets of non-collaborative LA articles and thus that country affiliation of authors from non-developed LA countries does affect the IF of a journal detrimentally. There are no data to indicate whether the lower IFs of LA articles were due to their inherent inferior quality/relevance or psycho-social trend towards under-citation of articles from these countries. However, further study is required since there are foreseeable consequences of this trend as it may stimulate strategies by editors to turn down articles that tend to be under-cited.     

Citability of Original Research and Reviews in Journals and Their Sponsored Supplements

Background

The contents of pharmaceutical industry sponsored supplements to medical journals are perceived to be less credible than the contents of their parent journals. It is unknown if their contents are cited as often. The objective of this study was to quantify the citability of original research and reviews contained in supplements and compare it with that for the parent journal.

Methodology/Principal Findings

This was a cohort study of 446 articles published in the Journal of Clinical Psychiatry (JCP) and its supplements for calendar years 2000 and 2005. The total citation counts for each article up to October 5, 2009 were retrieved from the ISI Web of Science database. The main outcome measure was the number of citations received by an article since publication. Regular journal articles included 114 from calendar year 2000 and 190 from 2005. Articles from supplements included 90 from 2000 and 52 from 2005. The median citation counts for the 3 years post-publication were 10 (interquartile range [IQR], 4–20), 14 (IQR, 8–20), 13.5 (IQR, 8–23), and 13.5 (IQR, 8–20), for the 2000 parent journal, 2000 supplements, 2005 parent journal, and 2005 supplements, respectively. Citation counts were higher for the articles in the supplements than the parent journal for the cohorts from 2000 (p = .02), and no different for the year 2005 cohorts (p = .88). The 2005 parent journal cohort had higher citation counts than the 2000 cohort (p = .007), in contrast to the supplements where citation counts remained the same (p = .94).

Conclusions/Significance

Articles published in JCP supplements are robustly cited and thus can be influential in guiding clinical and research practice, as well as shaping critical thinking. Because they are printed under the sponsorship of commercial interests, they may be perceived as less than objective. A reasonable step to help improve this perception would be to ensure that supplements are peer-reviewed in the same way as regular articles in the parent journal.     

Scientific publications in nephrology and urology journals from Chinese authors in East Asia: a 10-year survey of the literature

Background

Diseases of the kidneys and genitourinary tract are common health problems that affect people of all ages and demographic backgrounds. In this study, we compared the quantity and quality of nephrological and urological articles published in international journals from the three major regions of China: the mainland (ML), Hong Kong (HK), and Taiwan (TW).

Methods

Nephrological and urological articles originating from ML, TW, and HK that were published in 61 journals from 1999–2008 were retrieved from the PubMed database. We recorded the numbers of total articles, clinical trials, randomized controlled trials, case reports, impact factors (IF), citations, and articles published in the leading general-medicine journals. We used these data to compare the quantity and quality of publication output from the three regions.

Results

The total number of articles increased significantly from 1999 to 2008 in the three regions. The number of articles from ML has exceeded that from HK since 2004, and surpassed that from TW in 2008. Publications from TW had the highest accumulated IF, total citations of articles, and the most articles published in leading general-medicine journals. However, HK publications had the highest average IF. Although ML produced the largest quantity of articles, it exhibited the lowest quality among the three regions.

Conclusion

The number of nephrological and urological publications originating from the three major regions of China increased significantly from 1999 to 2008. The annual number of publications by ML researchers exceeded those from TW and HK. However, the quality of articles from TW and HK was higher than that from ML.     

Universality of Citation Distributions for Academic Institutions and Journals

Citations measure the importance of a publication, and may serve as a proxy for its popularity and quality of its contents. Here we study the distributions of citations to publications from individual academic institutions for a single year. The average number of citations have large variations between different institutions across the world, but the probability distributions of citations for individual institutions can be rescaled to a common form by scaling the citations by the average number of citations for that institution. We find this feature seems to be universal for a broad selection of institutions irrespective of the average number of citations per article. A similar analysis for citations to publications in a particular journal in a single year reveals similar results. We find high absolute inequality for both these sets, Gini coefficients being around 0.66 and 0.58 for institutions and journals respectively. We also find that the top 25% of the articles hold about 75% of the total citations for institutions and the top 29% of the articles hold about 71% of the total citations for journals.     

Integrative Approach to Quality Assessment of Medical Journals Using Impact Factor,Eigenfactor, and Article Influence Scores

Background

Impact factor (IF) is a commonly used surrogate for assessing the scientific quality of journals and articles. There is growing discontent in the medical community with the use of this quality assessment tool because of its many inherent limitations. To help address such concerns, Eigenfactor (ES) and Article Influence scores (AIS) have been devised to assess scientific impact of journals. The principal aim was to compare the temporal trends in IF, ES, and AIS on the rank order of leading medical journals over time.

Methods

The 2001 to 2008 IF, ES, AIS, and number of citable items (CI) of 35 leading medical journals were collected from the Institute of Scientific Information (ISI) and the http://www.eigenfactor.org databases. The journals were ranked based on the published 2008 ES, AIS, and IF scores. Temporal score trends and variations were analyzed.

Results

In general, the AIS and IF values provided similar rank orders. Using ES values resulted in large changes in the rank orders with higher ranking being assigned to journals that publish a large volume of articles. Since 2001, the IF and AIS of most journals increased significantly; however the ES increased in only 51% of the journals in the analysis. Conversely, 26% of journals experienced a downward trend in their ES, while the rest experienced no significant changes (23%). This discordance between temporal trends in IF and ES was largely driven by temporal changes in the number of CI published by the journals.

Conclusion

The rank order of medical journals changes depending on whether IF, AIS or ES is used. All of these metrics are sensitive to the number of citable items published by journals. Consumers should thus consider all of these metrics rather than just IF alone in assessing the influence and importance of medical journals in their respective disciplines.     

Is Quality and Completeness of Reporting of Systematic Reviews and Meta-Analyses Published in High Impact Radiology Journals Associated with Citation Rates?

Purpose

The purpose of this study is to determine whether study quality and completeness of reporting of systematic reviews (SR) and meta-analyses (MA) published in high impact factor (IF) radiology journals is associated with citation rates.

Methods

All SR and MA published in English between Jan 2007–Dec 2011, in radiology journals with an IF >2.75, were identified on Ovid MEDLINE. The Assessing the Methodologic Quality of Systematic Reviews (AMSTAR) checklist for study quality, and the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) checklist for study completeness, was applied to each SR & MA. Each SR & MA was then searched in Google Scholar to yield a citation rate. Spearman correlation coefficients were used to assess the relationship between AMSTAR and PRISMA results with citation rate. Multivariate analyses were performed to account for the effect of journal IF and journal 5-year IF on correlation with citation rate. Values were reported as medians with interquartile range (IQR) provided.

Results

129 studies from 11 journals were included (50 SR and 79 MA). Median AMSTAR result was 8.0/11 (IQR: 5–9) and median PRISMA result was 23.0/27 (IQR: 21–25). The median citation rate for SR & MA was 0.73 citations/month post-publication (IQR: 0.40–1.17). There was a positive correlation between both AMSTAR and PRISMA results and SR & MA citation rate; ρ=0.323 (P=0.0002) and ρ=0.327 (P=0.0002) respectively. Positive correlation persisted for AMSTAR and PRISMA results after journal IF was partialed out; ρ=0.243 (P=0.006) and ρ=0.256 (P=0.004), and after journal 5-year IF was partialed out; ρ=0.235 (P=0.008) and ρ=0.243 (P=0.006) respectively.

Conclusion

There is a positive correlation between the quality and the completeness of a reported SR or MA with citation rate which persists when adjusted for journal IF and journal 5-year IF.     

Open Access Meets Discoverability: Citations to Articles Posted to Academia.edu

Using matching and regression analyses, we measure the difference in citations between articles posted to Academia.edu and other articles from similar journals, controlling for field, impact factor, and other variables. Based on a sample size of 31,216 papers, we find that a paper in a median impact factor journal uploaded to Academia.edu receives 16% more citations after one year than a similar article not available online, 51% more citations after three years, and 69% after five years. We also found that articles also posted to Academia.edu had 58% more citations than articles only posted to other online venues, such as personal and departmental home pages, after five years.     

Impact of Article Language in Multi-Language Medical Journals - a Bibliometric Analysis of Self-Citations and Impact Factor

Background

In times of globalization there is an increasing use of English in the medical literature. The aim of this study was to analyze the influence of English-language articles in multi-language medical journals on their international recognition – as measured by a lower rate of self-citations and higher impact factor (IF).

Methods and Findings

We analyzed publications in multi-language journals in 2008 and 2009 using the Web of Science (WoS) of Thomson Reuters (former Institute of Scientific Information) and PubMed as sources of information. The proportion of English-language articles during the period was compared with both the share of self-citations in the year 2010 and the IF with and without self-citations. Multivariable linear regression analysis was performed to analyze these factors as well as the influence of the journals‘ countries of origin and of the other language(s) used in publications besides English.We identified 168 multi-language journals that were listed in WoS as well as in PubMed and met our criteria. We found a significant positive correlation of the share of English articles in 2008 and 2009 with the IF calculated without self-citations (Pearson r=0.56, p = <0.0001), a correlation with the overall IF (Pearson r = 0.47, p = <0.0001) and with the cites to years of IF calculation (Pearson r = 0.34, p = <0.0001), and a weak negative correlation with the share of self-citations (Pearson r = -0.2, p = 0.009). The IF without self-citations also correlated with the journal‘s country of origin – North American journals had a higher IF compared to Middle and South American or European journals.

Conclusion

Our findings suggest that a larger share of English articles in multi-language medical journals is associated with greater international recognition. Fewer self-citations were found in multi-language journals with a greater share of original articles in English.     

Dora the Brave

The San Francisco Declaration on Research Assessment (DORA) points out that using the Journal Impact Factor as a proxy measure for the value or quality of specific research and individual scientists leads to biased research assessment. How can we resist misusing metrics?If you notice any particularly fidgety journal editors this month don''t worry—this is merely a symptom of the imminent release of the next round of the dreaded, dreadful Journal Impact Factors (JIFs). Editors are concerned, because the JIF directly impacts their journal, as it influences if researchers choose it to publish their research. JIF has a number of flaws, but one entirely outside an editor''s control is noise: a few citations to a single paper can displace a journal in the IF rank list pecking order. Indeed, the JIF would appear to be elaborated to the astonishing significance of three decimal places precisely to minimize the number of ties in journal ranking tables—even if this is at the expense of statistical significance (see ASCB post “A False Sense of Precision”).Matters are worse for journals just below an arbitrary IF threshold set by research assessment policies. A few years ago, when this journal dipped below 10, its editors were on occasion invited back by senior faculty to discuss submission of their work once the JIF had returned to a level deemed relevant by their institution. The only immunity to such JIF excesses appears to be to sport a well-recognized journal name in lieu of perceived JIF deficiencies. Indeed, the remarkable influence of brand recognition is borne testament by the rapid proliferation of journal families around a number of well-recognized names.As always, there will be winners and losers in this year''s JIF league tables—but do these numbers reflect real differences in the quality and interest of the science published in the affected journals?     

Why Has the Number of Scientific Retractions Increased?

Background

The number of retracted scientific publications has risen sharply, but it is unclear whether this reflects an increase in publication of flawed articles or an increase in the rate at which flawed articles are withdrawn.

Methods and Findings

We examined the interval between publication and retraction for 2,047 retracted articles indexed in PubMed. Time-to-retraction (from publication of article to publication of retraction) averaged 32.91 months. Among 714 retracted articles published in or before 2002, retraction required 49.82 months; among 1,333 retracted articles published after 2002, retraction required 23.82 months (p<0.0001). This suggests that journals are retracting papers more quickly than in the past, although recent articles requiring retraction may not have been recognized yet. To test the hypothesis that time-to-retraction is shorter for articles that receive careful scrutiny, time-to-retraction was correlated with journal impact factor (IF). Time-to-retraction was significantly shorter for high-IF journals, but only ∼1% of the variance in time-to-retraction was explained by increased scrutiny. The first article retracted for plagiarism was published in 1979 and the first for duplicate publication in 1990, showing that articles are now retracted for reasons not cited in the past. The proportional impact of authors with multiple retractions was greater in 1972–1992 than in the current era (p<0.001). From 1972–1992, 46.0% of retracted papers were written by authors with a single retraction; from 1993 to 2012, 63.1% of retracted papers were written by single-retraction authors (p<0.001).

Conclusions

The increase in retracted articles appears to reflect changes in the behavior of both authors and institutions. Lower barriers to publication of flawed articles are seen in the increase in number and proportion of retractions by authors with a single retraction. Lower barriers to retraction are apparent in an increase in retraction for “new” offenses such as plagiarism and a decrease in the time-to-retraction of flawed work.     

The Evolution of Academic Performance in Nine Subspecialties of Internal Medicine: An Analysis of Journal Citation Reports from 1998 to 2010

Background

Internal medicine includes several subspecialties. This study aimed to describe change trend of impact factors in different subspecialties of internal medicine during the past 12 years, as well as the developmental differences among each subspecialty, and the possible influencing factors behind these changes and differences.

Methods

Nine subspecialties of internal medicine were chosen for comparison. All data were collected from the Science Citation Index Expanded and Journal Citation Reports database.

Results

(1) Journal numbers in nine subspecialties increased significantly from 1998 to 2010, with an average increment of 80.23%, in which cardiac and cardiovascular system diseases increased 131.2% rank the first; hematology increased 45% rank the least. (2) Impact Factor in subspecialties of infectious disease, cardiac and cardiovascular system diseases, gastroenterology and hepatology, hematology, endocrinology and metabolism increased significantly (p<0.05), in which gastroenterology and hepatology had the largest increase of 65.4%. (3) Journal impact factor of 0–2 had the largest proportion in all subspecialties. Among the journals with high impact factor (IF>6), hematology had the maximum proportion of 10%, nephrology and respiratory system disease had the minimum of 4%. Among the journal with low impact factor (IF<2), journal in nephrology and allergy had the most (60%), while endocrinology and metabolism had the least (40%). There were differences in median number of IF among the different subspecialties (p<0.05), in which endocrinology and metabolism had the highest, nephrology had the lowest. (4) The highest IF had a correlation with journal numbers and total paper numbers in each field.

Conclusion

The IF of internal medicine journals showed an increasingly positive trend, in which gastroenterology and hepatology increase the most. Hematology had more high IF journals. Endocrinology and metabolism had higher average IF. Nephrology remained the lowest position. Numbers of journals and total papers were associated with the highest IF.     

Geographic origin of journal articles on nonhuman primates

During the 1970s variations in the number of citations published in CURRENT PRIMATE REFERENCES seemed to parallel changes in the fortunes of primate research in the United States. In 1981 and 1982, however, the trends diverged, with the number of citations increasing 25% over the two years. In an effort to identify more specifically some factors underlying the trend in literature growth, citations of journal articles appearing in 1971, 1976, and 1981 issues were tabulated according to the general subject indicated by the title and the site of research indicated by the author's address. The number of journal articles was 20% larger in 1981 than in 1971. The United States was the largest contributor in all three years, but its relative contribution declined. Continental Western Europe ranked second consistently, and its contribution grew at an above-average rate. Japan ranked fifth in 1971 and third in 1981. India contributed less than 2% of the literature in 1971 and more than 3% in 1981, as a result of a doubling of its contribution. The subject composition of the literature did not change greatly.