Scientific publications in nephrology and urology journals from Chinese authors in East Asia: a 10-year survey of the literature

Background

Diseases of the kidneys and genitourinary tract are common health problems that affect people of all ages and demographic backgrounds. In this study, we compared the quantity and quality of nephrological and urological articles published in international journals from the three major regions of China: the mainland (ML), Hong Kong (HK), and Taiwan (TW).

Methods

Nephrological and urological articles originating from ML, TW, and HK that were published in 61 journals from 1999–2008 were retrieved from the PubMed database. We recorded the numbers of total articles, clinical trials, randomized controlled trials, case reports, impact factors (IF), citations, and articles published in the leading general-medicine journals. We used these data to compare the quantity and quality of publication output from the three regions.

Results

The total number of articles increased significantly from 1999 to 2008 in the three regions. The number of articles from ML has exceeded that from HK since 2004, and surpassed that from TW in 2008. Publications from TW had the highest accumulated IF, total citations of articles, and the most articles published in leading general-medicine journals. However, HK publications had the highest average IF. Although ML produced the largest quantity of articles, it exhibited the lowest quality among the three regions.

Conclusion

The number of nephrological and urological publications originating from the three major regions of China increased significantly from 1999 to 2008. The annual number of publications by ML researchers exceeded those from TW and HK. However, the quality of articles from TW and HK was higher than that from ML.     

Author self-citation in the general medicine literature

Background

Author self-citation contributes to the overall citation count of an article and the impact factor of the journal in which it appears. Little is known, however, about the extent of self-citation in the general clinical medicine literature. The objective of this study was to determine the extent and temporal pattern of author self-citation and the article characteristics associated with author self-citation.

Methodology/Principal Findings

We performed a retrospective cohort study of articles published in three high impact general medical journals (JAMA, Lancet, and New England Journal of Medicine) between October 1, 1999 and March 31, 2000. We retrieved the number and percentage of author self-citations received by the article since publication, as of June 2008, from the Scopus citation database. Several article characteristics were extracted by two blinded, independent reviewers for each article in the cohort and analyzed in multivariable linear regression analyses. Since publication, author self-citations accounted for 6.5% (95% confidence interval 6.3–6.7%) of all citations received by the 328 articles in our sample. Self-citation peaked in 2002, declining annually thereafter. Studies with more authors, in cardiovascular medicine or infectious disease, and with smaller sample size were associated with more author self-citations and higher percentage of author self-citation (all p≤0.01).

Conclusions/Significance

Approximately 1 in 15 citations of articles in high-profile general medicine journals are author self-citations. Self-citation peaks within about 2 years of publication and disproportionately affects impact factor. Studies most vulnerable to this effect are those with more authors, small sample size, and in cardiovascular medicine or infectious disease.     

Comparative study of scientific publications in urology and nephrology journals originating from USA, china and Japan (2001-2010)

Background

In the past decade, scientific research has developed rapidly in China, but the growth seems to vary widely between different disciplines. In this study, we aimed to compare the quantity and quality of publications in urology and nephrology journals from USA, China and Japan.

Methods

Journals listed in the “Urology and Nephrology” category of Science Citation Index Expanded subject categories were included. Scientific papers in these journals written by researchers from USA, Japan and China were retrieved from the “PubMed” and “Web of Knowledge” online databases.

Results

The annual number of total scientific articles increased significantly from 2001 to 2010 in China, and has ranked second in the world since 2006. In the field of urology and nephrology, the annual number increased significantly from 2001 to 2010 in USA and China; but not in Japan. The share of articles increased significantly over time in China, decreased significantly in Japan, and remained unchanged in USA. In 2010, USA contributed 32.17% of the total world output in urology and nephrology field and ranked 1^st; Japan contributed 5.19% and ranked 5^th; China contributed 3.83% and ranked 9^th. Publications from USA had the highest accumulated IFs and the highest total citations of articles (USA>Japan>China, p<0.001). No significant difference was found in average IF among the three countries. USA published the most articles in the top 10 urology and nephrology journals (USA(35165)>Japan(6704)>China(2233), p<0.001). Researchers from USA published more clinical trials and randomized controlled trials than Japan and China (USA>Japan>China, p<0.001).

Conclusion

Although China has undergone significant increase in annual number and percentage of scientific publication in urology and nephrology journals in the past decade, it still lags far behind USA and Japan in the field of urology and nephrology in terms of quantity and quality.     

Scientific publications on primary biliary cirrhosis from 2000 through 2010: an 11-year survey of the literature

Background

Primary biliary cirrhosis (PBC) is a chronic liver disease characterized by intrahepatic bile-duct destruction, cholestasis, and fibrosis. It can lead to cirrhosis and eventually liver failure. PBC also shows some regional differences with respect to incidence and prevalence that are becoming more pronounced each year. Recently, researchers have paid more attention to PBC. To evaluate the development of PBC research during the past 11 years, we determined the quantity and quality of articles on this subject. We also compared the contributions of scientists from the US, UK, Japan, Italy, Germany, and China.

Methods

The English-language papers covering PBC published in journals from 2000 through 2010 were retrieved from the PubMed database. We recorded the number of papers published each year, analyzed the publication type, and calculated the accumulated, average impact factors (IFs) and citations from every country. The quantity and quality of articles on PBC were compared by country. We also contrasted the level of PBC research in China and other countries.

Results

The total number of articles did not significantly increase during the past 11 years. The number of articles from the US exceeded those from any other country; the publications from the US also had the highest IFs and the most citations. Four other countries showed complex trends with respect to the quantity and quality of articles about PBC.

Conclusion

The researchers from the US have contributed the most to the development of PBC research. They currently represent the highest level of research. Some high-level studies, such as RCTs, meta-analyses, and in-depth basic studies should be launched. The gap between China and the advanced level is still enormous. Chinese investigators still have a long way to go.     

Impact of GDP,Spending on R&D,Number of Universities and Scientific Journals on Research Publications among Asian Countries

Objectives

This study aimed to compare the impact of Gross Domestic Product (GDP) per capita, spending on Research and Development (R&D), number of universities, and Indexed Scientific Journals on total number of research documents (papers), citations per document and Hirsch index (H-index) in various science and social science subjects among Asian countries.

Materials and Methods

In this study, 40 Asian countries were included. The information regarding Asian countries, their GDP per capita, spending on R&D, total number of universities and indexed scientific journals were collected. We recorded the bibliometric indicators, including total number of research documents, citations per document and H-index in various science and social sciences subjects during the period 1996–2011. The main sources for information were World Bank, SCI-mago/Scopus and Web of Science; Thomson Reuters.

Results

The mean per capita GDP for all the Asian countries is 14448.31±2854.40 US$, yearly per capita spending on R&D 0.64±0.16 US$, number of universities 72.37±18.32 and mean number of ISI indexed journal per country is 17.97±7.35. The mean of research documents published in various science and social science subjects among all the Asian countries during the period 1996–2011 is 158086.92±69204.09; citations per document 8.67±0.48; and H-index 122.8±19.21. Spending on R&D, number of universities and indexed journals have a positive correlation with number of published documents, citations per document and H-index in various science and social science subjects. However, there was no association between the per capita GDP and research outcomes.

Conclusion

The Asian countries who spend more on R&D have a large number of universities and scientific indexed journals produced more in research outcomes including total number of research publication, citations per documents and H-index in various science and social science subjects.     

The h’-Index,Effectively Improving the h-Index Based on the Citation Distribution

Background

Although being a simple and effective index that has been widely used to evaluate academic output of scientists, the h-index suffers from drawbacks. One critical disadvantage is that only h-squared citations can be inferred from the h-index, which completely ignores excess and h-tail citations, leading to unfair and inaccurate evaluations in many cases.

Methodology /Principal Findings

To solve this problem, I propose the h’-index, in which h-squared, excess and h-tail citations are all considered. Based on the citation data of the 100 most prolific economists, comparing to h-index, the h’-index shows better correlation with indices of total-citation number and citations per publication, which, although relatively reliable and widely used, do not carry the information of the citation distribution. In contrast, the h’-index possesses the ability to discriminate the shapes of citation distributions, thus leading to more accurate evaluation.

Conclusions /Significance

The h’-index improves the h-index, as well as indices of total-citation number and citations per publication, by possessing the ability to discriminate shapes of citation distribution, thus making the h’-index a better single-number index for evaluating scientific output in a way that is fairer and more reasonable.     

Why Has the Number of Scientific Retractions Increased?

Background

The number of retracted scientific publications has risen sharply, but it is unclear whether this reflects an increase in publication of flawed articles or an increase in the rate at which flawed articles are withdrawn.

Methods and Findings

We examined the interval between publication and retraction for 2,047 retracted articles indexed in PubMed. Time-to-retraction (from publication of article to publication of retraction) averaged 32.91 months. Among 714 retracted articles published in or before 2002, retraction required 49.82 months; among 1,333 retracted articles published after 2002, retraction required 23.82 months (p<0.0001). This suggests that journals are retracting papers more quickly than in the past, although recent articles requiring retraction may not have been recognized yet. To test the hypothesis that time-to-retraction is shorter for articles that receive careful scrutiny, time-to-retraction was correlated with journal impact factor (IF). Time-to-retraction was significantly shorter for high-IF journals, but only ∼1% of the variance in time-to-retraction was explained by increased scrutiny. The first article retracted for plagiarism was published in 1979 and the first for duplicate publication in 1990, showing that articles are now retracted for reasons not cited in the past. The proportional impact of authors with multiple retractions was greater in 1972–1992 than in the current era (p<0.001). From 1972–1992, 46.0% of retracted papers were written by authors with a single retraction; from 1993 to 2012, 63.1% of retracted papers were written by single-retraction authors (p<0.001).

Conclusions

The increase in retracted articles appears to reflect changes in the behavior of both authors and institutions. Lower barriers to publication of flawed articles are seen in the increase in number and proportion of retractions by authors with a single retraction. Lower barriers to retraction are apparent in an increase in retraction for “new” offenses such as plagiarism and a decrease in the time-to-retraction of flawed work.     

Rising Publication Delays Inflate Journal Impact Factors

Journal impact factors have become an important criterion to judge the quality of scientific publications over the years, influencing the evaluation of institutions and individual researchers worldwide. However, they are also subject to a number of criticisms. Here we point out that the calculation of a journal’s impact factor is mainly based on the date of publication of its articles in print form, despite the fact that most journals now make their articles available online before that date. We analyze 61 neuroscience journals and show that delays between online and print publication of articles increased steadily over the last decade. Importantly, such a practice varies widely among journals, as some of them have no delays, while for others this period is longer than a year. Using a modified impact factor based on online rather than print publication dates, we demonstrate that online-to-print delays can artificially raise a journal’s impact factor, and that this inflation is greater for longer publication lags. We also show that correcting the effect of publication delay on impact factors changes journal rankings based on this metric. We thus suggest that indexing of articles in citation databases and calculation of citation metrics should be based on the date of an article’s online appearance, rather than on that of its publication in print.     

Effectiveness of journal ranking schemes as a tool for locating information

Background

The rise of electronic publishing [1], preprint archives, blogs, and wikis is raising concerns among publishers, editors, and scientists about the present day relevance of academic journals and traditional peer review [2]. These concerns are especially fuelled by the ability of search engines to automatically identify and sort information [1]. It appears that academic journals can only remain relevant if acceptance of research for publication within a journal allows readers to infer immediate, reliable information on the value of that research.

Methodology/Principal Findings

Here, we systematically evaluate the effectiveness of journals, through the work of editors and reviewers, at evaluating unpublished research. We find that the distribution of the number of citations to a paper published in a given journal in a specific year converges to a steady state after a journal-specific transient time, and demonstrate that in the steady state the logarithm of the number of citations has a journal-specific typical value. We then develop a model for the asymptotic number of citations accrued by papers published in a journal that closely matches the data.

Conclusions/Significance

Our model enables us to quantify both the typical impact and the range of impacts of papers published in a journal. Finally, we propose a journal-ranking scheme that maximizes the efficiency of locating high impact research.     

A tale of four stories: soil ecology, theory, evolution and the publication system

Background

Soil ecology has produced a huge corpus of results on relations between soil organisms, ecosystem processes controlled by these organisms and links between belowground and aboveground processes. However, some soil scientists think that soil ecology is short of modelling and evolutionary approaches and has developed too independently from general ecology. We have tested quantitatively these hypotheses through a bibliographic study (about 23000 articles) comparing soil ecology journals, generalist ecology journals, evolutionary ecology journals and theoretical ecology journals.

Findings

We have shown that soil ecology is not well represented in generalist ecology journals and that soil ecologists poorly use modelling and evolutionary approaches. Moreover, the articles published by a typical soil ecology journal (Soil Biology and Biochemistry) are cited by and cite low percentages of articles published in generalist ecology journals, evolutionary ecology journals and theoretical ecology journals.

Conclusion

This confirms our hypotheses and suggests that soil ecology would benefit from an effort towards modelling and evolutionary approaches. This effort should promote the building of a general conceptual framework for soil ecology and bridges between soil ecology and general ecology. We give some historical reasons for the parsimonious use of modelling and evolutionary approaches by soil ecologists. We finally suggest that a publication system that classifies journals according to their Impact Factors and their level of generality is probably inadequate to integrate “particularity” (empirical observations) and “generality” (general theories), which is the goal of all natural sciences. Such a system might also be particularly detrimental to the development of a science such as ecology that is intrinsically multidisciplinary.     

Meta-Analysis of Gadoxetic Acid Disodium (Gd-EOB-DTPA)-Enhanced Magnetic Resonance Imaging for the Detection of Liver Metastases

Objective

To determine the accuracy of MR imaging with Gd-EOB-DTPA for the detection of liver metastases.

Materials and Methods

PUBMED, EMBASE, the Web of Science, and the Cochrane Library were searched for original articles published prior to February 2012. The criteria for the inclusion of articles were as follows: reported in the English language; MR imaging with Gd-EOB-DTPA was performed to detect liver metastases; histopathologic analysis (surgery, biopsy), intraoperative observation (manual palpatation, intraoperative ultrasonography), and/or follow-up US was the reference standard; and data were sufficient for the calculation of true-positive or false-negative values. The methodological quality was assessed by using the quality assessment of diagnostic studies instrument. The data were extracted to calculate sensitivity, specificity, predictive value, diagnostic odds ratio, and areas under hierarchical summary receiver operating characteristic (HSROC) curve to perform heterogeneity test and threshold effect test, as well as publication bias analysis and subgroup analyses.

Results

From 229 citations, 13 were included in the meta-analysis with a total of 1900 lesions. We detected heterogeneity between studies and evidence of publication bias. The methodological quality was moderate. The pooled weighted sensitivity with a corresponding 95% confidence interval (CI) was 0.93 (95% CI: 0.90, 0. 95), the specificity was 0.95 (95% CI: 0.91, 0.97), the positive likelihood ratio was 18.07 (95% CI: 10.52, 31.04), the negative likelihood ratio was 0.07 (95% CI: 0.05, 0.10), and the diagnostic odds ratio was 249.81 (95% CI: 125.12, 498.74). The area under the receiver operator characteristic curve was 0.98 (95% CI: 0.96, 0.99).

Conclusion

MR imaging with Gd-EOB-DTPA is a reliable, non-invasive, and no-radiation-exposure imaging modality with a high sensitivity and specificity for detection of liver metastases. Nonetheless, it should be applied cautiously, and large scale, well-designed trials are necessary to assess its clinical value.     

Fifty-Year Fate and Impact of General Medical Journals

Background

Influential medical journals shape medical science and practice and their prestige is usually appraised by citation impact metrics, such as the journal impact factor. However, how permanent are medical journals and how stable is their impact over time?

Methods and Results

We evaluated what happened to general medical journals that were publishing papers half a century ago, in 1959. Data were retrieved from ISI Web of Science for citations and PubMed (Journals function) for journal history. Of 27 eligible journals publishing in 1959, 4 have stopped circulation (including two of the most prestigious journals in 1959) and another 7 changed name between 1959 and 2009. Only 6 of these 27 journals have been published continuously with their initial name since they started circulation. The citation impact of papers published in 1959 gives a very different picture from the current journal impact factor; the correlation between the two is non-significant and very close to zero. Only 13 of the 5,223 papers published in 1959 received at least 5 citations in 2009.

Conclusions

Journals are more permanent entities than single papers, but they are also subject to major change and their relative prominence can change markedly over time.     

Differences in the Volume of Pharmaceutical Advertisements between Print General Medical Journals

Background

Pharmaceutical advertisements have been argued to provide revenue that medical journals require but they are intended to alter prescribing behaviour and they are known to include low quality information. We determined whether a difference exists in the current level of pharmaceutical advertising in print general medical journals, and we estimated the revenue generated from print pharmaceutical advertising.

Methods

Six print general medical journals in Canada, the United States, and the United Kingdom were sampled between 2007 and 2012. The number of advertisements and other journal content in selected issues of the Canadian Medical Association Journal (CMAJ), Canadian Family Physician (CFP), Journal of the American Medical Association (JAMA), New England Journal of Medicine (NEJM), British Medical Journal (BMJ), and Lancet were determined. Revenue gained from pharmaceutical advertising was estimated using each journal''s 2013 advertising price list.

Findings

The two Canadian journals sampled (CMAJ, CFP) contained five times more advertisements than the two American journals (JAMA, NEJM), and two British journals (BMJ, Lancet) (p<0.0001). The estimated annual revenue from pharmaceutical advertisements ranged from £0.025 million (for Lancet) to £3.8 million (for JAMA). The cost savings due to revenue from pharmaceutical advertising to each individual subscriber ranged from £0.02 (for Lancet) to £3.56 (for CFP) per issue.

Conclusion

The volume of pharmaceutical advertisements differs between general medical journals, with the two Canadian journals sampled containing the most advertisements. International and temporal variations suggest that there is an opportunity for all general medical journals to reduce the number of pharmaceutical advertisements, explore other sources of revenue, and increase transparency regarding sources of revenue.     

Is Qualitative Research Second Class Science? A Quantitative Longitudinal Examination of Qualitative Research in Medical Journals

Background

Qualitative research appears to be gaining acceptability in medical journals. Yet, little is actually known about the proportion of qualitative research and factors affecting its publication. This study describes the proportion of qualitative research over a 10 year period and correlates associated with its publication.

Design

A quantitative longitudinal examination of the proportion of original qualitative research in 67 journals of general medicine during a 10 year period (1998–2007). The proportion of qualitative research was determined by dividing original qualitative studies published (numerator) by all original research articles published (denominator). We used a generalized estimating equations approach to assess the longitudinal association between the proportion of qualitative studies and independent variables (i.e. journals'' country of publication and impact factor; editorial/methodological papers discussing qualitative research; and specific journal guidelines pertaining to qualitative research).

Findings

A 2.9% absolute increase and 3.4-fold relative increase in qualitative research publications occurred over a 10 year period (1.2% in 1998 vs. 4.1% in 2007). The proportion of original qualitative research was independently and significantly associated with the publication of editorial/methodological papers in the journal (b = 3.688, P = 0.012); and with qualitative research specifically mentioned in guidelines for authors (b = 6.847, P<0.001). Additionally, a higher proportion of qualitative research was associated only with journals published in the UK in comparison to other countries, yet with borderline statistical significance (b = 1.776, P = 0.075). The journals'' impact factor was not associated with the publication of qualitative research.

Conclusions

Despite an increase in the proportion of qualitative research in medical journals over a 10 year period, the proportion remains low. Journals'' policies pertaining to qualitative research, as expressed by the appearance of specific guidelines and editorials/methodological papers on the subject, are independently associated with the publication of original qualitative research; irrespective of the journals'' impact factor.     

Citation Classics in Systematic Reviews and Meta-Analyses: Who Wrote the Top 100 Most Cited Articles?

Background

Systematic reviews of the literature occupy the highest position in currently proposed hierarchies of evidence. The aims of this study were to assess whether citation classics exist in published systematic review and meta-analysis (SRM), examine the characteristics of the most frequently cited SRM articles, and evaluate the contribution of different world regions.

Methods

The 100 most cited SRM were identified in October 2012 using the Science Citation Index database of the Institute for Scientific Information. Data were extracted by one author. Spearman’s correlation was used to assess the association between years since publication, numbers of authors, article length, journal impact factor, and average citations per year.

Results

Among the 100 citation classics, published between 1977 and 2008, the most cited article received 7308 citations and the least-cited 675 citations. The average citations per year ranged from 27.8 to 401.6. First authors from the USA produced the highest number of citation classics (n=46), followed by the UK (n=28) and Canada (n=15). The 100 articles were published in 42 journals led by the Journal of the American Medical Association (n=18), followed by the British Medical Journal (n=14) and The Lancet (n=13). There was a statistically significant positive correlation between number of authors (Spearman’s rho=0.320, p=0.001), journal impact factor (rho=0.240, p=0.016) and average citations per year. There was a statistically significant negative correlation between average citations per year and year since publication (rho = -0.636, p=0.0001). The most cited papers identified seminal contributions and originators of landmark methodological aspects of SRM and reflect major advances in the management of and predisposing factors for chronic diseases.

Conclusions

Since the late 1970s, the USA, UK, and Canada have taken leadership in the production of citation classic papers. No first author from low or middle-income countries (LMIC) led one of the most cited 100 SRM.     

Representation of Women and Pregnant Women in HIV Research: A Limited Systematic Review

Background

HIV-related outcomes may be affected by biological sex and by pregnancy. Including women in general and pregnant women in particular in HIV-related research is important for generalizability of findings.

Objective

To characterize representation of pregnant and non-pregnant women in HIV-related research conducted in general populations.

Data Sources

All HIV-related articles published in fifteen journals from January to March of 2011. We selected the top five journals by 2010 impact factor, in internal medicine, infectious diseases, and HIV/AIDS.

Study Eligibility Criteria

HIV-related studies reporting original research on questions applicable to both men and women of reproductive age were considered; studies were excluded if they did not include individual-level patient data.

Study appraisal and synthesis methods.

Articles were doubly reviewed and abstracted; discrepancies were resolved through consensus. We recorded proportion of female study participants, whether pregnant women were included or excluded, and other key factors.

Results

In total, 2014 articles were published during this period. After screening, 259 articles were included as original HIV-related research reporting individual-level data; of these, 226 were determined to be articles relevant to both men and women of reproductive age. In these articles, women were adequately represented within geographic region. The vast majority of published articles, 183/226 (81%), did not mention pregnancy (or related issues); still fewer included pregnant women (n=33), reported numbers of pregnant women (n=19), or analyzed using pregnancy status (n=9).

Limitations

Data were missing for some key variables, including pregnancy. The time period over which published works were evaluated was relatively short.

Conclusions and implications of key findings.

The under-reporting and inattention to pregnancy in the HIV literature may reduce policy-makers’ ability to set evidence-based policy around HIV/AIDS care for pregnant women and women of child-bearing age.     

Conflicts of interest at medical journals: the influence of industry-supported randomised trials on journal impact factors and revenue - cohort study

Background

Transparency in reporting of conflict of interest is an increasingly important aspect of publication in medical journals. Publication of large industry-supported trials may generate many citations and journal income through reprint sales and thereby be a source of conflicts of interest for journals. We investigated industry-supported trials'' influence on journal impact factors and revenue.

Methods and Findings

We sampled six major medical journals (Annals of Internal Medicine, Archives of Internal Medicine, BMJ, JAMA, The Lancet, and New England Journal of Medicine [NEJM]). For each journal, we identified randomised trials published in 1996–1997 and 2005–2006 using PubMed, and categorized the type of financial support. Using Web of Science, we investigated citations of industry-supported trials and the influence on journal impact factors over a ten-year period. We contacted journal editors and retrieved tax information on income from industry sources. The proportion of trials with sole industry support varied between journals, from 7% in BMJ to 32% in NEJM in 2005–2006. Industry-supported trials were more frequently cited than trials with other types of support, and omitting them from the impact factor calculation decreased journal impact factors. The decrease varied considerably between journals, with 1% for BMJ to 15% for NEJM in 2007. For the two journals disclosing data, income from the sales of reprints contributed to 3% and 41% of the total income for BMJ and The Lancet in 2005–2006.

Conclusions

Publication of industry-supported trials was associated with an increase in journal impact factors. Sales of reprints may provide a substantial income. We suggest that journals disclose financial information in the same way that they require them from their authors, so that readers can assess the potential effect of different types of papers on journals'' revenue and impact. Please see later in the article for the Editors'' Summary     

Academic Impact of Qualitative Studies in Healthcare: Bibliometric Analysis

Context

Although qualitative studies are becoming more appreciated in healthcare, the number of publications of quality studies remains low. Little is known about the frequency and characteristics of citation in qualitative studies.

Objective

To compare the academic impact of qualitative studies to that of two quantitative studies: systematic reviews and randomized controlled trials.

Methods

Publications in BMJ between 1997 and 2006 (BMJ’s median impact factor was 7.04 during this period) employing qualitative methods were matched to two quantitative studies appearing the same year using PubMed. Using Web of Science, citations within a 24-month publication period were determined. Additionally, three hypotheses were examined: qualitative studies are 1) infrequently cited in original articles or reviews; 2) rarely cited by authors in non-English-speaking countries; and 3) more frequently cited in non-medical disciplines (e.g., psychology or sociology).

Results

A total of 121 qualitative studies, 270 systematic reviews, and 515 randomised controlled trials were retrieved. Qualitative studies were cited a total of 1,089 times, with a median of 7.00 times (range, 0–34) for each study. Matched systematic reviews and randomized controlled trials were cited 2,411times and 1,600 times, respectively. With respect to citing documents, original articles and reviews exceeded 60% for each study design. Relative to quantitative studies, qualitative studies were cited more often by authors in English-speaking countries. With respect to subject area, medical disciplines were more frequently cited than non-medical disciplines for all three study designs (>80%).

Conclusion

The median number of citations for qualitative studies was almost the same as the median of BMJ’s impact factor during the survey period. For a suitable evaluation of qualitative studies in healthcare, it will be necessary to develop a reporting framework and include explicit discussions of clinical implications when reporting findings. Coordination between researchers and editors will be needed to achieve this goal.