Assessment of transparency indicators across the biomedical literature: How open is open?

Recent concerns about the reproducibility of science have led to several calls for more open and transparent research practices and for the monitoring of potential improvements over time. However, with tens of thousands of new biomedical articles published per week, manually mapping and monitoring changes in transparency is unrealistic. We present an open-source, automated approach to identify 5 indicators of transparency (data sharing, code sharing, conflicts of interest disclosures, funding disclosures, and protocol registration) and apply it across the entire open access biomedical literature of 2.75 million articles on PubMed Central (PMC). Our results indicate remarkable improvements in some (e.g., conflict of interest [COI] disclosures and funding disclosures), but not other (e.g., protocol registration and code sharing) areas of transparency over time, and map transparency across fields of science, countries, journals, and publishers. This work has enabled the creation of a large, integrated, and openly available database to expedite further efforts to monitor, understand, and promote transparency and reproducibility in science.

This study uses novel open source automated tools to monitor transparency across all 2.75 million open access articles on PubMed Central, discovering that different disciplines, journals and publishers abide by principles of transparency to varying degrees over time.     

Financial Conflicts of Interest and Reporting Bias Regarding the Association between Sugar-Sweetened Beverages and Weight Gain: A Systematic Review of Systematic Reviews

Background

Industry sponsors'' financial interests might bias the conclusions of scientific research. We examined whether financial industry funding or the disclosure of potential conflicts of interest influenced the results of published systematic reviews (SRs) conducted in the field of sugar-sweetened beverages (SSBs) and weight gain or obesity.

Methods and Findings

We conducted a search of the PubMed, Cochrane Library, and Scopus databases to identify published SRs from the inception of the databases to August 31, 2013, on the association between SSB consumption and weight gain or obesity. SR conclusions were independently classified by two researchers into two groups: those that found a positive association and those that did not. These two reviewers were blinded with respect to the stated source of funding and the disclosure of conflicts of interest.We identified 17 SRs (with 18 conclusions). In six of the SRs a financial conflict of interest with some food industry was disclosed. Among those reviews without any reported conflict of interest, 83.3% of the conclusions (10/12) were that SSB consumption could be a potential risk factor for weight gain. In contrast, the same percentage of conclusions, 83.3% (5/6), of those SRs disclosing some financial conflict of interest with the food industry were that the scientific evidence was insufficient to support a positive association between SSB consumption and weight gain or obesity. Those reviews with conflicts of interest were five times more likely to present a conclusion of no positive association than those without them (relative risk: 5.0, 95% CI: 1.3–19.3).An important limitation of this study is the impossibility of ruling out the existence of publication bias among those studies not declaring any conflict of interest. However, the best large randomized trials also support a direct association between SSB consumption and weight gain or obesity.

Conclusions

Financial conflicts of interest may bias conclusions from SRs on SSB consumption and weight gain or obesity. Please see later in the article for the Editors'' Summary     

Who Shares? Who Doesn't? Factors Associated with Openly Archiving Raw Research Data

Many initiatives encourage investigators to share their raw datasets in hopes of increasing research efficiency and quality. Despite these investments of time and money, we do not have a firm grasp of who openly shares raw research data, who doesn''t, and which initiatives are correlated with high rates of data sharing. In this analysis I use bibliometric methods to identify patterns in the frequency with which investigators openly archive their raw gene expression microarray datasets after study publication.Automated methods identified 11,603 articles published between 2000 and 2009 that describe the creation of gene expression microarray data. Associated datasets in best-practice repositories were found for 25% of these articles, increasing from less than 5% in 2001 to 30%–35% in 2007–2009. Accounting for sensitivity of the automated methods, approximately 45% of recent gene expression studies made their data publicly available.First-order factor analysis on 124 diverse bibliometric attributes of the data creation articles revealed 15 factors describing authorship, funding, institution, publication, and domain environments. In multivariate regression, authors were most likely to share data if they had prior experience sharing or reusing data, if their study was published in an open access journal or a journal with a relatively strong data sharing policy, or if the study was funded by a large number of NIH grants. Authors of studies on cancer and human subjects were least likely to make their datasets available.These results suggest research data sharing levels are still low and increasing only slowly, and data is least available in areas where it could make the biggest impact. Let''s learn from those with high rates of sharing to embrace the full potential of our research output.     

Species,Habitats, Society: An Evaluation of Research Supporting EU's Natura 2000 Network

The Natura 2000 network is regarded as one of the conservation success stories in the global effort to protect biodiversity. However, significant challenges remain in Natura 2000 implementation, owing to its rapid expansion, and lack of a coherent vision for its future. Scientific research is critical for identifying conservation priorities, setting management goals, and reconciling biodiversity protection and society in the complex political European landscape. Thus, there is an urgent need for a comprehensive evaluation of published Natura 2000 research to highlight prevalent research themes, disciplinary approaches, and spatial entities. We conducted a systematic review of 572 scientific articles and conference proceedings focused on Natura 2000 research, published between 1996 and 2014. We grouped these articles into ‘ecological’ and ‘social and policy’ categories. Using a novel application of network analysis of article keywords, we found that Natura 2000 research forms a cohesive small-world network, owing to the emphasis on ecological research (79% of studies, with a strong focus on spatial conservation planning), and the underrepresentation of studies addressing ‘social and policy’ issues (typically focused on environmental impact assessment, multi-level governance, agri-environment policy, and ecosystem services valuation). ‘Ecological’ and ‘social and policy’ research shared only general concepts (e.g., Natura 2000, Habitats Directive) suggesting a disconnection between these disciplines. The UK and the Mediterranean basin countries dominated Natura 2000 research, and there was a weak correlation between number of studies and proportion of national territory protected. Approximately 40% of ‘social and policy’ research and 26% of ‘ecological’ studies highlighted negative implications of Natura 2000, while 21% of studies found positive social and biodiversity effects. We emphasize the need for designing inter- and transdisciplinary research in order to promote a social-ecological understanding of Natura 2000, and advance EU conservation policies.     

Promotional tone in reviews of menopausal hormone therapy after the Women's Health Initiative: an analysis of published articles

Background

Even after the Women''s Health Initiative (WHI) found that the risks of menopausal hormone therapy (hormone therapy) outweighed benefit for asymptomatic women, about half of gynecologists in the United States continued to believe that hormones benefited women''s health. The pharmaceutical industry has supported publication of articles in medical journals for marketing purposes. It is unknown whether author relationships with industry affect promotional tone in articles on hormone therapy. The goal of this study was to determine whether promotional tone could be identified in narrative review articles regarding menopausal hormone therapy and whether articles identified as promotional were more likely to have been authored by those with conflicts of interest with manufacturers of menopausal hormone therapy.

Methods and Findings

We analyzed tone in opinion pieces on hormone therapy published in the four years after the estrogen-progestin arm of the WHI was stopped. First, we identified the ten authors with four or more MEDLINE-indexed reviews, editorials, comments, or letters on hormone replacement therapy or menopausal hormone therapy published between July 2002 and June 2006. Next, we conducted an additional search using the names of these authors to identify other relevant articles. Finally, after author names and affiliations were removed, 50 articles were evaluated by three readers for scientific accuracy and for tone. Scientific accuracy was assessed based on whether or not the findings of the WHI were accurately reported using two criteria: (1) Acknowledgment or lack of denial of the risk of breast cancer diagnosis associated with hormone therapy, and (2) acknowledgment that hormone therapy did not benefit cardiovascular disease endpoints. Determination of promotional tone was based on the assessment by each reader of whether the article appeared to promote hormone therapy. Analysis of inter-rater consistency found moderate agreement for scientific accuracy (κ = 0.57) and substantial agreement for promotional tone (κ = 0.65). After discussion, readers found 86% of the articles to be scientifically accurate and 64% to be promotional in tone. Themes that were common in articles considered promotional included attacks on the methodology of the WHI, arguments that clinical trial results should not guide treatment for individuals, and arguments that observational studies are as good as or better than randomized clinical trials for guiding clinical decisions. The promotional articles we identified also implied that the risks associated with hormone therapy have been exaggerated and that the benefits of hormone therapy have been or will be proven. Of the ten authors studied, eight were found to have declared payment for speaking or consulting on behalf of menopausal hormone manufacturers or for research support (seven of these eight were speakers or consultants). Thirty of 32 articles (90%) evaluated as promoting hormone therapy were authored by those with potential financial conflicts of interest, compared to 11 of 18 articles (61%) by those without such conflicts (p = 0.0025). Articles promoting the use of menopausal hormone therapy were 2.41 times (95% confidence interval 1.49–4.93) as likely to have been authored by authors with conflicts of interest as by authors without conflicts of interest. In articles from three authors with conflicts of interest some of the same text was repeated word-for-word in different articles.

Conclusion

There may be a connection between receiving industry funding for speaking, consulting, or research and the publication of promotional opinion pieces on menopausal hormone therapy. Please see later in the article for the Editors'' Summary     

The Effects of Research & Development Funding on Scientific Productivity: Academic Chemistry, 1990-2009

This article examines the relationship between Research & Development (R&D) funding and the production of knowledge by academic chemists. Using articles published, either raw counts or adjusted for quality, we find a strong, positive causal effect of funding on knowledge production. This effect is similar across subsets of universities, suggesting a relatively efficient allocation of R&D funds. Finally, we document a rapid acceleration in the rate at which chemical knowledge was produced in the late 1990s and early 2000s relative to the financial and human resources devoted to its production.     

La depresión: un predictor de demencia

Many studies suggest that in 10-25% of cases of Alzheimer's, the most common dementia in our society, can be prevented with the elimination of some risk factors. Barnes and Yaffe found that one-third of Alzheimer's cases are attributable to depression, but in the scientific literature it is not clear if it has a real causal effect on the development of dementia. The purpose of this study is to analyse the scientific evidence on the hypothesis that depression increases the risk of developing dementia.A systematic review and a meta-analysis were performed on the scientific literature published up until the present day, searching articles that were published between 1990 and 2014. Ten of the studies found met the selection criteria –similar to a) size and characteristics of the sample (origin, age…), b) process of gathering data c) method of studying the relationship (within and/or between group comparison), and d) statistical analysis of the results– and the previously established quality.The value of odds ratio varied from 1.72 to 3.59, and the hazard ratio from 1,72 to 5.44. This indicates that the subjects with a history of depression have a higher risk of developing dementia than others who did not suffer depression.     

An Assessment of the Methodological Quality of Published Network Meta-Analyses: A Systematic Review

ObjectiveTo assess the methodological quality of published network meta-analysis.DesignSystematic review.MethodsWe searched the medical literature for network meta-analyses of pharmaceuticals. We assessed general study characteristics, study transparency and reproducibility, methodological approach, and reporting of findings. We compared studies published in journals with lower impact factors with those published in journals with higher impact factors, studies published prior to January 1^st, 2013 with those published after that date, and studies supported financially by industry with those supported by non-profit institutions or that received no support.ResultsThe systematic literature search identified 854 citations. Three hundred and eighteen studies met our inclusion criteria. The number of network meta-analyses has grown rapidly, with 48% of studies published since January 2013. The majority of network meta-analyses were supported by a non-profit institution or received no support (68%). We found considerable inconsistencies among reviewed studies. Eighty percent reported search terms, 61% a network diagram, 65% sufficient data to replicate the analysis, and 90% the characteristics of included trials. Seventy percent performed a risk of bias assessment of included trials, 40% an assessment of model fit, and 56% a sensitivity analysis. Among studies with a closed loop, 69% examined the consistency of direct and indirect evidence. Sixty-four percent of studies presented the full matrix of head-to-head treatment comparisons. For Bayesian studies, 41% reported the probability that each treatment was best, 31% reported treatment ranking, and 16% included the model code or referenced publicly-available code. Network meta-analyses published in higher impact factors journals and those that did not receive industry support performed better across the assessment criteria. We found few differences between older and newer studies.ConclusionsThere is substantial variation in the network meta-analysis literature. Consensus among guidelines is needed improve the methodological quality, transparency, and consistency of study conduct and reporting.     

The Impact of Funding through the RF President’s Grants for Young Scientists (the field – Medicine) on Research Productivity: A Quasi-Experimental Study and a Brief Systematic Review

The impact of grants on research productivity has been investigated by a number of retrospective studies. The results of these studies vary considerably. The objective of my study was to investigate the impact of funding through the RF President’s grants for young scientists on the research productivity of awarded applicants. The study compared the number of total articles and citations for awarded and rejected applicants who in 2007 took part in competitions for young candidates of science (CoS’s) and doctors of science (DoS’s) in the scientific field of medicine. The bibliometric analysis was conducted for the period from 2003 to 2012 (five years before and after the competition). The source of bibliometric data is the eLIBRARY.RU database. The impact of grants on the research productivity of Russian young scientists was assessed using the meta-analytical approach based on data from quasi-experimental studies conducted in other countries. The competition featured 149 CoS’s and 41 DoS’s, out of which 24 (16%) and 22 (54%) applicants, respectively, obtained funding. No difference in the number of total articles and citations at baseline, as well as in 2008–2012, for awarded and rejected applicants was found. The combination of data from the Russian study and other quasi-experimental studies (6 studies, 10 competitions) revealed a small treatment effect – an increase in the total number of publications over a 4–5-year period after the competition by 1.23 (95% CI 0.48–1.97). However, the relationship between the number of total publications published by applicants before and after the competition revealed that this treatment effect is an effect of the “maturation” of scientists with a high baseline publication activity – not of grant funding.     

The Unbearable Lightness of Health Science Reporting: A Week Examining Italian Print Media

Background

Although being an important source of science news information to the public, print news media have often been criticized in their credibility. Health-related content of press media articles has been examined by many studies underlining that information about benefits, risks and costs are often incomplete or inadequate and financial conflicts of interest are rarely reported. However, these studies have focused their analysis on very selected science articles. The present research aimed at adopting a wider explorative approach, by analysing all types of health science information appearing on the Italian national press in one-week period. Moreover, we attempted to score the balance of the articles.

Methodology/Principal Findings

We collected 146 health science communication articles defined as articles aiming at improving the reader''s knowledge on health from a scientific perspective. Articles were evaluated by 3 independent physicians with respect to different divulgation parameters: benefits, costs, risks, sources of information, disclosure of financial conflicts of interest and balance. Balance was evaluated with regard to exaggerated or non correct claims. The selected articles appeared on 41 Italian national daily newspapers and 41 weekly magazines, representing 89% of national circulation copies: 97 articles (66%) covered common medical treatments or basic scientific research and 49 (34%) were about new medical treatments, procedures, tests or products. We found that only 6/49 (12%) articles on new treatments, procedures, tests or products mentioned costs or risks to patients. Moreover, benefits were always maximized and in 16/49 cases (33%) they were presented in relative rather than absolute terms. The majority of stories (133/146, 91%) did not report any financial conflict of interest. Among these, 15 were shown to underreport them (15/146, 9.5%), as we demonstrated that conflicts of interest did actually exist. Unbalanced articles were 27/146 (18%). Specifically, the probability of unbalanced reporting was significantly increased in stories about a new treatment, procedure, test or product (22/49, 45%), compared to stories covering common treatments or basic scientific research (5/97, 5%) (risk ratio, 8.72).

Conclusions/Significance

Consistent with prior research on health science communication in other countries, we report undisclosed costs and risks, emphasized benefits, unrevealed financial conflicts of interest and exaggerated claims in Italian print media. In addition, we show that the risk for a story about a new medical approach to be unbalanced is almost 9 times higher with respect to stories about any other kind of health science-related topics. These findings raise again the fundamental issue whether popular media is detrimental rather than useful to public health.     

Big Science vs. Little Science: How Scientific Impact Scales with Funding

Agencies that fund scientific research must choose: is it more effective to give large grants to a few elite researchers, or small grants to many researchers? Large grants would be more effective only if scientific impact increases as an accelerating function of grant size. Here, we examine the scientific impact of individual university-based researchers in three disciplines funded by the Natural Sciences and Engineering Research Council of Canada (NSERC). We considered four indices of scientific impact: numbers of articles published, numbers of citations to those articles, the most cited article, and the number of highly cited articles, each measured over a four-year period. We related these to the amount of NSERC funding received. Impact is positively, but only weakly, related to funding. Researchers who received additional funds from a second federal granting council, the Canadian Institutes for Health Research, were not more productive than those who received only NSERC funding. Impact was generally a decelerating function of funding. Impact per dollar was therefore lower for large grant-holders. This is inconsistent with the hypothesis that larger grants lead to larger discoveries. Further, the impact of researchers who received increases in funding did not predictably increase. We conclude that scientific impact (as reflected by publications) is only weakly limited by funding. We suggest that funding strategies that target diversity, rather than “excellence”, are likely to prove to be more productive.     

Quantitative analysis of research topics and public concern on V. velutina as invasive species in Asian and European countries

We analyzed the research topics and public concern on Vespa velutina as an invasive species in Asian and European countries quantitatively though digitized texts such as scientometrics and conservation culturomics. We collected 126 scientific publications of V. velutina published during the last three decades, and 423 news articles on V. velutina in South Korea. In total, 22 countries have contributed to scientific research on V. velutina. Research topics were clearly classified to studies on taxonomy and invasion of V. velutina. In South Korea, people have recognized the ecological and health impact of V. velutina. Although scientific and public interest in V. velutina has increased since 2000, research topics may be narrow and information of news articles may be simple. To achieve the sustainable management of V. velutina, multilateral studies including environmental, social, and economic research on V. velutina are necessary.     

Species Distribution Models and Impact Factor Growth in Environmental Journals: Methodological Fashion or the Attraction of Global Change Science

In this work, I evaluate the impact of species distribution models (SDMs) on the current status of environmental and ecological journals by asking the question to which degree development of SDMs in the literature is related to recent changes in the impact factors of ecological journals. The hypothesis evaluated states that research fronts are likely to attract research attention and potentially drive citation patterns, with journals concentrating papers related to the research front receiving more attention and benefiting from faster increases in their impact on the ecological literature. My results indicate a positive relationship between the number of SDM related articles published in a journal and its impact factor (IF) growth during the period 2000–09. However, the percentage of SDM related papers in a journal was strongly and positively associated with the percentage of papers on climate change and statistical issues. The results support the hypothesis that global change science has been critical in the development of SDMs and that interest in climate change research in particular, rather than the usage of SDM per se, appears as an important factor behind journal IF increases in ecology and environmental sciences. Finally, our results on SDM application in global change science support the view that scientific interest rather than methodological fashion appears to be the major driver of research attraction in the scientific literature.     

Programas para combatir la soledad en las personas mayores en el ámbito institucionalizado: una revisión de la literatura científica

Loneliness is a problem that affects a very high percentage of older people, and can lead to physical and/or mental illness. This article aims to present a systematic review of programs to combat loneliness directed towards the institutionalised elderly. Experimental and quasi-experimental articles published between 2000 and 2016 were included in the review. An analysis was carried out on the quality of the methodological evidence of each of the articles. The programs analysed included animal-assisted, contacts with families through videoconferencing, gardening and/or horticulture workshops, reminiscence therapies, humour therapies, and cognitive interventions. The results show that intervention programs are effective to reduce the feeling of loneliness. The methodological limitations of the studies are discussed and future research is recommended.     

Citizen science and wildlife biology: Synergies and challenges

Citizen science (CS) has evolved over the past decades as a working method involving interested citizens in scientific research, for example by reporting observations, taking measurements or analysing data. In the past, research on animal behaviour has been benefitting from contributions of citizen scientists mainly in the field of ornithology but the full potential of CS in ecological and behavioural sciences is surely still untapped. Here, we present case studies that successfully applied CS to research projects in wildlife biology and discuss potentials and challenges experienced. Our case studies cover a broad range of opportunities: large‐scale CS projects with interactive online tools on bird song dialects, engagement of stakeholders as citizen scientists to reduce human–wildlife conflicts, involvement of students of primary and secondary schools in CS projects as well as collaboration with the media leading to successful recruitment of citizen scientists. Each case study provides a short overview of the scientific questions and how they were approached to showcase the potentials and challenges of CS in wildlife biology. Based on the experience of the case studies, we highlight how CS may support research in wildlife biology and emphasise the value of fostering communication in CS to improve recruitment of participants and to facilitate learning and mutual trust among different groups of interest (e.g., researchers, stakeholders, students). We further show how specific training for the participants may be needed to obtain reliable data. We consider CS as a suitable tool to enhance research in wildlife biology through the application of open science procedures (i.e., open access to articles and the data on publicly available repositories) to support transparency and sharing experiences.     

The Measurement of the Effect on Citation Inequality of Differences in Citation Practices across Scientific Fields

This paper has two aims: (i) to introduce a novel method for measuring which part of overall citation inequality can be attributed to differences in citation practices across scientific fields, and (ii) to implement an empirical strategy for making meaningful comparisons between the number of citations received by articles in 22 broad fields. The number of citations received by any article is seen as a function of the article’s scientific influence, and the field to which it belongs. A key assumption is that articles in the same quantile of any field citation distribution have the same degree of citation impact in their respective field. Using a dataset of 4.4 million articles published in 1998–2003 with a five-year citation window, we estimate that differences in citation practices between the 22 fields account for 14% of overall citation inequality. Our empirical strategy is based on the strong similarities found in the behavior of citation distributions. We obtain three main results. Firstly, we estimate a set of average-based indicators, called exchange rates, to express the citations received by any article in a large interval in terms of the citations received in a reference situation. Secondly, using our exchange rates as normalization factors of the raw citation data reduces the effect of differences in citation practices to, approximately, 2% of overall citation inequality in the normalized citation distributions. Thirdly, we provide an empirical explanation of why the usual normalization procedure based on the fields’ mean citation rates is found to be equally successful.