Managing Conflicts of Interest in the UK National Institute for Health and Care Excellence (NICE) Clinical Guidelines Programme: Qualitative Study

Background

There is international concern that conflicts of interest (COI) may bias clinical guideline development and render it untrustworthy. Guideline COI policies exist with the aim of reducing this bias but it is not known how such policies are interpreted and used by guideline producing organisations. This study sought to determine how conflicts of interest (COIs) are disclosed and managed by a national clinical guideline developer (NICE: the UK National Institute for Health and Care Excellence).

Methods

Qualitative study using semi-structured telephone interviews with 14 key informants: 8 senior staff of NICE’s guideline development centres and 6 chairs of guideline development groups (GDGs). We conducted a thematic analysis.

Results

Participants regard the NICE COI policy as comprehensive leading to transparent and independent guidance. The application of the NICE COI policy is, however, not straightforward and clarity could be improved. Disclosure of COI relies on self reporting and guideline developers have to take “on trust” the information they receive, certain types of COI (non-financial) are difficult to categorise and manage and disclosed COI can impact on the ability to recruit clinical experts to GDGs. Participants considered it both disruptive and stressful to exclude members from GDG meetings when required by the COI policy. Nonetheless the impact of this disruption can be minimised with good group chairing skills.

Conclusions

We consider that the successful implementation of a COI policy in clinical guideline development requires clear policies and procedures, appropriate training of GDG chairs and an evaluation of how the policy is used in practice.     

Mitochondrial cytochrome C oxidase subunit I of Manduca sexta and a comparison with other invertebrate genes

A cDNA encoding mitochondrial cytochrome c oxidase subunit I (mt COI) from Manduca sexta (Lepidoptera: Sphingidae) was cloned and sequenced. AT (adenine-thymine) content is high and codon usage is biased and likely reflects the role of mt COI in electron transport. The encoded protein is 514 amino acids long, contains seven invariant His residues observed in COIs in all organisms and would be predicted to be composed of 12 transmembrane regions.     

Survey of the Quality of Experimental Design,Statistical Analysis and Reporting of Research Using Animals

For scientific, ethical and economic reasons, experiments involving animals should be appropriately designed, correctly analysed and transparently reported. This increases the scientific validity of the results, and maximises the knowledge gained from each experiment. A minimum amount of relevant information must be included in scientific publications to ensure that the methods and results of a study can be reviewed, analysed and repeated. Omitting essential information can raise scientific and ethical concerns. We report the findings of a systematic survey of reporting, experimental design and statistical analysis in published biomedical research using laboratory animals. Medline and EMBASE were searched for studies reporting research on live rats, mice and non-human primates carried out in UK and US publicly funded research establishments. Detailed information was collected from 271 publications, about the objective or hypothesis of the study, the number, sex, age and/or weight of animals used, and experimental and statistical methods. Only 59% of the studies stated the hypothesis or objective of the study and the number and characteristics of the animals used. Appropriate and efficient experimental design is a critical component of high-quality science. Most of the papers surveyed did not use randomisation (87%) or blinding (86%), to reduce bias in animal selection and outcome assessment. Only 70% of the publications that used statistical methods described their methods and presented the results with a measure of error or variability. This survey has identified a number of issues that need to be addressed in order to improve experimental design and reporting in publications describing research using animals. Scientific publication is a powerful and important source of information; the authors of scientific publications therefore have a responsibility to describe their methods and results comprehensively, accurately and transparently, and peer reviewers and journal editors share the responsibility to ensure that published studies fulfil these criteria.     

Randomised Trial Support for Orthopaedic Surgical Procedures

We investigated the proportion of orthopaedic procedures supported by evidence from randomised controlled trials comparing operative procedures to a non-operative alternative. Orthopaedic procedures conducted in 2009, 2010 and 2011 across three metropolitan teaching hospitals were identified, grouped and ranked according to frequency. Searches of the Cochrane Central Register of Controlled Trials (CENTRAL), the Cochrane Database of Systematic Reviews (CDSR) and the Database of Abstracts of Reviews of Effects (DARE) were performed to identify RCTs evaluating the most commonly performed orthopaedic procedures. Included studies were categorised as “supportive” or “not supportive” of operative treatment. A risk of bias analysis was conducted for included studies using the Cochrane Collaboration''s Risk of Bias tool. A total of 9,392 orthopaedic procedures were performed across the index period. 94.6% (8886 procedures) of the total volume, representing the 32 most common operative procedure categories, were used for this analysis. Of the 83 included RCTs, 22.9% (19/83) were classified as supportive of operative intervention. 36.9% (3279/8886) of the total volume of procedures performed were supported by at least one RCT showing surgery to be superior to a non-operative alternative. 19.6% (1743/8886) of the total volume of procedures performed were supported by at least one low risk of bias RCT showing surgery to be superior to a non-operative alternative. The level of RCT support for common orthopaedic procedures compares unfavourably with other fields of medicine.     

Conflict of interest in clinical practice guideline development: a systematic review

Background

There is an emerging literature on the existence and effect of industry relationships on physician and researcher behavior. Much less is known, however, about the effects of these relationships and other conflicts of interest (COI) on clinical practice guideline (CPG) development and recommendations. We performed a systematic review of the prevalence of COI and its effect on CPG recommendations.

Methodology/Principal Findings

We searched Medline (1980 to March, 2011) for studies that examined the effect of COI on CPG development and/or recommendations. Data synthesis was qualitative. Twelve studies fulfilled inclusion criteria; 9 were conducted in the US. All studies reported on financial relationships of CPG authors with the pharmaceutical industry; 1 study also examined relationships with diagnostic testing and insurance companies. The majority of guidelines had authors with industry affiliations, including consultancies (authors with relationship, range 6–80%); research support (4–78%); equity/stock ownership (2–17%); or any COI (56–87%). Four studies reported multiple types of financial interactions for individual authors (number of types per author: range 2 to 10 or more). Data on the effect of COI on CPG recommendations were confined to case studies wherein authors with specific financial ties appeared to benefit from the related CPG recommendations. In a single study, few authors believed that their relationships influenced their recommendations. No studies reported on intellectual COI in CPGs.

Conclusions/Significance

There are limited data describing the high prevalence of COI among CPG authors, and only case studies of the effect of COI on CPG recommendations. Further research is needed to explore this potential source of bias.     

Outcome reporting bias in randomized trials funded by the Canadian Institutes of Health Research

Background

The reporting of outcomes within published randomized trials has previously been shown to be incomplete, biased and inconsistent with study protocols. We sought to determine whether outcome reporting bias would be present in a cohort of government-funded trials subjected to rigorous peer review.

Methods

We compared protocols for randomized trials approved for funding by the Canadian Institutes of Health Research (formerly the Medical Research Council of Canada) from 1990 to 1998 with subsequent reports of the trials identified in journal publications. Characteristics of reported and unreported outcomes were recorded from the protocols and publications. Incompletely reported outcomes were defined as those with insufficient data provided in publications for inclusion in meta-analyses. An overall odds ratio measuring the association between completeness of reporting and statistical significance was calculated stratified by trial. Finally, primary outcomes specified in trial protocols were compared with those reported in publications.

Results

We identified 48 trials with 68 publications and 1402 outcomes. The median number of participants per trial was 299, and 44% of the trials were published in general medical journals. A median of 31% (10th–90th percentile range 5%–67%) of outcomes measured to assess the efficacy of an intervention (efficacy outcomes) and 59% (0%–100%) of those measured to assess the harm of an intervention (harm outcomes) per trial were incompletely reported. Statistically significant efficacy outcomes had a higher odds than nonsignificant efficacy outcomes of being fully reported (odds ratio 2.7; 95% confidence interval 1.5–5.0). Primary outcomes differed between protocols and publications for 40% of the trials.

Interpretation

Selective reporting of outcomes frequently occurs in publications of high-quality government-funded trials.Selective reporting of results from randomized trials can occur either at the level of end points within published studies (outcome reporting bias)¹ or at the level of entire trials that are selectively published (study publication bias).² Outcome reporting bias has previously been demonstrated in a broad cohort of published trials approved by a regional ethics committee.¹ The Canadian Institutes of Health Research (CIHR) — the primary federal funding agency, known before 2000 as the Medical Research Council of Canada (MRC) — recognized the need to address this issue and conducted an internal review process in 2002 to evaluate the reporting of results from its funded trials. The primary objectives were to determine (a) the prevalence of incomplete outcome reporting in journal publications of randomized trials; (b) the degree of association between adequate outcome reporting and statistical significance; and (c) the consistency between primary outcomes specified in trial protocols and those specified in subsequent journal publications.     

Evidence for the Selective Reporting of Analyses and Discrepancies in Clinical Trials: A Systematic Review of Cohort Studies of Clinical Trials

Background

Most publications about selective reporting in clinical trials have focussed on outcomes. However, selective reporting of analyses for a given outcome may also affect the validity of findings. If analyses are selected on the basis of the results, reporting bias may occur. The aims of this study were to review and summarise the evidence from empirical cohort studies that assessed discrepant or selective reporting of analyses in randomised controlled trials (RCTs).

Methods and Findings

A systematic review was conducted and included cohort studies that assessed any aspect of the reporting of analyses of RCTs by comparing different trial documents, e.g., protocol compared to trial report, or different sections within a trial publication. The Cochrane Methodology Register, Medline (Ovid), PsycInfo (Ovid), and PubMed were searched on 5 February 2014. Two authors independently selected studies, performed data extraction, and assessed the methodological quality of the eligible studies. Twenty-two studies (containing 3,140 RCTs) published between 2000 and 2013 were included. Twenty-two studies reported on discrepancies between information given in different sources. Discrepancies were found in statistical analyses (eight studies), composite outcomes (one study), the handling of missing data (three studies), unadjusted versus adjusted analyses (three studies), handling of continuous data (three studies), and subgroup analyses (12 studies). Discrepancy rates varied, ranging from 7% (3/42) to 88% (7/8) in statistical analyses, 46% (36/79) to 82% (23/28) in adjusted versus unadjusted analyses, and 61% (11/18) to 100% (25/25) in subgroup analyses. This review is limited in that none of the included studies investigated the evidence for bias resulting from selective reporting of analyses. It was not possible to combine studies to provide overall summary estimates, and so the results of studies are discussed narratively.

Conclusions

Discrepancies in analyses between publications and other study documentation were common, but reasons for these discrepancies were not discussed in the trial reports. To ensure transparency, protocols and statistical analysis plans need to be published, and investigators should adhere to these or explain discrepancies. Please see later in the article for the Editors'' Summary     

Conflict of Interest Disclosures for Clinical Practice Guidelines in the National Guideline Clearinghouse

Background

Conflict of interest (COI) is an important potential source of bias in the development of clinical practice guidelines (CPGs) and high rates of COI among guideline authors have been reported in the past. Our objective was to report current rates of disclosure and specific author COI across a broad range of CPGs and to examine whether CPG characteristics were associated with the presence of disclosures and of conflicts.

Methods and Findings

We selected a random sample of 250 CPGs listed in the National Guideline Clearinghouse on November 22, 2010, representing approximately a 10% sample of guidelines listed in the NGC on that date. We abstracted information on author COI from each CPG and examined predictors of the disclosures and COI using a logistic generalized estimating equation regression model. 87% of organizations developing guidelines had a CPG-specific policy, however, 40% of CPGs did not indicate that they had collected disclosures from guideline authors. In addition, 42% of organizations that did collect author disclosures did not have those disclosures available in the public domain. Of CPGs where we had disclosures for all authors, 60% had one or more authors with a conflict. On average, 28% of the authors of CPGs with available disclosures had a COI. Guidelines that were published in journals with an impact factor greater than 5.0 were more likely to have one or more authors with a COI than guidelines not published in journals.

Conclusions

Rates of disclosure of author COI and the public availability of that information are unacceptably low, however rates of COI among guideline authors may have decreased in recent years. Continued efforts are needed to establish and enforce optimal COI policies in clinical practice guideline development in order to minimize the risk of bias associated with those conflicts.     

Cholesterol lowering trials in coronary heart disease: frequency of citation and outcome.

OBJECTIVE--To see if the claim that lowering cholesterol values prevents coronary heart disease is true or if it is based on citation of supportive trials only. DESIGN--Comparison of frequency of citation with outcome of all controlled cholesterol lowering trials using coronary heart disease or death, or both, as end point. SUBJECTS--22 controlled cholesterol lowering trials. RESULTS--Trials considered by their directors as supportive of the contention were cited almost six times more often than others, according to Science Citation Index. Apart from trials discontinued because of alleged side effects of treatment, unsupportive trials were not cited after 1970, although their number almost equalled the number considered supportive. In three supportive reviews the outcome of the selected trials was more favourable than the outcome of the excluded and ignored trials. In the 22 controlled cholesterol lowering trials studied total and coronary heart disease mortality was not changed significantly either overall or in any subgroup. A statistically significant 0.32% reduction in non-fatal coronary heart disease seemed to be due to bias as event frequencies were unrelated to trial length and to mean net reduction in cholesterol value; individual changes in cholesterol values were unsystematically or not related to outcome; and after correction for a small but significant increase in non-medical deaths in the intervention groups total mortality remained unchanged (odds ratio 1.02). CONCLUSIONS--Lowering serum cholesterol concentrations does not reduce mortality and is unlikely to prevent coronary heart disease. Claims of the opposite are based on preferential citation of supportive trials.     

Systematic Review of the Empirical Evidence of Study Publication Bias and Outcome Reporting Bias — An Updated Review

Background

The increased use of meta-analysis in systematic reviews of healthcare interventions has highlighted several types of bias that can arise during the completion of a randomised controlled trial. Study publication bias and outcome reporting bias have been recognised as a potential threat to the validity of meta-analysis and can make the readily available evidence unreliable for decision making.

Methodology/Principal Findings

In this update, we review and summarise the evidence from cohort studies that have assessed study publication bias or outcome reporting bias in randomised controlled trials. Twenty studies were eligible of which four were newly identified in this update. Only two followed the cohort all the way through from protocol approval to information regarding publication of outcomes. Fifteen of the studies investigated study publication bias and five investigated outcome reporting bias. Three studies have found that statistically significant outcomes had a higher odds of being fully reported compared to non-significant outcomes (range of odds ratios: 2.2 to 4.7). In comparing trial publications to protocols, we found that 40–62% of studies had at least one primary outcome that was changed, introduced, or omitted. We decided not to undertake meta-analysis due to the differences between studies.

Conclusions

This update does not change the conclusions of the review in which 16 studies were included. Direct empirical evidence for the existence of study publication bias and outcome reporting bias is shown. There is strong evidence of an association between significant results and publication; studies that report positive or significant results are more likely to be published and outcomes that are statistically significant have higher odds of being fully reported. Publications have been found to be inconsistent with their protocols. Researchers need to be aware of the problems of both types of bias and efforts should be concentrated on improving the reporting of trials.     

Systematic review of the empirical evidence of study publication bias and outcome reporting bias

Background

The increased use of meta-analysis in systematic reviews of healthcare interventions has highlighted several types of bias that can arise during the completion of a randomised controlled trial. Study publication bias has been recognised as a potential threat to the validity of meta-analysis and can make the readily available evidence unreliable for decision making. Until recently, outcome reporting bias has received less attention.

Methodology/Principal Findings

We review and summarise the evidence from a series of cohort studies that have assessed study publication bias and outcome reporting bias in randomised controlled trials. Sixteen studies were eligible of which only two followed the cohort all the way through from protocol approval to information regarding publication of outcomes. Eleven of the studies investigated study publication bias and five investigated outcome reporting bias. Three studies have found that statistically significant outcomes had a higher odds of being fully reported compared to non-significant outcomes (range of odds ratios: 2.2 to 4.7). In comparing trial publications to protocols, we found that 40–62% of studies had at least one primary outcome that was changed, introduced, or omitted. We decided not to undertake meta-analysis due to the differences between studies.

Conclusions

Recent work provides direct empirical evidence for the existence of study publication bias and outcome reporting bias. There is strong evidence of an association between significant results and publication; studies that report positive or significant results are more likely to be published and outcomes that are statistically significant have higher odds of being fully reported. Publications have been found to be inconsistent with their protocols. Researchers need to be aware of the problems of both types of bias and efforts should be concentrated on improving the reporting of trials.     

The methodological quality of 176,620 randomized controlled trials published between 1966 and 2018 reveals a positive trend but also an urgent need for improvement

Many randomized controlled trials (RCTs) are biased and difficult to reproduce due to methodological flaws and poor reporting. There is increasing attention for responsible research practices and implementation of reporting guidelines, but whether these efforts have improved the methodological quality of RCTs (e.g., lower risk of bias) is unknown. We, therefore, mapped risk-of-bias trends over time in RCT publications in relation to journal and author characteristics. Meta-information of 176,620 RCTs published between 1966 and 2018 was extracted. The risk-of-bias probability (random sequence generation, allocation concealment, blinding of patients/personnel, and blinding of outcome assessment) was assessed using a risk-of-bias machine learning tool. This tool was simultaneously validated using 63,327 human risk-of-bias assessments obtained from 17,394 RCTs evaluated in the Cochrane Database of Systematic Reviews (CDSR). Moreover, RCT registration and CONSORT Statement reporting were assessed using automated searches. Publication characteristics included the number of authors, journal impact factor (JIF), and medical discipline. The annual number of published RCTs substantially increased over 4 decades, accompanied by increases in authors (5.2 to 7.8) and institutions (2.9 to 4.8). The risk of bias remained present in most RCTs but decreased over time for allocation concealment (63% to 51%), random sequence generation (57% to 36%), and blinding of outcome assessment (58% to 52%). Trial registration (37% to 47%) and the use of the CONSORT Statement (1% to 20%) also rapidly increased. In journals with a higher impact factor (>10), the risk of bias was consistently lower with higher levels of RCT registration and the use of the CONSORT Statement. Automated risk-of-bias predictions had accuracies above 70% for allocation concealment (70.7%), random sequence generation (72.1%), and blinding of patients/personnel (79.8%), but not for blinding of outcome assessment (62.7%). In conclusion, the likelihood of bias in RCTs has generally decreased over the last decades. This optimistic trend may be driven by increased knowledge augmented by mandatory trial registration and more stringent reporting guidelines and journal requirements. Nevertheless, relatively high probabilities of bias remain, particularly in journals with lower impact factors. This emphasizes that further improvement of RCT registration, conduct, and reporting is still urgently needed.

Many randomized controlled trials (RCTs) are biased and difficult to reproduce due to methodological flaws and poor reporting. Analysis of 176,620 RCTs published between 1966 and 2018 reveals that the risk of bias in RCTs generally decreased. Nevertheless, relatively high probabilities of bias remain, showing that further improvement of RCT registration, conduct, and reporting is still urgently needed.     

Reduction of diabetes-induced oxidative stress by phosphodiesterase inhibitors in rats

Increased oxidative stress has been suggested to be involved in the pathogenesis and progression of diabetic tissue damage. The aim of this study was to investigate the effect of different phosphodiesterase inhibitors on lipid peroxidation and total antioxidant capacity (TAC) of plasma in streptozotocin-induced diabetic rats (Rattus norvegicus). Rats became diabetic by a single administration of streptozotocin (STZ, 45 mg/kg). The effects of 15-days treatment by milrinone, sildenafil, and theophylline as cyclic-AMP and -GMP phosphodiesterase inhibitors (PDEIs) on diabetes-induced oxidative stress were studied. The levels of glucose, malonedialdehyde (MDA) the by product of lipid peroxides, and TAC (FRAP test) were estimated in plasma of control and experimental groups of rats. A significant increase in the levels of plasma glucose, and MDA and a concomitant decrease in the levels of TAC were observed in diabetic rats. These alterations were reverted back to near normal level after the treatment with PDEIs. Treatment of diabetic rats by PDEIs reduced MDA levels and increased TAC in the order of milrinone>sildenafil>theophylline. In conclusion, the present investigation show that PDIS possesses antioxidant activities, which may be attributed to their enhancing effect on cellular cyclic nucleotides contributing to the protection against oxidative stress in streptozotocin-induced diabetes. Exact mechanism of protective actions of cAMP- and cGMP-phosphodiesterase remains to be elucidated by further studies. This finding may suggest a place for PDEIs in maintaining health in diabetes.     

Utility and evolution of cytochrome b in insects

Cytochrome b (cyt-b) is widely used in molecular phylogenetic studies of vertebrate, but not invertebrate, taxa. To determine whether this situation is an historical accident or reflects the utility of cyt-b, we compared the abilities of cyt-b, COI, and one nuclear ribosomal gene region (D1 of 28S) to recover intergeneric relationships within the tiger moth tribes Ctenuchini and Euchromiini. Additionally, we compared the rate of sequence and amino acid evolution of cyt-b across insects. Cytochrome b had the same level of sequence variation and A/T bias as COI, but was less useful for recovering intergeneric relationships. The total evidence tree casts doubt on the traditional taxonomy of the group. For the class Insecta, we found that functional conservation of amino acids occurs for the same regions as those found in vertebrates with the exception of Mallophaga (lice). Lice have an accelerated rate of nonsynonymous substitutions. Accelerated rate of cyt-b nucleotide and amino acid evolution in Apidae (bees) may be correlated with increased metabolic rates associated with facultative endothermy (= heterothermy).     

Impact of reporting bias in network meta-analysis of antidepressant placebo-controlled trials

Background

Indirect comparisons of competing treatments by network meta-analysis (NMA) are increasingly in use. Reporting bias has received little attention in this context. We aimed to assess the impact of such bias in NMAs.

Methods

We used data from 74 FDA-registered placebo-controlled trials of 12 antidepressants and their 51 matching publications. For each dataset, NMA was used to estimate the effect sizes for 66 possible pair-wise comparisons of these drugs, the probabilities of being the best drug and ranking the drugs. To assess the impact of reporting bias, we compared the NMA results for the 51 published trials and those for the 74 FDA-registered trials. To assess how reporting bias affecting only one drug may affect the ranking of all drugs, we performed 12 different NMAs for hypothetical analysis. For each of these NMAs, we used published data for one drug and FDA data for the 11 other drugs.

Findings

Pair-wise effect sizes for drugs derived from the NMA of published data and those from the NMA of FDA data differed in absolute value by at least 100% in 30 of 66 pair-wise comparisons (45%). Depending on the dataset used, the top 3 agents differed, in composition and order. When reporting bias hypothetically affected only one drug, the affected drug ranked first in 5 of the 12 NMAs but second (n = 2), fourth (n = 1) or eighth (n = 2) in the NMA of the complete FDA network.

Conclusions

In this particular network, reporting bias biased NMA-based estimates of treatments efficacy and modified ranking. The reporting bias effect in NMAs may differ from that in classical meta-analyses in that reporting bias affecting only one drug may affect the ranking of all drugs.     

Conflicts of Interest among Authors of Clinical Practice Guidelines for Glycemic Control in Type 2 Diabetes Mellitus

Background

Conflict of interest (COI) is an important potential source of bias in the development of clinical practice guidelines (CPGs).

Objectives

To examine rates of disclosure of COI, including financial interests in companies that manufacture drugs that are recommended in CPGs on glycemic control in type 2 diabetes mellitus, and to explore the relationship between recommendations for specific drugs in a guideline and author COI.

Methods

We identified a cohort of relevant guidelines from the National Guideline Clearinghouse (NGC) and abstracted COI disclosures from all guideline authors for this observational, cross-sectional study. We determined which hypoglycemic drugs were recommended in each guideline, and explored the relationship between specific disclosures and whether a drug was recommended.

Results

Among 13 included guidelines, the percentage of authors with one or more financial disclosures varied from 0 to 94% (mean 44.2%), and was particularly high for two US-based guidelines (91% and 94%). Three guidelines disclosed no author financial COI. The percentage of authors with disclosures of financial interests in manufacturers of recommended drugs was also high (mean 30%). On average, 56% of manufacturers of patented drugs recommended in each guideline had one or more authors with a financial interest in their company. We did not find a significant relationship between financial interests and whether a drug was recommended in our sample; US-based guidelines were more likely to make recommendations for a specific drug compared to non-US based guidelines.

Discussion

Authors of this cohort of guidelines have financial interests directly related to the drugs that they are recommending. Although we did not find an association between author COI and drugs recommended in these guidelines and we cannot draw conclusions about the validity of the recommendations, the credibility of many of these guidelines is in doubt.     

Multiplexed and reiterative fluorescence labeling via DNA circuitry

A class of reactive DNA circuits was adapted as erasable molecular imaging probes that allow fluorescent reporting complexes to be assembled and disassembled on a biological specimen. Circuit reactions are sequence-dependent and therefore facilitate multiplexed (multicolor) detection. Yet, the ability to disassemble reporting complexes also allows fluorophores to be removed and new circuit complexes to be used to label additional markers. Thus, these probes present opportunities to increase the total number of molecular targets that can be visualized on a biological sample by allowing multiple rounds of fluorescence microscopy to be performed.