Linking ClinicalTrials.gov and PubMed to Track Results of Interventional Human Clinical Trials

Objective

In an effort to understand how results of human clinical trials are made public, we analyze a large set of clinical trials registered at ClinicalTrials.gov, the world’s largest clinical trial registry.

Materials and Methods

We considered two trial result artifacts: (1) existence of a trial result journal article that is formally linked to a registered trial or (2) the deposition of a trial’s basic summary results within the registry.

Results

The study sample consisted of 8907 completed, interventional, phase 2-or-higher clinical trials that were completed in 2006-2009. The majority of trials (72.2%) had no structured trial-article link present. A total of 2367 trials (26.6%) deposited basic summary results within the registry. Of those , 969 trials (10.9%) were classified as trials with extended results and 1398 trials (15.7%) were classified as trials with only required basic results. The majority of the trials (54.8%) had no evidence of results, based on either linked result articles or basic summary results (silent trials), while a minimal number (9.2%) report results through both registry deposition and publication.

Discussion

Our study analyzes the body of linked knowledge around clinical trials (which we refer to as the “trialome”). Our results show that most trials do not report results and, for those that do, there is minimal overlap in the types of reporting. We identify several mechanisms by which the linkages between trials and their published results can be increased.

Conclusion

Our study shows that even when combining publications and registry results, and despite availability of several information channels, trial sponsors do not sufficiently meet the mandate to inform the public either via a linked result publication or basic results submission.     

The Definition of Placebo in the Informed Consent Forms of Clinical Trials

Aim

Lack of knowledge concerning the nature of placebo and why it is necessary may influence the participation of patients in clinical trials. The objective of the present study is to review how placebo is described in written information for participants in clinical trials to be evaluated by a Human Research Ethics Committee.

Methods

All research protocols submitted for evaluation in a Spanish hospital during 2007–2013 were reviewed. The main characteristics of the studies using a placebo were collected. Three authors read each of them to determine how the term “placebo” was explained and if there was any comment on its efficacy and safety.

Results

Two thousand seven-hundred and forty research protocols were evaluated, of which three hundred and fifty-nine used a placebo. Pharmaceutical companies sponsored most placebo-controlled clinical trials (91.9%), and phase III studies were the commonest (59.9%). Oncology (15.0%), cardiology (14.2%), and neurology (13.1%) made the greatest contributions. A review of the informed consent forms showed that placebo was described in a similar manner in most studies: the explanation was limited to between four and eight words. Very few gave information about the risks of its use or adverse reactions from its administration. None of the studies provided details about the placebo effect. And 23 lacked any information about placebo at all.

Conclusions

Explanations about placebo in informed consent forms is often scarce, and information about the placebo effect and associated risks are absent. This situation may influence a full understanding of placebo by participants in clinical trials and might reduce their informed decision to participate.     

All Nations Depend on the Global Knowledge Pool – Analysis of Country of Origin of Studies Used for Health Technology Assessments in Germany

Background

Health Technology Assessments (HTAs) are used to inform decision-making and their usefulness depends on the quality and relevance of research and specific studies for health-policy decisions. Little is known about the country of origin of studies used for HTAs.

Objective

To investigate which countries have made the largest contributions to inform health policy decisions through studies included in HTAs in Germany.

Methods

The country of origin was extracted from all studies included in HTAs of the German Institute for Quality and Efficiency in Health Care, (IQWiG), published from 2/2006 to 9/2010. Studies were ranked according to the total number of studies per country, adjusted for population size, gross domestic product (GDP), and total health expenditure.

Results

1087 studies were included in 54 HTA reports. Studies were assigned to 45 countries. Most of the studies (27%) originated from the United States (USA), 18% were multinational, followed by 7% from the United Kingdom (UK) and 5% from Germany. Nordic countries led the ranking when adjusting for population size/million (ranks 1-3,6,9/45 countries), GDP/billion US$ (1,2,5,9,14/45), or health expenditure/billion US$ (1,3,5,12,13/45). The relative contribution of the UK was stable in the analyses when adjusted for population size (7/45), GDP (7/45), and health expenditure (9/45), whereas the USA (13, 18, and 30/45) and Germany (17, 19, and 21/45) dropped in the ranking.

Conclusions

More than half of the studies relevant for evidence-informed decision-making in Germany originated from the USA, followed by multinational research and the UK. Only 5% of the studies originated from Germany. According to our findings, there appears to be some discrepancy between the use of globally generated evidence and the contribution to the knowledge pool by individual countries.     

Methodological Rigour and Transparency of Clinical Practice Guidelines Developed by Neurology Professional Societies in Croatia

Background

Clinical practice guidelines are systematically created documents that summarize knowledge and assist in delivering high-quality medicine by identifying evidence that supports best clinical care. They are produced not only by international professional groups but also by local professionals to address locally-relevant clinical practice. We evaluated the methodological rigour and transparency of guideline development in neurology formulated by professionals in a local medical community.

Methods

We analyzed clinical guidelines in neurology publicly available at the web-site of the Physicians’ Assembly in Croatia in 2012: 6 guidelines developed by Croatian authors and 1 adapted from the European Federation of Neurological Societies. The quality was assessed by 2 independent evaluators using the AGREE II instrument. We also conducted a search of the Cochrane Library to identify potential changes in recommendation from Cochrane systematic reviews included in guideline preparation.

Results

The methodological quality of the guidelines greatly varied across different domains. „Scope and Purpose” and „Clarity of Presentation“ domains received high scores (100% [95% confidence interval (CI) 98.5–100] and 97% [77.9–100], respectively), the lowest scores were in “Stakeholder Involvement“ (19% [15.5–34.6]) and “Editorial Independence” (0% [0–19.2]). Conclusions of 3 guidelines based on Cochrane systematic reviews were confirmed in updated versions and one update provided new information on the effectiveness of another antidepressant. Two Cochrane reviews used in guidelines were withdrawn and split into new reviews and their findings are now considered to be out of date.

Conclusion

Neurological guidelines used in Croatia differ in structure and their methodological quality. We recommend to national societies and professional groups to develop a more systematic and rigorous approach to the development of the guidelines, timely inclusion of best evidences and an effort to involve target users and patients in the guideline development procedures.     

Impact of Inclusion of Industry Trial Results Registries as an Information Source for Systematic Reviews

Background

Clinical trial results registries may contain relevant unpublished information. Our main aim was to investigate the potential impact of the inclusion of reports from industry results registries on systematic reviews (SRs).

Methods

We identified a sample of 150 eligible SRs in PubMed via backward selection. Eligible SRs investigated randomized controlled trials of drugs and included at least 2 bibliographic databases (original search date: 11/2009). We checked whether results registries of manufacturers and/or industry associations had also been searched. If not, we searched these registries for additional trials not considered in the SRs, as well as for additional data on trials already considered. We reanalysed the primary outcome and harm outcomes reported in the SRs and determined whether results had changed. A “change” was defined as either a new relevant result or a change in the statistical significance of an existing result. We performed a search update in 8/2013 and identified a sample of 20 eligible SRs to determine whether mandatory results registration from 9/2008 onwards in the public trial and results registry ClinicalTrials.gov had led to its inclusion as a standard information source in SRs, and whether the inclusion rate of industry results registries had changed.

Results

133 of the 150 SRs (89%) in the original analysis did not search industry results registries. For 23 (17%) of these SRs we found 25 additional trials and additional data on 31 trials already included in the SRs. This additional information was found for more than twice as many SRs of drugs approved from 2000 as approved beforehand. The inclusion of the additional trials and data yielded changes in existing results or the addition of new results for 6 of the 23 SRs. Of the 20 SRs retrieved in the search update, 8 considered ClinicalTrials.gov or a meta-registry linking to ClinicalTrials.gov, and 1 considered an industry results registry.

Conclusion

The inclusion of industry and public results registries as an information source in SRs is still insufficient and may result in publication and outcome reporting bias. In addition to an essential search in ClinicalTrials.gov, authors of SRs should consider searching industry results registries.     

The Quality of Registration of Clinical Trials: Still a Problem

Introduction

The benefits of clinical trials registration include improved transparency on clinical trials for healthcare workers and patients, increased accountability of trialists, the potential to address publication bias and selective reporting, and possibilities for research collaboration and prioritization. However, poor quality of information in registered records of trials has been found to undermine these benefits in the past. Trialists'' increasing experience with trial registration and recent developments in registration systems may have positively affected data quality. This study was conducted to investigate whether the quality of registration has improved.

Methods

We repeated a study from 2009, using the same methods and the same research team. A random sample of 400 records of clinical trials that were registered between 01/01/2012 and 01/01/2013 was taken from the International Clinical Trials Registry Platform (ICTRP) and assessed for the quality of information on 1) contact details, 2) interventions and 3) primary outcomes. Results were compared to the equivalent assessments from our previous study.

Results

There was a small and not statistically significant increase from 81.0% to 85.5% in the percentage of records that provided a name of a contact person. There was a significant increase from 68.7% to 74.9% in the number of records that provided either an email address or a telephone number. There was a significant increase from 44.2% to 51.9% in the number of intervention arms that were complete in registering intervention specifics. There was a significant increase from 38.2% to 57.6% in the number of primary outcomes that were specific measures with a meaningful timeframe. Approximately half of all trials continued to be retrospectively registered.

Discussion

There have been small but significant improvements in the quality of registration since 2009. Important problems with quality remain and continue to constitute an impediment to the meaningful utilization of registered trial information.     

The Method Quality of Cross-Over Studies Involved in Cochrane Systematic Reviews

Background

It is possible that cross-over studies included in current systematic reviews are being inadequately assessed, because the current risk of bias tools do not consider possible biases specific to cross-over design. We performed this study to evaluate whether this was being done in cross-over studies included in Cochrane Systematic Reviews (CSRs).

Methods

We searched the Cochrane Library (up to 2013 issue 5) for CSRs that included at least one cross-over trial. Two authors independently undertook the study selection and data extraction. A random sample of the CSRs was selected and we evaluated whether the cross-over trials in these CSRs were assessed according to criteria suggested by the Cochrane handbook. In addition we reassessed the risk of bias of these cross-over trials by a checklist developed form the Cochrane handbook.

Results

We identified 688 CSRs that included one or more cross-over studies. We chose a random sample of 60 CSRs and these included 139 cross-over studies. None of these CSRs undertook a risk of bias assessment specific for cross-over studies. In fact items specific for cross-over studies were seldom considered anywhere in quality assessment of these CSRs. When we reassessed the risk of bias, including the 3 items specific to cross-over trials, of these 139 studies, a low risk of bias was judged for appropriate cross-over design in 110(79%), carry-over effects in 48(34%) and for reporting data in all stages of the trial in 114(82%).Assessment of biases in cross-over trials could affect the GRADE assessment of a review’s findings.

Conclusion

The current Cochrane risk of bias tool is not adequate to assess cross-over studies. Items specific to cross-over trials leading to potential risk of bias are generally neglected in CSRs. A proposed check list for the evaluation of cross-over trials is provided.     

Risk of bias tool in systematic reviews/meta-analyses of acupuncture in Chinese journals

Background

Use of a risk of bias (ROB) tool has been encouraged and advocated to reviewers writing systematic reviews (SRs) and meta-analyses (MAs). Selective outcome reporting and other sources of bias are included in the Cochrane ROB tool. It is important to know how this specific tool for assessing ROB has been applied since its release. Our objectives were to evaluate whether and to what extent the new Cochrane ROB tool has been used in Chinese journal papers of acupuncture.

Methods

We searched CBM, TCM database, CJFD, CSJD, and the Wanfang Database from inception to March 2011. Two reviewers independently selected SRs that primarily focused on acupuncture and moxibustion, from which the data was extracted and analyzed.

Results

A total of 836 SRs were identified from the search, of which, 105 were included and four are awaiting assessment. Thirty-six of the 105 SRs were published before release of the Cochrane ROB tool (up to 2009). Most used the Cochrane Handbook 4.2 or Jadad''s scale for risk or quality assessment. From 2009 to March 2011 69 SRs were identified. While “risk of bias” was reported for approximately two-thirds of SRs, only two SRs mentioned use of a “risk of bias tool” in their assessment. Only 5.8% (4/69) of reviews reported information on all six domains which are involved in the ROB tool. A risk of bias graph/summary figure was provided in 2.9% (2/69) of reviews. Most SRs gave information about sequence generation, allocation concealment, blindness, and incomplete outcome data, however, few reviews (5.8%; 4/69) described selective reporting or other potential sources of bias.

Conclusions

The Cochrane “risk of bias” tool has not been used in all SRs/MAs of acupuncture published in Chinese Journals after 2008. When the ROB tool was used, reporting of relevant information was often incomplete.     

Outcome reporting bias in randomized trials funded by the Canadian Institutes of Health Research

Background

The reporting of outcomes within published randomized trials has previously been shown to be incomplete, biased and inconsistent with study protocols. We sought to determine whether outcome reporting bias would be present in a cohort of government-funded trials subjected to rigorous peer review.

Methods

We compared protocols for randomized trials approved for funding by the Canadian Institutes of Health Research (formerly the Medical Research Council of Canada) from 1990 to 1998 with subsequent reports of the trials identified in journal publications. Characteristics of reported and unreported outcomes were recorded from the protocols and publications. Incompletely reported outcomes were defined as those with insufficient data provided in publications for inclusion in meta-analyses. An overall odds ratio measuring the association between completeness of reporting and statistical significance was calculated stratified by trial. Finally, primary outcomes specified in trial protocols were compared with those reported in publications.

Results

We identified 48 trials with 68 publications and 1402 outcomes. The median number of participants per trial was 299, and 44% of the trials were published in general medical journals. A median of 31% (10th–90th percentile range 5%–67%) of outcomes measured to assess the efficacy of an intervention (efficacy outcomes) and 59% (0%–100%) of those measured to assess the harm of an intervention (harm outcomes) per trial were incompletely reported. Statistically significant efficacy outcomes had a higher odds than nonsignificant efficacy outcomes of being fully reported (odds ratio 2.7; 95% confidence interval 1.5–5.0). Primary outcomes differed between protocols and publications for 40% of the trials.

Interpretation

Selective reporting of outcomes frequently occurs in publications of high-quality government-funded trials.Selective reporting of results from randomized trials can occur either at the level of end points within published studies (outcome reporting bias)¹ or at the level of entire trials that are selectively published (study publication bias).² Outcome reporting bias has previously been demonstrated in a broad cohort of published trials approved by a regional ethics committee.¹ The Canadian Institutes of Health Research (CIHR) — the primary federal funding agency, known before 2000 as the Medical Research Council of Canada (MRC) — recognized the need to address this issue and conducted an internal review process in 2002 to evaluate the reporting of results from its funded trials. The primary objectives were to determine (a) the prevalence of incomplete outcome reporting in journal publications of randomized trials; (b) the degree of association between adequate outcome reporting and statistical significance; and (c) the consistency between primary outcomes specified in trial protocols and those specified in subsequent journal publications.     

Quantification of Errors in Ordinal Outcome Scales Using Shannon Entropy: Effect on Sample Size Calculations

Objective

Clinical trial outcomes often involve an ordinal scale of subjective functional assessments but the optimal way to quantify results is not clear. In stroke, the most commonly used scale, the modified Rankin Score (mRS), a range of scores (“Shift”) is proposed as superior to dichotomization because of greater information transfer. The influence of known uncertainties in mRS assessment has not been quantified. We hypothesized that errors caused by uncertainties could be quantified by applying information theory. Using Shannon’s model, we quantified errors of the “Shift” compared to dichotomized outcomes using published distributions of mRS uncertainties and applied this model to clinical trials.

Methods

We identified 35 randomized stroke trials that met inclusion criteria. Each trial’s mRS distribution was multiplied with the noise distribution from published mRS inter-rater variability to generate an error percentage for “shift” and dichotomized cut-points. For the SAINT I neuroprotectant trial, considered positive by “shift” mRS while the larger follow-up SAINT II trial was negative, we recalculated sample size required if classification uncertainty was taken into account.

Results

Considering the full mRS range, error rate was 26.1%±5.31 (Mean±SD). Error rates were lower for all dichotomizations tested using cut-points (e.g. mRS 1; 6.8%±2.89; overall p<0.001). Taking errors into account, SAINT I would have required 24% more subjects than were randomized.

Conclusion

We show when uncertainty in assessments is considered, the lowest error rates are with dichotomization. While using the full range of mRS is conceptually appealing, a gain of information is counter-balanced by a decrease in reliability. The resultant errors need to be considered since sample size may otherwise be underestimated. In principle, we have outlined an approach to error estimation for any condition in which there are uncertainties in outcome assessment. We provide the user with programs to calculate and incorporate errors into sample size estimation.     

Continuous versus Conventional Infusion of Amphotericin B Deoxycholate: A Meta-Analysis

Background

Treatment with Amphotericin B (AmB) deoxycholate, which is still used widely, particularly in low-resource countries, has been challenged due to nephrotoxicity. We sought to study whether continuous infusion of AmB deoxycholate reduces nephrotoxicity retaining, however, the effectiveness of the drug.

Methods

PubMed and Scopus databases were systematically searched to identify studies comparing the outcomes of patients receiving 24-h infusion of AmB (“continuous group”) and those receiving 2–6-h infusion of AmB (“conventional group”). Nephrotoxicity and all-cause mortality were the primary outcomes of the review, while treatment failure was the secondary outcome.

Results

Five studies met the inclusion criteria; one randomized controlled trial, two prospective cohort studies, and two retrospective cohort studies. The majority of patients were neutropenic with an underlying hematologic malignancy. All 5 studies (392 patients) provided data regarding the development of nephrotoxicity. A non-significant trend towards lower nephrotoxicity was observed for patients receiving continuous infusion of AmB compared with those receiving conventional infusion [RR = 0.61 (95% CI 0.36, 1.02)]. Four studies (365 patients) provided data regarding mortality; no relevant difference was detected between patients receiving continuous and those receiving conventional infusion of AmB [RR = 0.81 (95% CI 0.36, 1.83)]. Data on treatment failure of the two methods of administration was insufficient for meaningful conclusions.

Conclusion

The available evidence from mainly non-randomized studies suggests that continuous infusion of AmB deoxycholate might offer an advantage over the conventional infusion regarding the development of nephrotoxicity, without compromising patient survival. Further randomized studies are needed to investigate this issue.     

The Impact of National Institutes of Health Funding on U.S. Cardiovascular Disease Research

Background

Intense interest surrounds the recent expansion of US National Institutes of Health (NIH) budgets as part of economic stimulus legislation. However, the relationship between NIH funding and cardiovascular disease research is poorly understood, making the likely impact of this policy change unclear.

Methods

The National Library of Medicine''s PubMed database was searched for articles published from 1996 to 2006, originating from U.S. institutions, and containing the phrases “cardiolog,” “cardiovascular,” or “cardiac,” in the first author''s department. Research methodology, journal of publication, journal impact factor, and receipt of NIH funding were recorded. Differences in means and trends were tested with t-tests and linear regression, respectively, with P≤0.05 for significance.

Results

Of 117,643 world cardiovascular articles, 36,684 (31.2%) originated from the U.S., of which 10,293 (28.1%) received NIH funding. The NIH funded 40.1% of U.S. basic science articles, 20.3% of overall clinical trials, 18.1% of randomized-controlled, and 12.2% of multicenter clinical trials. NIH-funded and total articles grew significantly (65 articles/year, P<0.001 and 218 articles/year, P<0.001, respectively). The proportion of articles receiving NIH funding was stable, but grew significantly for basic science and clinical trials (0.87%/year, P<0.001 and 0.67%/year, P = 0.029, respectively). NIH-funded articles had greater journal impact factors than non NIH-funded articles (5.76 vs. 3.71, P<0.001).

Conclusions

NIH influence on U.S. cardiovascular research expanded in the past decade, during the period of NIH budget doubling. A substantial fraction of research is now directly funded and thus likely sensitive to budget fluctuations, particularly in basic science research. NIH funding predicts greater journal impact.     

Efficiency of New Dose Escalation Designs in Dose-Finding Phase I Trials of Molecularly Targeted Agents

Background

Statistical simulations have consistently demonstrated that new dose-escalation designs such as accelerated titration design (ATD) and continual reassessment method (CRM)-type designs outperform the standard “3+3” design in phase I cancer clinical trials.

Methods

We evaluated the actual efficiency of different dose escalation methods employed in first-in-human phase I clinical trials of targeted agents administered as single agents published over the last decade.

Results

Forty-nine per cent of the 84 retrieved trials used the standard “3+3” design. Newer designs used included ATD in 42%, modified CRM [mCRM] in 7%, and pharmacologically guided dose escalation in 1%. The median numbers of dose levels explored in trials using “3+3”, ATD and mCRM designs were 6, 8 and 10, respectively. More strikingly, the mean MTD to starting dose ratio appeared to be at least twice as high for trials using mCRM or ATD designs as for trials using a standard “3+3” design. Despite this, the mean number of patients exposed to a dose below the MTD was similar in trials using “3+3”, ATD and mCRM designs.

Conclusion

Our results support a more extensive implementation of innovative dose escalation designs such as mCRM and ATD in phase I cancer clinical trials of molecularly targeted agents.     

Statistical Analysis of Individual Participant Data Meta-Analyses: A Comparison of Methods and Recommendations for Practice

Background

Individual participant data (IPD) meta-analyses that obtain “raw” data from studies rather than summary data typically adopt a “two-stage” approach to analysis whereby IPD within trials generate summary measures, which are combined using standard meta-analytical methods. Recently, a range of “one-stage” approaches which combine all individual participant data in a single meta-analysis have been suggested as providing a more powerful and flexible approach. However, they are more complex to implement and require statistical support. This study uses a dataset to compare “two-stage” and “one-stage” models of varying complexity, to ascertain whether results obtained from the approaches differ in a clinically meaningful way.

Methods and Findings

We included data from 24 randomised controlled trials, evaluating antiplatelet agents, for the prevention of pre-eclampsia in pregnancy. We performed two-stage and one-stage IPD meta-analyses to estimate overall treatment effect and to explore potential treatment interactions whereby particular types of women and their babies might benefit differentially from receiving antiplatelets. Two-stage and one-stage approaches gave similar results, showing a benefit of using anti-platelets (Relative risk 0.90, 95% CI 0.84 to 0.97). Neither approach suggested that any particular type of women benefited more or less from antiplatelets. There were no material differences in results between different types of one-stage model.

Conclusions

For these data, two-stage and one-stage approaches to analysis produce similar results. Although one-stage models offer a flexible environment for exploring model structure and are useful where across study patterns relating to types of participant, intervention and outcome mask similar relationships within trials, the additional insights provided by their usage may not outweigh the costs of statistical support for routine application in syntheses of randomised controlled trials. Researchers considering undertaking an IPD meta-analysis should not necessarily be deterred by a perceived need for sophisticated statistical methods when combining information from large randomised trials.