Publication Bias in Antipsychotic Trials: An Analysis of Efficacy Comparing the Published Literature to the US Food and Drug Administration Database

Background

Publication bias compromises the validity of evidence-based medicine, yet a growing body of research shows that this problem is widespread. Efficacy data from drug regulatory agencies, e.g., the US Food and Drug Administration (FDA), can serve as a benchmark or control against which data in journal articles can be checked. Thus one may determine whether publication bias is present and quantify the extent to which it inflates apparent drug efficacy.

Methods and Findings

FDA Drug Approval Packages for eight second-generation antipsychotics—aripiprazole, iloperidone, olanzapine, paliperidone, quetiapine, risperidone, risperidone long-acting injection (risperidone LAI), and ziprasidone—were used to identify a cohort of 24 FDA-registered premarketing trials. The results of these trials according to the FDA were compared with the results conveyed in corresponding journal articles. The relationship between study outcome and publication status was examined, and effect sizes derived from the two data sources were compared. Among the 24 FDA-registered trials, four (17%) were unpublished. Of these, three failed to show that the study drug had a statistical advantage over placebo, and one showed the study drug was statistically inferior to the active comparator. Among the 20 published trials, the five that were not positive, according to the FDA, showed some evidence of outcome reporting bias. However, the association between trial outcome and publication status did not reach statistical significance. Further, the apparent increase in the effect size point estimate due to publication bias was modest (8%) and not statistically significant. On the other hand, the effect size for unpublished trials (0.23, 95% confidence interval 0.07 to 0.39) was less than half that for the published trials (0.47, 95% confidence interval 0.40 to 0.54), a difference that was significant.

Conclusions

The magnitude of publication bias found for antipsychotics was less than that found previously for antidepressants, possibly because antipsychotics demonstrate superiority to placebo more consistently. Without increased access to regulatory agency data, publication bias will continue to blur distinctions between effective and ineffective drugs. Please see later in the article for the Editors'' Summary     

Bias in Spontaneous Reporting of Adverse Drug Reactions in Japan

Background

Attitudes of healthcare professionals regarding spontaneous reporting of adverse drug reactions (ADRs) in Japan are not well known, and Japan’s unique system of surveillance, called early post-marketing phase vigilance (EPPV), may affect these reporting attitudes. Our objectives were to describe potential effects of EPPV and to test whether ADR seriousness, prominence, and frequency are related to changes in reporting over time.

Methods

A manufacturer’s database of spontaneous ADR reports was used to extract data from individual case safety reports for 5 drugs subject to EPPV. The trend of reporting and the time lag between ADR onset and reporting to the manufacturer were examined. The following indices for ADRs occurring with each drug were calculated and analyzed to assess reporting trends: Serious:Non-serious ratio, High prominence:Low prominence ratio, and High frequency:Low frequency ratio.

Results

For all 5 drugs, the time lag between ADR onset and reporting to the manufacturer was shorter in the EPPV period than in the post-EPPV period. All drugs showed higher Serious:Non-serious ratios in the post-EPPV period. No specific patterns were observed for the High prominence:Low prominence ratio. The High frequency:Low frequency ratio for peginterferon alpha-2a and sevelamer hydrochloride decreased steadily throughout the study period.

Conclusions

Healthcare professionals may be more likely to report serious ADRs than to report non-serious ADRs, but the effect of event prominence on reporting trends is still unclear. Factors associated with ADR reporting attitude in Japan might be different from those in other countries because of EPPV and the involvement of medical representatives in the spontaneous reporting process. Pharmacovigilance specialists should therefore be cautious when comparing data between different time periods or different countries. Further studies are needed to elucidate the underlying mechanism of spontaneous ADR reporting in Japan.     

Introduction to the Food and Drug Administration (FDA) regulatory process

FDA oversight of medical devices, including in vitro diagnostic devices (IVDs or laboratory tests), in the United States was a direct result of the passage of the Medical Device Amendments of 1976. This law introduced a series of general controls for medical devices including registration and listing, requirements for production using good manufacturing practices, and requirements for post-market reporting of device failures. This produced for the first time a menu of laboratory tests on the market, a system to ensure these were produced consistently over time, and a mechanism for FDA to identify problems with device use and to work with companies to ensure corrective action. This law also introduced the requirement for premarket review of new versions of old devices and of fundamentally new medical devices.     

Evidence for the Selective Reporting of Analyses and Discrepancies in Clinical Trials: A Systematic Review of Cohort Studies of Clinical Trials

Background

Most publications about selective reporting in clinical trials have focussed on outcomes. However, selective reporting of analyses for a given outcome may also affect the validity of findings. If analyses are selected on the basis of the results, reporting bias may occur. The aims of this study were to review and summarise the evidence from empirical cohort studies that assessed discrepant or selective reporting of analyses in randomised controlled trials (RCTs).

Methods and Findings

A systematic review was conducted and included cohort studies that assessed any aspect of the reporting of analyses of RCTs by comparing different trial documents, e.g., protocol compared to trial report, or different sections within a trial publication. The Cochrane Methodology Register, Medline (Ovid), PsycInfo (Ovid), and PubMed were searched on 5 February 2014. Two authors independently selected studies, performed data extraction, and assessed the methodological quality of the eligible studies. Twenty-two studies (containing 3,140 RCTs) published between 2000 and 2013 were included. Twenty-two studies reported on discrepancies between information given in different sources. Discrepancies were found in statistical analyses (eight studies), composite outcomes (one study), the handling of missing data (three studies), unadjusted versus adjusted analyses (three studies), handling of continuous data (three studies), and subgroup analyses (12 studies). Discrepancy rates varied, ranging from 7% (3/42) to 88% (7/8) in statistical analyses, 46% (36/79) to 82% (23/28) in adjusted versus unadjusted analyses, and 61% (11/18) to 100% (25/25) in subgroup analyses. This review is limited in that none of the included studies investigated the evidence for bias resulting from selective reporting of analyses. It was not possible to combine studies to provide overall summary estimates, and so the results of studies are discussed narratively.

Conclusions

Discrepancies in analyses between publications and other study documentation were common, but reasons for these discrepancies were not discussed in the trial reports. To ensure transparency, protocols and statistical analysis plans need to be published, and investigators should adhere to these or explain discrepancies. Please see later in the article for the Editors'' Summary     

Initial severity and antidepressant benefits: a meta-analysis of data submitted to the Food and Drug Administration

Background

Meta-analyses of antidepressant medications have reported only modest benefits over placebo treatment, and when unpublished trial data are included, the benefit falls below accepted criteria for clinical significance. Yet, the efficacy of the antidepressants may also depend on the severity of initial depression scores. The purpose of this analysis is to establish the relation of baseline severity and antidepressant efficacy using a relevant dataset of published and unpublished clinical trials.

Methods and Findings

We obtained data on all clinical trials submitted to the US Food and Drug Administration (FDA) for the licensing of the four new-generation antidepressants for which full datasets were available. We then used meta-analytic techniques to assess linear and quadratic effects of initial severity on improvement scores for drug and placebo groups and on drug–placebo difference scores. Drug–placebo differences increased as a function of initial severity, rising from virtually no difference at moderate levels of initial depression to a relatively small difference for patients with very severe depression, reaching conventional criteria for clinical significance only for patients at the upper end of the very severely depressed category. Meta-regression analyses indicated that the relation of baseline severity and improvement was curvilinear in drug groups and showed a strong, negative linear component in placebo groups.

Conclusions

Drug–placebo differences in antidepressant efficacy increase as a function of baseline severity, but are relatively small even for severely depressed patients. The relationship between initial severity and antidepressant efficacy is attributable to decreased responsiveness to placebo among very severely depressed patients, rather than to increased responsiveness to medication.     

药物临床试验伦理审查存在的问题及对策研究

依据GCP药物临床试验管理规范,分析我国药物临床试验伦理审查工作中存在的问题,从如何保障受试者知情同意、加强伦理委员伦理知识培训、全程伦理监督及引入伦理验收、加强监管力度和完善监督体制等4个方面对药物临床试验伦理审查工作提出针对性建议,以逐步完善我国药物临床试验伦理审查工作。     

Food and Drug Administration low-level extremely-low-frequency magnetic field exposure facility

We describe the design, construction details, and performance characteristics of an exposure system designed to provide very well controlled extremely-low-frequency magnetic field exposures of in vitro samples. This system uses Helmholtz coils placed inside temperature-controlled mu-metal chambers to provide simultaneous ac and dc field exposures at any relative angle with minimal residual background field. The system has both exposed and sham-exposed chambers and is operated under computer control in such a way as to ensure blind exposure of samples.     

Associated Factors and Consequences of Risk of Bias in Randomized Controlled Trials of Yoga: A Systematic Review

Background

Bias in randomized controlled trials (RCTs) of complementary therapy interventions seems to be associated with specific factors and to potentially distort the studies’ conclusions. This systematic review assessed associated factors of risk of bias and consequences for the studies’ conclusions in RCTs of yoga as one of the most commonly used complementary therapies.

Methods

Medline/PubMed, Scopus, IndMED and the Cochrane Library were searched through February 2014 for yoga RCTs. Risk of selection bias was assessed using the Cochrane tool and regressed to a) publication year; b) country of origin; c) journal type; and d) impact factor using multiple logistic regression analysis. Likewise, the authors’ conclusions were regressed to risk of bias.

Results

A total of 312 RCTs were included. Impact factor ranged from 0.0 to 39.2 (median = 1.3); 60 RCT (19.2%) had a low risk of selection bias, and 252 (80.8%) had a high or unclear risk of selection bias. Only publication year and impact factor significantly predicted low risk of bias; RCTs published after 2001 (adjusted odds ratio (OR) = 12.6; 95% confidence interval (CI) = 1.7, 94.0; p<0.001) and those published in journals with impact factor (adjusted OR = 2.6; 95%CI = 1.4, 4.9; p = 0.004) were more likely to have low risk of bias. The authors’ conclusions were not associated with risk of bias.

Conclusions

Risk of selection bias was generally high in RCTs of yoga; although the situation has improved since the publication of the revised CONSORT statement 2001. Pre-CONSORT RCTs and those published in journals without impact factor should be handled with increased care; although risk of bias is unlikely to distort the RCTs’ conclusions.     

Potential Reporting Bias in fMRI Studies of the Brain

Background

Functional magnetic resonance imaging (fMRI) studies have reported multiple activation foci associated with a variety of conditions, stimuli or tasks. However, most of these studies used fewer than 40 participants.

Methodology

After extracting data (number of subjects, condition studied, number of foci identified and threshold) from 94 brain fMRI meta-analyses (k = 1,788 unique datasets) published through December of 2011, we analyzed the correlation between individual study sample sizes and number of significant foci reported. We also performed an analysis where we evaluated each meta-analysis to test whether there was a correlation between the sample size of the meta-analysis and the number of foci that it had identified. Correlation coefficients were then combined across all meta-analyses to obtain a summary correlation coefficient with a fixed effects model and we combine correlation coefficients, using a Fisher’s z transformation.

Principal Findings

There was no correlation between sample size and the number of foci reported in single studies (r = 0.0050) but there was a strong correlation between sample size and number of foci in meta-analyses (r = 0.62, p<0.001). Only studies with sample sizes <45 identified larger (>40) numbers of foci and claimed as many discovered foci as studies with sample sizes ≥45, whereas meta-analyses yielded a limited number of foci relative to the yield that would be anticipated from smaller single studies.

Conclusions

These results are consistent with possible reporting biases affecting small fMRI studies and suggest the need to promote standardized large-scale evidence in this field. It may also be that small studies may be analyzed and reported in ways that may generate a larger number of claimed foci or that small fMRI studies with inconclusive, null, or not very promising results may not be published at all.     

Mechanical performance of lumbar intervertebral body fusion devices: An analysis of data submitted to the Food and Drug Administration

Lumbar intervertebral body fusion devices (L-IBFDs) are intended to provide stability to promote fusion in patients with a variety of lumbar pathologies. Different L-IBFD designs have been developed to accommodate various surgical approaches for lumbar interbody fusion procedures including anterior, lateral, posterior, and transforaminal lumbar interbody fusions (ALIF, LLIF, PLIF, and TLIF, respectively). Due to design differences, there is a potential for mechanical performance differences between ALIF, LLIF, PLIF, and TLIF devices. To evaluate this, mechanical performance and device dimension data were collected from 124 Traditional 510(k) submissions to the FDA for L-IBFDs cleared for marketing from 2007 through 2016. From these submissions, mechanical test results were aggregated for seven commonly performed tests: static and dynamic axial compression, compression-shear, and torsion testing per ASTM F2077, and subsidence testing per ASTM F2267. The Kruskal-Wallis test and Wilcoxon signed-rank test were used to determine if device type (ALIF, LLIF, PLIF, TLIF) had a significant effect on mechanical performance parameters (static testing: stiffness and yield strength; dynamic testing: runout load; subsidence testing: stiffness [Kp]). Generally, ALIFs and LLIFs were found to be stiffer, stronger, and had higher subsidence resistance than PLIF and TLIF designs. These results are likely due to the larger footprints of the ALIF and LLIF devices. The relative mechanical performance and subsidence resistance can be considered when determining the appropriate surgical approach and implant for a given patient. Overall, the mechanical performance data presented here can be utilized for future L-IBFD development and design verification.