Assessment of the Reporting Quality of Randomized Controlled Trials on the Treatment of Diabetes Mellitus with Traditional Chinese Medicine: A Systematic Review

Background

After the publication of the CONSORT 2010 statement, few studies have been conducted to assess the reporting quality of randomized clinical trials (RCTs) on treatment of diabetes mellitus with Traditional Chinese Medicine (TCM) published in Chinese journals.

Objective

To investigate the current situation of the reporting quality of RCTs in leading medical journals in China with the CONSORT 2010 statement as criteria.

Methods

The China National Knowledge Infrastructure (CNKI) electronic database was searched for RCTs on the treatment of diabetes mellitus with TCM published in the Journal of Traditional Chinese Medicine, Chinese Journal of Integrated Traditional & Western Medicine, and the China Journal of Chinese Materia Medica from January to December 2011. We excluded trials reported as “animal studies”, “in vitro studies”, “case studies”, or “systematic reviews”. The CONSORT checklist was applied by two independent raters to evaluate the reporting quality of all eligible trials after discussing and comprehending the items thoroughly. Each item in the checklist was graded as either “yes” or “no” depending on whether it had been reported by the authors.

Results

We identified 27 RCTs. According to the 37 items in the CONSORT checklist, the average reporting percentage was 45.0%, in which the average reporting percentage for the “title and abstract”, the “introduction”, the “methods”, the “results”, the “discussion” and the “other information” was 33.3%, 88.9%, 36.4%, 54.4%, 71.6% and 14.8%, respectively. In the Journal of Traditional Chinese Medicine, Chinese Journal of Integrated Traditional & Western Medicine, and the China Journal of Chinese Materia Medica the average reporting percentage was 42.2%, 56.8%, and 46.0%, respectively.

Conclusions

The reporting quality of RCTs in these three journals was insufficient to allow readers to assess the validity of the trials. We recommend that editors require authors to use the CONSORT statement when reporting their trial results as a condition of publication.     

A systematic evaluation of the impact of STRICTA and CONSORT recommendations on quality of reporting for acupuncture trials

Background

We investigated whether there had been an improvement in quality of reporting for randomised controlled trials of acupuncture since the publication of the STRICTA and CONSORT statements. We conducted a before-and-after study, comparing ratings for quality of reporting following the publication of both STRICTA and CONSORT recommendations.

Methodology and Principal Findings

Ninety peer reviewed journal articles reporting the results of acupuncture trials were selected at random from a wider sample frame of 266 papers. Papers published in three distinct time periods (1994–1995, 1999–2000 and 2004–2005) were compared. Assessment criteria were developed directly from CONSORT and STRICTA checklists. Papers were independently assessed for quality of reporting by two assessors, one of whom was blind to information which could have introduced systematic bias (e.g. date of publication). We detected a statistically significant increase in the reporting of CONSORT items for papers published in each time period measured. We did not, however, find a difference between the number of STRICTA items reported in journal articles published before and 3 to 4 years following the introduction of STRICTA recommendations.

Conclusions and Significance

The results of this study suggest that general standards of reporting for acupuncture trials have significantly improved since the introduction of the CONSORT statement in 1996, but that quality in reporting details specific to acupuncture interventions has yet to change following the more recent introduction of STRICTA recommendations. Wider targeting and revision of the guidelines is recommended.     

The Method Quality of Cross-Over Studies Involved in Cochrane Systematic Reviews

Background

It is possible that cross-over studies included in current systematic reviews are being inadequately assessed, because the current risk of bias tools do not consider possible biases specific to cross-over design. We performed this study to evaluate whether this was being done in cross-over studies included in Cochrane Systematic Reviews (CSRs).

Methods

We searched the Cochrane Library (up to 2013 issue 5) for CSRs that included at least one cross-over trial. Two authors independently undertook the study selection and data extraction. A random sample of the CSRs was selected and we evaluated whether the cross-over trials in these CSRs were assessed according to criteria suggested by the Cochrane handbook. In addition we reassessed the risk of bias of these cross-over trials by a checklist developed form the Cochrane handbook.

Results

We identified 688 CSRs that included one or more cross-over studies. We chose a random sample of 60 CSRs and these included 139 cross-over studies. None of these CSRs undertook a risk of bias assessment specific for cross-over studies. In fact items specific for cross-over studies were seldom considered anywhere in quality assessment of these CSRs. When we reassessed the risk of bias, including the 3 items specific to cross-over trials, of these 139 studies, a low risk of bias was judged for appropriate cross-over design in 110(79%), carry-over effects in 48(34%) and for reporting data in all stages of the trial in 114(82%).Assessment of biases in cross-over trials could affect the GRADE assessment of a review’s findings.

Conclusion

The current Cochrane risk of bias tool is not adequate to assess cross-over studies. Items specific to cross-over trials leading to potential risk of bias are generally neglected in CSRs. A proposed check list for the evaluation of cross-over trials is provided.     

Assessment of the Quality of Reporting in Abstracts of Randomized Controlled Trials Published in Five Leading Chinese Medical Journals

Background

Clear, transparent and sufficiently detailed abstracts of randomized trials (RCTs), published in journal articles are important because readers will often base their initial assessment of a trial on such information. However, little is known about the quality of reporting in abstracts of RCTs published in medical journals in China.

Methods

We identified RCTs abstracts from 5 five leading Chinese medical journals published between 1998 and 2007 and indexed in MEDLINE. We assessed the quality of reporting of these abstracts based on the Consolidated Standards of Reporting Trials (CONSORT) abstract checklist. We also sought to identify whether any differences exist in reporting between the Chinese and English language version of the same abstract.

Results

We identified 332 RCT abstracts eligible for examination. Overall, the abstracts we examined reported 0–8 items as designated in the CONSORT checklist. On average, three items were reported per abstract. Details of the interventions (288/332; 87%), the number of participants randomized (216/332; 65%) and study objectives (109/332; 33%) were the top three items reported. Only two RCT abstracts reported details of trial registration, no abstracts reported the method of allocation concealment and only one mentioned specifically who was blinded. In terms of the proportion of RCT abstracts fulfilling a criterion, the absolute difference (percentage points) between the Chinese and English abstracts was 10% (ranging from 0 to 25%) on average, per item.

Conclusions

The quality of reporting in abstracts of RCTs published in Chinese medical journals needs to be improved. We hope that the introduction and endorsement of the CONSORT for Abstracts guidelines by journals reporting RCTs will lead to improvements in the quality of reporting.     

Assessment of the Quality of Reporting of Randomised Controlled Trials in Otorhinolaryngologic Literature – Adherence to the CONSORT Statement

Background

Randomised Controlled Trials (RCTs) are the preferred study design when comparing therapeutical interventions in medicine. To improve clarity, consistency and transparency of reporting RCTs, the Consolidated Standards of Reporting Trials (CONSORT) statement was developed.

Objectives

(1) To assess the quality of reports and abstracts of RCTs in otorhinolaryngologic literature by using CONSORT checklists, (2) to compare the quality of reports and abstracts of otorhinolaryngologic RCTs between the top 5 general medical journals and top 5 otorhinolaryngologic journals, and (3) to formulate recommendations for authors and editors of otorhinolaryngologic (‘ENT’) journals.

Methods

Based on 2012 ISI Web of Knowledge impact factors, the top 5 general medical and ENT journals were selected. On 25 June 2014, using a highly sensitive Cochrane RCT filter and ENT filter, possibly relevant articles since January 1st, 2010 were retrieved and relevant RCTs were selected. We assessed how many CONSORT items were reported adequately in reports and abstracts and compared the two journal types.

Results

Otorhinolaryngologic RCTs (n = 15) published in general medical journals reported a mean of 92.1% (95% confidence interval: 89.5%–94.7%) of CONSORT items adequately, whereas RCTs (n = 18) published in ENT journals reported a mean of 71.8% (66.7%–76.8%) adequately (p < 0.001). For abstracts, means of 70.0% (63.7%–76.3%) and 32.3% (26.6–38.0%) were found respectively (p < 0.001). Large differences for specific items exist between the two journal types.

Conclusion

The quality of reporting of RCTs in otorhinolaryngologic journals is suboptimal. RCTs published in general medical journals have a higher quality of reporting than RCTs published in ENT journals. We recommend authors to report their trial according to the CONSORT Statement and advise editors to endorse the CONSORT Statement and implement the CONSORT Statement in the editorial process to ensure more adequate reporting of RCTs and their abstracts.     

Ten Years after the International Committee of Medical Journal Editors’ Clinical Trial Registration Initiative,One Quarter of Phase 3 Pediatric Epilepsy Clinical Trials Still Remain Unpublished: A Cross Sectional Analysis

Introduction

Although selective reporting of clinical trial results introduces bias into evidence based clinical decision making, publication bias in pediatric epilepsy is unknown today. Since there is a considerable ambiguity in the treatment of an important and common clinical problem, pediatric seizures, we assessed the public availability of results of phase 3 clinical trials that evaluated treatments of seizures in children and adolescents as a surrogate for the extent of publication bias in pediatric epilepsy.

Methods

We determined the proportion of published and unpublished study results of phase 3 clinical trials that were registered as completed on ClinicalTrials.gov. We searched ClinicalTrials.gov, PubMed, and Google Scholar for publications and contacted principal investigators or sponsors. The analysis was performed according to STROBE criteria.

Results

Considering studies that were completed before 2014 (N = 99), 75 (76%) pediatric phase 3 clinical trials were published but 24 (24%) remained unpublished. The unpublished studies concealed evidence from 4,437 patients. Mean time-to-publication was 25 SD ± 15.6 months, more than twice as long as mandated.

Conclusion

Ten years after the ICMJE’s clinical trials registration initiative there is still a considerable amount of selective reporting and delay of publication that potentially distorts the body of evidence in the treatment of pediatric seizures.     

Design,Analysis, and Reporting of Crossover Trials for Inclusion in a Meta-Analysis

Randomized crossover trials are clinical experiments in which participants are assigned randomly to a sequence of treatments and each participant serves as his/her own control in estimating treatment effect. We need a better understanding of the validity of their results to enable recommendations as to which crossover trials can be included in meta-analysis and for development of reporting guidelines.

Objective

To evaluate the characteristics of the design, analysis, and reporting of crossover trials for inclusion in a meta-analysis of treatment for primary open-angle glaucoma and to provide empirical evidence to inform the development of tools to assess the validity of the results from crossover trials and reporting guidelines.

Methods

We searched MEDLINE, EMBASE, and Cochrane’s CENTRAL register for randomized crossover trials for a systematic review and network meta-analysis we are conducting. Two individuals independently screened the search results for eligibility and abstracted data from each included report.

Results

We identified 83 crossover trials eligible for inclusion. Issues affecting the risk of bias in crossover trials, such as carryover, period effects and missing data, were often ignored. Some trials failed to accommodate the within-individual differences in the analysis. For a large proportion of the trials, the authors tabulated the results as if they arose from a parallel design. Precision estimates properly accounting for the paired nature of the design were often unavailable from the study reports; consequently, to include trial findings in a meta-analysis would require further manipulation and assumptions.

Conclusions

The high proportion of poorly reported analyses and results has the potential to affect whether crossover data should or can be included in a meta-analysis. There is pressing need for reporting guidelines for crossover trials.     

How Psychiatry Journals Support the Unbiased Translation of Clinical Research. A Cross-Sectional Study of Editorial Policies

Introduction

Reporting guidelines (e.g. CONSORT) have been developed as tools to improve quality and reduce bias in reporting research findings. Trial registration has been recommended for countering selective publication. The International Committee of Medical Journal Editors (ICMJE) encourages the implementation of reporting guidelines and trial registration as uniform requirements (URM). For the last two decades, however, biased reporting and insufficient registration of clinical trials has been identified in several literature reviews and other investigations. No study has so far investigated the extent to which author instructions in psychiatry journals encourage following reporting guidelines and trial registration.

Method

Psychiatry Journals were identified from the 2011 Journal Citation Report. Information given in the author instructions and during the submission procedure of all journals was assessed on whether major reporting guidelines, trial registration and the ICMJE’s URM in general were mentioned and adherence recommended.

Results

We included 123 psychiatry journals (English and German language) in our analysis. A minority recommend or require 1) following the URM (21%), 2) adherence to reporting guidelines such as CONSORT, PRISMA, STROBE (23%, 7%, 4%), or 3) registration of clinical trials (34%). The subsample of the top-10 psychiatry journals (ranked by impact factor) provided much better but still improvable rates. For example, 70% of the top-10 psychiatry journals do not ask for the specific trial registration number.

Discussion

Under the assumption that better reported and better registered clinical research that does not lack substantial information will improve the understanding, credibility, and unbiased translation of clinical research findings, several stakeholders including readers (physicians, patients), authors, reviewers, and editors might benefit from improved author instructions in psychiatry journals. A first step of improvement would consist in requiring adherence to the broadly accepted reporting guidelines and to trial registration.     

Quality of Reporting of Randomised Controlled Trials of Herbal Interventions in ASEAN Plus Six Countries: A Systematic Review

Background

Many randomised controlled trials (RCTs) of herbal interventions have been conducted in the ASEAN Communities. Good quality reporting of RCTs is essential for assessing clinical significance. Given the importance ASEAN placed on herbal medicines, the reporting quality of RCTs of herbal interventions among the ASEAN Communities deserved a special attention.

Objectives

To systematically review the quality of reporting of RCTs of herbal interventions conducted in the ASEAN Plus Six Countries.

Methods

Searches were performed using PubMed, EMBASE, The Cochrane Library, and Allied and Complementary Medicine (AMED), from inception through October 2013. These were limited to studies specific to humans and RCTs. Herbal species search terms were based on those listed in the National List of Essential Medicines [NLEM (Thailand, 2011)]. Studies conducted in the ASEAN Plus Six Countries, published in English were included.

Results

Seventy-one articles were identified. Thirty (42.25%) RCTs were from ASEAN Countries, whereas 41 RCTs (57.75%) were from Plus Six Group. Adherence to the recommended CONSORT checklist items for reporting of RCTs of herbal interventions among ASEAN Plus Six Countries ranged from 0% to 97.18%. Less than a quarter of the RCTs (18.31%) reported information on standardisation of the herbal products. However, the scope of our interventions of interest was limited to those developed from 20 herbal species listed in the NLEM of Thailand.

Conclusions

The present study highlights the need to improve reporting quality of RCTs of herbal interventions across ASEAN Plus Six Communities.     

Reporting quality of randomized controlled trials in patients with HIV on antiretroviral therapy: a systematic review

Background

To allow for correct evaluation of clinical trial results, readers require comprehensive, clear, and highly transparent information on the methodology used and the results obtained. This study aimed to evaluate the quality of reporting in articles on randomized controlled trials (RCTs) of antiretroviral therapy (ART) in the field of HIV/AIDS.

Methods

We searched for original articles on RCTs of ART developed in the field of HIV/AIDS in PubMed database by 5 April 2016. Searched articles were divided into three groups based on the revision year in which the Consolidated Standards of Reporting Trials (CONSORT) guidelines were published: Period 1 (1996–2001); Period 2 (2002–2010); and Period 3 (2011–2016). We evaluated the articles using the reporting rates of the 37 items in the CONSORT 2010 checklist, five items in the protocol deviation, and the three items in the ethics.

Results

Fifty-two articles were extracted and included in this study. Many of the reporting rates calculated using the CONSORT 2010 checklist showed a significantly increasing trend over the successive periods (65% in Period 1, 67% in Period 2, 79% in Period 3; p?<?0.0001). The items with reporting rates?<?50% were “the presence or absence of a protocol change and the reason for such a change,” “randomization and blinding,” and “where the full trial protocol can be accessed.” Reporting rates of deviations were as low as?<?30%, while the reporting rates for patient compliance were the highest (>80% in Period 3) among the five items. The reporting rates for obtaining informed consent and approval by the ethics committee or institutional review board were high (>88%), regardless of the time period assessed.

Conclusion

In terms of representative RCT articles in the field of HIV/AIDS, the reporting rate of the items defined by CONSORT was approximately 70%, improving over the successive CONSORT statement revision periods.

     

Accuracy of Assessment of Eligibility for Early Medical Abortion by Community Health Workers in Ethiopia,India and South Africa

Objective

To assess the accuracy of assessment of eligibility for early medical abortion by community health workers using a simple checklist toolkit.

Design

Diagnostic accuracy study.

Setting

Ethiopia, India and South Africa.

Methods

Two hundred seventeen women in Ethiopia, 258 in India and 236 in South Africa were enrolled into the study. A checklist toolkit to determine eligibility for early medical abortion was validated by comparing results of clinician and community health worker assessment of eligibility using the checklist toolkit with the reference standard exam.

Results

Accuracy was over 90% and the negative likelihood ratio <0.1 at all three sites when used by clinician assessors. Positive likelihood ratios were 4.3 in Ethiopia, 5.8 in India and 6.3 in South Africa. When used by community health workers the overall accuracy of the toolkit was 92% in Ethiopia, 80% in India and 77% in South Africa negative likelihood ratios were 0.08 in Ethiopia, 0.25 in India and 0.22 in South Africa and positive likelihood ratios were 5.9 in Ethiopia and 2.0 in India and South Africa.

Conclusion

The checklist toolkit, as used by clinicians, was excellent at ruling out participants who were not eligible, and moderately effective at ruling in participants who were eligible for medical abortion. Results were promising when used by community health workers particularly in Ethiopia where they had more prior experience with use of diagnostic aids and longer professional training. The checklist toolkit assessments resulted in some participants being wrongly assessed as eligible for medical abortion which is an area of concern. Further research is needed to streamline the components of the tool, explore optimal duration and content of training for community health workers, and test feasibility and acceptability.     

Cluster Randomised Trials in Cochrane Reviews: Evaluation of Methodological and Reporting Practice

Objective

Systematic reviews can include cluster-randomised controlled trials (C-RCTs), which require different analysis compared with standard individual-randomised controlled trials. However, it is not known whether review authors follow the methodological and reporting guidance when including these trials. The aim of this study was to assess the methodological and reporting practice of Cochrane reviews that included C-RCTs against criteria developed from existing guidance.

Methods

Criteria were developed, based on methodological literature and personal experience supervising review production and quality. Criteria were grouped into four themes: identifying, reporting, assessing risk of bias, and analysing C-RCTs. The Cochrane Database of Systematic Reviews was searched (2^nd December 2013), and the 50 most recent reviews that included C-RCTs were retrieved. Each review was then assessed using the criteria.

Results

The 50 reviews we identified were published by 26 Cochrane Review Groups between June 2013 and November 2013. For identifying C-RCTs, only 56% identified that C-RCTs were eligible for inclusion in the review in the eligibility criteria. For reporting C-RCTs, only eight (24%) of the 33 reviews reported the method of cluster adjustment for their included C-RCTs. For assessing risk of bias, only one review assessed all five C-RCT-specific risk-of-bias criteria. For analysing C-RCTs, of the 27 reviews that presented unadjusted data, only nine (33%) provided a warning that confidence intervals may be artificially narrow. Of the 34 reviews that reported data from unadjusted C-RCTs, only 13 (38%) excluded the unadjusted results from the meta-analyses.

Conclusions

The methodological and reporting practices in Cochrane reviews incorporating C-RCTs could be greatly improved, particularly with regard to analyses. Criteria developed as part of the current study could be used by review authors or editors to identify errors and improve the quality of published systematic reviews incorporating C-RCTs.     

Associated Factors and Consequences of Risk of Bias in Randomized Controlled Trials of Yoga: A Systematic Review

Background

Bias in randomized controlled trials (RCTs) of complementary therapy interventions seems to be associated with specific factors and to potentially distort the studies’ conclusions. This systematic review assessed associated factors of risk of bias and consequences for the studies’ conclusions in RCTs of yoga as one of the most commonly used complementary therapies.

Methods

Medline/PubMed, Scopus, IndMED and the Cochrane Library were searched through February 2014 for yoga RCTs. Risk of selection bias was assessed using the Cochrane tool and regressed to a) publication year; b) country of origin; c) journal type; and d) impact factor using multiple logistic regression analysis. Likewise, the authors’ conclusions were regressed to risk of bias.

Results

A total of 312 RCTs were included. Impact factor ranged from 0.0 to 39.2 (median = 1.3); 60 RCT (19.2%) had a low risk of selection bias, and 252 (80.8%) had a high or unclear risk of selection bias. Only publication year and impact factor significantly predicted low risk of bias; RCTs published after 2001 (adjusted odds ratio (OR) = 12.6; 95% confidence interval (CI) = 1.7, 94.0; p<0.001) and those published in journals with impact factor (adjusted OR = 2.6; 95%CI = 1.4, 4.9; p = 0.004) were more likely to have low risk of bias. The authors’ conclusions were not associated with risk of bias.

Conclusions

Risk of selection bias was generally high in RCTs of yoga; although the situation has improved since the publication of the revised CONSORT statement 2001. Pre-CONSORT RCTs and those published in journals without impact factor should be handled with increased care; although risk of bias is unlikely to distort the RCTs’ conclusions.     

Assessment of a Standardized Pre-Operative Telephone Checklist Designed to Avoid Late Cancellation of Ambulatory Surgery: The AMBUPROG Multicenter Randomized Controlled Trial

Objectives

To assess the impact of a standardized pre-operative telephone checklist on the rate of late cancellations of ambulatory surgery (AMBUPROG trial).

Design

Multicenter, two-arm, parallel-group, open-label randomized controlled trial.

Setting

11 university hospital ambulatory surgery units in Paris, France.

Participants

Patients scheduled for ambulatory surgery and able to be reached by telephone.

Intervention

A 7-item checklist designed to prevent late cancellation, available in five languages and two versions (for children and adults), was administered between 7 and 3 days before the planned date of surgery, by an automated phone system or a research assistant. The control group received standard management alone.

Main Outcome Measures

Rate of cancellation on the day of surgery or the day before.

Results

The study population comprised 3900 patients enrolled between November 2012 and September 2013: 1950 patients were randomized to the checklist arm and 1950 patients to the control arm. The checklist was administered to 68.8% of patients in the intervention arm, 1002 by the automated phone system and 340 by a research assistant. The rate of late cancellation did not differ significantly between the checklist and control arms (109 (5.6%) vs. 113 (5.8%), adjusted odds ratio [95% confidence interval] = 0.91 [0.65–1.29], (p = 0.57)). Checklist administration revealed that 355 patients (28.0%) had not undergone tests ordered by the surgeon or anesthetist, and that 254 patients (20.0%) still had questions concerning the fasting state.

Conclusions

A standardized pre-operative telephone checklist did not avoid late cancellations of ambulatory surgery but enabled us to identify several frequent causes.

Trial Registration

ClinicalTrials.gov {"type":"clinical-trial","attrs":{"text":"NCT01732159","term_id":"NCT01732159"}}NCT01732159