Systematic reviews and meta-analyses on treatment of asthma: critical evaluation

ObjectiveTo evaluate the clinical, methodological, and reporting aspects of systematic reviews and meta-analyses on the treatment of asthma and to compare those published by the Cochrane Collaboration with those published in paper based journals.DesignAnalysis of studies identified from Medline, CINAHL, HealthSTAR, EMBASE, Cochrane Library, personal collections, and reference lists.StudiesArticles describing a systematic review or a meta-analysis of the treatment of asthma that were published as a full report, in any language or format, in a peer reviewed journal or the Cochrane Library.Results50 systematic reviews and meta-analyses were included. More than half were published in the past two years. Twelve reviews were published in the Cochrane Library and 38 were published in 22 peer reviewed journals. Forced expiratory volume in one second was the most frequently used outcome, but few reviews evaluated the effect of treatment on costs or patient preferences. Forty reviews were judged to have serious or extensive flaws. All six reviews associated with industry were in this group. Seven of the 10 most rigorous reviews were published in the Cochrane Library.ConclusionsMost reviews published in peer reviewed journals or funded by industry have serious methodological flaws that limit their value to guide decisions. Cochrane reviews are more rigorous and better reported than those published in peer reviewed journals.     

Saffron (Crocus sativus L.) and health outcomes: a meta-research review of meta-analyses and an evidence mapping study

BackgroundAlthough a number of systematic reviews and meta-analyses of saffron (Crocus sativus L.) have been published, no study has comprehensively summarized the clinical evidence from meta-analyses, or assessed the reporting or methodological quality of these reviews.PurposeThe present meta-research study was designed to fill the gaps in knowledge to inform future studies and allow enhanced clinical decision-making on saffron.MethodsThe PubMed, Cochrane Library, Embase, and CNKI databases were systematically searched from inception to April 3 rd, 2021, for meta-analyses of clinical trials that assessed the efficacy and safety of saffron. PRISMA 2009 and AMSTAR-2 were employed to assess the reporting and methodological quality of meta-analyses identified in the search, respectively. The present study was registered on PROSPERO with registration number CRD42020220274.ResultsNineteen eligible systematic reviews with meta-analyses published in English were identified from 235 records. These meta-analyses were published in 12 peer-reviewed journals from 2013 to 2021. The heterogeneous results indicated that saffron significantly reduced fasting blood glucose, waist circumference, diastolic blood pressure, concentrations of total cholesterol and low-density lipoprotein cholesterol, and improved symptoms of depression, cognitive function and sexual dysfunction compared with controls (mainly placebos). Common side effects of saffron consumption included nausea, dry mouth, poor appetite, and headache, but no serious adverse reactions were reported. Primary analysis and sensitivity analysis confirmed that the reporting and methodological quality of reviews included in the study were highly correlated (p < 0.001). The quality of meta-analyses of saffron requires improvement by including a structured abstract, a prospective protocol and registration, explanation of the study designs within each study that is reviewed, the searches, risk of bias assessment, literature selection, and reporting of funding sources.ConclusionThe available evidence indicates that saffron is a safe plant for administration as a medicine and can improve diverse clinical outcomes, but the scientific quality of the published systematic reviews needs to be improved. Moreover, the clinical effects of saffron need to be confirmed through high-quality randomized trials in multiple countries with large sample sizes.     

Dissemination Bias in Systematic Reviews of Animal Research: A Systematic Review

Background

Systematic reviews of preclinical studies, in vivo animal experiments in particular, can influence clinical research and thus even clinical care. Dissemination bias, selective dissemination of positive or significant results, is one of the major threats to validity in systematic reviews also in the realm of animal studies. We conducted a systematic review to determine the number of published systematic reviews of animal studies until present, to investigate their methodological features especially with respect to assessment of dissemination bias, and to investigate the citation of preclinical systematic reviews on clinical research.

Methods

Eligible studies for this systematic review constitute systematic reviews that summarize in vivo animal experiments whose results could be interpreted as applicable to clinical care. We systematically searched Ovid Medline, Embase, ToxNet, and ScienceDirect from 1^st January 2009 to 9^th January 2013 for eligible systematic reviews without language restrictions. Furthermore we included articles from two previous systematic reviews by Peters et al. and Korevaar et al.

Results

The literature search and screening process resulted in 512 included full text articles. We found an increasing number of published preclinical systematic reviews over time. The methodological quality of preclinical systematic reviews was low. The majority of preclinical systematic reviews did not assess methodological quality of the included studies (71%), nor did they assess heterogeneity (81%) or dissemination bias (87%). Statistics quantifying the importance of clinical research citing systematic reviews of animal studies showed that clinical studies referred to the preclinical research mainly to justify their study or a future study (76%).

Discussion

Preclinical systematic reviews may have an influence on clinical research but their methodological quality frequently remains low. Therefore, systematic reviews of animal research should be critically appraised before translating them to a clinical context.     

Cochrane systematic reviews of Chinese herbal medicines: an overview

Objectives

Our study had two objectives: a) to systematically identify all existing systematic reviews of Chinese herbal medicines (CHM) published in Cochrane Library; b) to assess the methodological quality of included reviews.

Methodology/Principal Findings

We performed a systematic search of the Cochrane Database of Systematic Reviews (CDSR, Issue 5, 2010) to identify all reviews of CHM. A total of fifty-eight reviews were eligible for our study. Twenty-one of the included reviews had at least one Traditional Chinese Medicine (TCM) practitioner as its co-author. 7 reviews didn''t include any primary study, the remaining reviews (n = 51) included a median of 9 studies and 936 participants. 50% of reviews were last assessed as up-to-date prior to 2008. The questions addressed by 39 reviews were broad in scope, in which 9 reviews combined studies with different herbal medicines. For OQAQ, the mean of overall quality score (item 10) was 5.05 (95% CI; 4.58-5.52). All reviews assessed the methodological quality of primary studies, 16% of included primary studies used adequate sequence generation and 7% used adequate allocation concealment. Of the 51 nonempty reviews, 23 reviews were reported as being inconclusive, while 27 concluded that there might be benefit of CHM, which was limited by the poor quality or inadequate quantity of included studies. 58 reviews reported searching a median of seven electronic databases, while 10 reviews did not search any Chinese database.

Conclusions

Now CDSR has included large numbers of CHM reviews, our study identified some areas which could be improved, such as almost half of included reviews did not have the participation of TCM practitioners and were not up-to-date according to Cochrane criteria, some reviews pooled the results of different herbal medicines and ignored the searching of Chinese databases.     

Methodological Quality of Systematic Reviews in Subfertility: A Comparison of Two Different Approaches

Background

Systematic reviews are used widely to guide health care decisions. Several tools have been created to assess systematic review quality. The measurement tool for assessing the methodological quality of systematic reviews known as the AMSTAR tool applies a yes/no score to eleven relevant domains of review methodology. This tool has been reworked so that each domain is scored based on a four point scale, producing R-AMSTAR.

Methods and Findings

We aimed to compare the AMSTAR and R-AMSTAR tools in assessing systematic reviews in the field of assisted reproduction for subfertility. All published systematic reviews on assisted reproductive technology, with the latest search for studies taking place from 2007–2011, were considered. Reviews that contained no included studies or considered diagnostic outcomes were excluded. Thirty each of Cochrane and non-Cochrane reviews were randomly selected from a search of relevant databases. Both tools were then applied to all sixty reviews. The results were converted to percentage scores and all reviews graded and ranked based on this. AMSTAR produced a much wider variation in percentage scores and achieved higher inter-rater reliability than R-AMSTAR according to kappa statistics. The average rating for Cochrane reviews was consistent between the two tools (88.3% for R-AMSTAR versus 83.6% for AMSTAR) but inconsistent for non-Cochrane reviews (63.9% R-AMSTAR vs. 38.5% AMSTAR). In comparing the rankings generated between the two tools Cochrane reviews changed an average of 4.2 places, compared to 2.9 for non-Cochrane.

Conclusion

R-AMSTAR provided greater guidance in the assessment of domains and produced quantitative results. However, there were many problems with the construction of its criteria and AMSTAR was much easier to apply consistently. We recommend that AMSTAR incorporates the findings of this study and produces additional guidance for its application in order to improve its reliability and usefulness.     

Epidemiology, quality and reporting characteristics of systematic reviews of traditional Chinese medicine interventions published in Chinese journals

Background

Systematic reviews (SRs) of TCM have become increasingly popular in China and have been published in large numbers. This review provides the first examination of epidemiological characteristics of these SRs as well as compliance with the PRISMA and AMSTAR guidelines.

Objectives

To examine epidemiological and reporting characteristics as well as methodological quality of SRs of TCM published in Chinese journals.

Methods

Four Chinese databases were searched (CBM, CSJD, CJFD and Wanfang Database) for SRs of TCM, from inception through Dec 2009. Data were extracted into Excel spreadsheets. The PRISMA and AMSTAR checklists were used to assess reporting characteristics and methodological quality, respectively.

Results

A total of 369 SRs were identified, most (97.6%) of which used the terms systematic review or meta-analysis in the title. None of the reviews had been updated. Half (49.8%) were written by clinicians and nearly half (47.7%) were reported in specialty journals. The impact factors of 45.8% of the journals published in were zero. The most commonly treated conditions were diseases of the circulatory and digestive disease. Funding sources were not reported for any reviews. Most (68.8%) reported information about quality assessment, while less than half (43.6%) reported assessing for publication bias. Statistical mistakes appeared in one-third (29.3%) of reviews and most (91.9%) did not report on conflict of interest.

Conclusions

While many SRs of TCM interventions have been published in Chinese journals, the quality of these reviews is troubling. As a potential key source of information for clinicians and researchers, not only were many of these reviews incomplete, some contained mistakes or were misleading. Focusing on improving the quality of SRs of TCM, rather than continuing to publish them in great quantity, is urgently needed in order to increase the value of these studies.     

The evidence base for interventions delivered to children in primary care: an overview of cochrane systematic reviews

Background

As a first step in developing a framework to evaluate and improve the quality of care of children in primary care there is a need to identify the evidence base underpinning interventions relevant to child health. Our objective was to identify all Cochrane systematic reviews relevant to the management of childhood conditions in primary care and to assess the extent to which Cochrane reviews reflect the burden of childhood illness presenting in primary care.

Methodology/Principal Findings

We used the Cochrane Child Health Field register of child-relevant systematic reviews to complete an overview of Cochrane reviews related to the management of children in primary care. We compared the proportion of systematic reviews with the proportion of consultations in Australia, US, Dutch and UK general practice in children. We identified 396 relevant systematic reviews; 358 included primary studies on children while 251 undertook a meta-analysis. Most reviews (n = 218, 55%) focused on chronic conditions and over half (n = 216, 57%) evaluated drug interventions. Since 2000, the percentage of pediatric primary care relevant reviews only increased by 2% (7% to 9%) compared to 18% (10% to 28%) in all child relevant reviews. Almost a quarter of reviews (n = 78, 23%) were published on asthma treatments which only account for 3–5% of consultations. Conversely, 15–23% of consultations are due to skin conditions yet they represent only 7% (n = 23) of reviews.

Conclusions/Significance

Although Cochrane systematic reviews focus on clinical trials and do not provide a comprehensive picture of the evidence base underpinning the management of children in primary care, the mismatch between the focus of the published research and the focus of clinical activity is striking. Clinical trials are an important component of the evidence base and the lack of trial evidence to demonstrate intervention effectiveness in substantial areas of primary care for children should be addressed.     

An Assessment of the Methodological Quality of Published Network Meta-Analyses: A Systematic Review

ObjectiveTo assess the methodological quality of published network meta-analysis.DesignSystematic review.MethodsWe searched the medical literature for network meta-analyses of pharmaceuticals. We assessed general study characteristics, study transparency and reproducibility, methodological approach, and reporting of findings. We compared studies published in journals with lower impact factors with those published in journals with higher impact factors, studies published prior to January 1^st, 2013 with those published after that date, and studies supported financially by industry with those supported by non-profit institutions or that received no support.ResultsThe systematic literature search identified 854 citations. Three hundred and eighteen studies met our inclusion criteria. The number of network meta-analyses has grown rapidly, with 48% of studies published since January 2013. The majority of network meta-analyses were supported by a non-profit institution or received no support (68%). We found considerable inconsistencies among reviewed studies. Eighty percent reported search terms, 61% a network diagram, 65% sufficient data to replicate the analysis, and 90% the characteristics of included trials. Seventy percent performed a risk of bias assessment of included trials, 40% an assessment of model fit, and 56% a sensitivity analysis. Among studies with a closed loop, 69% examined the consistency of direct and indirect evidence. Sixty-four percent of studies presented the full matrix of head-to-head treatment comparisons. For Bayesian studies, 41% reported the probability that each treatment was best, 31% reported treatment ranking, and 16% included the model code or referenced publicly-available code. Network meta-analyses published in higher impact factors journals and those that did not receive industry support performed better across the assessment criteria. We found few differences between older and newer studies.ConclusionsThere is substantial variation in the network meta-analysis literature. Consensus among guidelines is needed improve the methodological quality, transparency, and consistency of study conduct and reporting.     

Cluster Randomised Trials in Cochrane Reviews: Evaluation of Methodological and Reporting Practice

Objective

Systematic reviews can include cluster-randomised controlled trials (C-RCTs), which require different analysis compared with standard individual-randomised controlled trials. However, it is not known whether review authors follow the methodological and reporting guidance when including these trials. The aim of this study was to assess the methodological and reporting practice of Cochrane reviews that included C-RCTs against criteria developed from existing guidance.

Methods

Criteria were developed, based on methodological literature and personal experience supervising review production and quality. Criteria were grouped into four themes: identifying, reporting, assessing risk of bias, and analysing C-RCTs. The Cochrane Database of Systematic Reviews was searched (2^nd December 2013), and the 50 most recent reviews that included C-RCTs were retrieved. Each review was then assessed using the criteria.

Results

The 50 reviews we identified were published by 26 Cochrane Review Groups between June 2013 and November 2013. For identifying C-RCTs, only 56% identified that C-RCTs were eligible for inclusion in the review in the eligibility criteria. For reporting C-RCTs, only eight (24%) of the 33 reviews reported the method of cluster adjustment for their included C-RCTs. For assessing risk of bias, only one review assessed all five C-RCT-specific risk-of-bias criteria. For analysing C-RCTs, of the 27 reviews that presented unadjusted data, only nine (33%) provided a warning that confidence intervals may be artificially narrow. Of the 34 reviews that reported data from unadjusted C-RCTs, only 13 (38%) excluded the unadjusted results from the meta-analyses.

Conclusions

The methodological and reporting practices in Cochrane reviews incorporating C-RCTs could be greatly improved, particularly with regard to analyses. Criteria developed as part of the current study could be used by review authors or editors to identify errors and improve the quality of published systematic reviews incorporating C-RCTs.     

Methodological Deficits in Diagnostic Research Using ‘-Omics’ Technologies: Evaluation of the QUADOMICS Tool and Quality of Recently Published Studies

Background

QUADOMICS is an adaptation of QUADAS (a quality assessment tool for use in systematic reviews of diagnostic accuracy studies), which takes into account the particular challenges presented by ‘-omics’ based technologies. Our primary objective was to evaluate the applicability and consistency of QUADOMICS. Subsequently we evaluated and describe the methodological quality of a sample of recently published studies using the tool.

Methodology/Principal Findings

45‘-omics’- based diagnostic studies were identified by systematic search of Pubmed using suitable MeSH terms (“Genomics”, “Sensitivity and specificity”, “Diagnosis”). Three investigators independently assessed the quality of the articles using QUADOMICS and met to compare observations and generate a consensus. Consistency and applicability was assessed by comparing each reviewer''s original rating with the consensus. Methodological quality was described using the consensus rating. Agreement was above 80% for all three reviewers. Four items presented difficulties with application, mostly due to the lack of a clearly defined gold standard. Methodological quality of our sample was poor; studies met roughly half of the applied criteria (mean ± sd, 54.7±18.4%). Few studies were carried out in a population that mirrored the clinical situation in which the test would be used in practice, (6, 13.3%); none described patient recruitment sufficiently; and less than half described clinical and physiological factors that might influence the biomarker profile (20, 44.4%).

Conclusions

The QUADOMICS tool can consistently be applied to diagnostic ‘-omics’ studies presently published in biomedical journals. A substantial proportion of reports in this research field fail to address design issues that are fundamental to make inferences relevant for patient care.     

Statistical multiplicity in systematic reviews of anaesthesia interventions: a quantification and comparison between Cochrane and non-Cochrane reviews

Background

Systematic reviews with meta-analyses often contain many statistical tests. This multiplicity may increase the risk of type I error. Few attempts have been made to address the problem of statistical multiplicity in systematic reviews. Before the implications are properly considered, the size of the issue deserves clarification. Because of the emphasis on bias evaluation and because of the editorial processes involved, Cochrane reviews may contain more multiplicity than their non-Cochrane counterparts. This study measured the quantity of statistical multiplicity present in a population of systematic reviews and aimed to assess whether this quantity is different in Cochrane and non-Cochrane reviews.

Methods/Principal Findings

We selected all the systematic reviews published by the Cochrane Anaesthesia Review Group containing a meta-analysis and matched them with comparable non-Cochrane reviews. We counted the number of statistical tests done in each systematic review. The median number of tests overall was 10 (interquartile range (IQR) 6 to 18). The median was 12 in Cochrane and 8 in non-Cochrane reviews (difference in medians 4 (95% confidence interval (CI) 2.0–19.0). The proportion that used an assessment of risk of bias as a reason for doing extra analyses was 42% in Cochrane and 28% in non-Cochrane reviews (difference in proportions 14% (95% CI −8 to 36). The issue of multiplicity was addressed in 6% of all the reviews.

Conclusion/Significance

Statistical multiplicity in systematic reviews requires attention. We found more multiplicity in Cochrane reviews than in non-Cochrane reviews. Many of the reasons for the increase in multiplicity may well represent improved methodological approaches and greater transparency, but multiplicity may also cause an increased risk of spurious conclusions. Few systematic reviews, whether Cochrane or non-Cochrane, address the issue of multiplicity.     

Reporting Quality of Systematic Reviews/Meta-Analyses of Acupuncture

Background

The QUOROM and PRISMA statements were published in 1999 and 2009, respectively, to improve the consistency of reporting systematic reviews (SRs)/meta-analyses (MAs) of clinical trials. However, not all SRs/MAs adhere completely to these important standards. In particular, it is not clear how well SRs/MAs of acupuncture studies adhere to reporting standards and which reporting criteria are generally ignored in these analyses.

Objectives

To evaluate reporting quality in SRs/MAs of acupuncture studies.

Methods

We performed a literature search for studies published prior to 2014 using the following public archives: PubMed, EMBASE, Web of Science, the Cochrane Database of Systematic Reviews (CDSR), the Chinese Biomedical Literature Database (CBM), the Traditional Chinese Medicine (TCM) database, the Chinese Journal Full-text Database (CJFD), the Chinese Scientific Journal Full-text Database (CSJD), and the Wanfang database. Data were extracted into pre-prepared Excel data-extraction forms. Reporting quality was assessed based on the PRISMA checklist (27 items).

Results

Of 476 appropriate SRs/MAs identified in our search, 203, 227, and 46 were published in Chinese journals, international journals, and the Cochrane Database, respectively. In 476 SRs/MAs, only 3 reported the information completely. By contrast, approximately 4.93% (1/203), 8.81% (2/227) and 0.00% (0/46) SRs/Mas reported less than 10 items in Chinese journals, international journals and CDSR, respectively. In general, the least frequently reported items (reported≤50%) in SRs/MAs were “protocol and registration”, “risk of bias across studies”, and “additional analyses” in both methods and results sections.

Conclusions

SRs/MAs of acupuncture studies have not comprehensively reported information recommended in the PRISMA statement. Our study underscores that, in addition to focusing on careful study design and performance, attention should be paid to comprehensive reporting standards in SRs/MAs on acupuncture studies.     

Discrepancies in Outcome Reporting Exist Between Protocols and Published Oral Health Cochrane Systematic Reviews

Objectives

To assess discrepancies in the analyzed outcomes between protocols and published reviews within Cochrane oral health systematic reviews (COHG) on the Cochrane Database of Systematic Reviews (CDSR).

Study Design and Setting

All COHG systematic reviews on the CDSR and the corresponding protocols were retrieved in November 2014 and information on the reported outcomes was recorded. Data was collected at the systematic review level by two reviewers independently.

Results

One hundred and fifty two reviews were included. In relation to primary outcomes, 11.2% were downgraded to secondary outcomes, 9.9% were omitted altogether in the final publication and new primary outcomes were identified in 18.4% of publications. For secondary outcomes, 2% were upgraded to primary, 12.5% were omitted and 30.9% were newly introduced in the publication. Overall, 45.4% of reviews had at least one discrepancy when compared to the protocol; these were reported in 14.5% reviews. The number of review updates appears to be associated with discrepancies between final review and protocol (OR: 3.18, 95% CI: 1.77, 5.74, p<0.001). The risk of reporting significant results was lower for both downgraded outcomes [RR: 0.52, 95% CI: 0.17, 1.58, p = 0.24] and upgraded or newly introduced outcomes [RR: 0.77, 95% CI: 0.36, 1.64, p = 0.50] compared to outcomes with no discrepancies. The risk of reporting significant results was higher for upgraded or newly introduced outcomes compared to downgraded outcomes (RR = 1.19, 95% CI: 0.65, 2.16, p = 0.57). None of the comparisons reached statistical significance.

Conclusion

While no evidence of selective outcome reporting was found in this study, based on the present analysis of SRs published within COHG systematic reviews, discrepancies between outcomes in pre-published protocols and final reviews continue to be common. Solutions such as the use of standardized outcomes to reduce the prevalence of this issue may need to be explored.     

The quality of reporting of RCTs used within a postoperative pain management meta-analysis, using the CONSORT statement

Background

Randomized controlled trials (RCTs) are routinely used in systematic reviews and meta-analyses that help inform healthcare and policy decision making. The proper reporting of RCTs is important because it acts as a proxy for health care providers and researchers to appraise the quality of the methodology, conduct and analysis of an RCT. The aims of this study are to analyse the overall quality of reporting in 23 RCTs that were used in a meta-analysis by assessing 3 key methodological items, and to determine factors associated with high quality of reporting. It is hypothesized that studies with larger sample sizes, that have funding reported, that are published in journals with a higher impact factor and that are in journals that have adopted or endorsed the CONSORT statement will be associated with better overall quality of reporting and reporting of key methodological items.

Methods

We systematically reviewed RCTs used within an anesthesiology related post-operative pain management meta-analysis. We included all of the 23 RCTs used, all of which were parallel design that addressed the use of femoral nerve block in improving outcomes after total knee arthroplasty. Data abstraction was done independently by two reviewers. The two main outcomes were: 1) 15 point overall quality of reporting score (OQRS) based on the Consolidated Standards for Reporting Trials (CONSORT) and 2) 3 point key methodological item score (KMIS) based on allocation concealment, blinding and intention-to-treat analysis.

Results

Twenty-three RCTs were included. The median OQRS was 9.0 (Interquartile Range?=?3). A multivariable regression analysis did not show any significant association between OQRS or KMIS and our four predictor variables hypothesized to improve reporting. The direction and magnitude of our results when compared to similar studies suggest that the sample size and impact factor are associated with improved key methodological item reporting.

Conclusions

The quality of reporting of RCTs used within an anesthesia related meta-analysis is poor to moderate. The information gained from this study should be used by journals to register the urgency for RCTs to be clear and transparent in reporting to help make literature accessible and comparable.     

Bias Due to Changes in Specified Outcomes during the Systematic Review Process

Background

Adding, omitting or changing outcomes after a systematic review protocol is published can result in bias because it increases the potential for unacknowledged or post hoc revisions of the planned analyses. The main objective of this study was to look for discrepancies between primary outcomes listed in protocols and in the subsequent completed reviews published on the Cochrane Library. A secondary objective was to quantify the risk of bias in a set of meta-analyses where discrepancies between outcome specifications in protocols and reviews were found.

Methods and Findings

New reviews from three consecutive issues of the Cochrane Library were assessed. For each review, the primary outcome(s) listed in the review protocol and the review itself were identified and review authors were contacted to provide reasons for any discrepancies. Over a fifth (64/288, 22%) of protocol/review pairings were found to contain a discrepancy in at least one outcome measure, of which 48 (75%) were attributable to changes in the primary outcome measure. Where lead authors could recall a reason for the discrepancy in the primary outcome, there was found to be potential bias in nearly a third (8/28, 29%) of these reviews, with changes being made after knowledge of the results from individual trials. Only 4(6%) of the 64 reviews with an outcome discrepancy described the reason for the change in the review, with no acknowledgment of the change in any of the eight reviews containing potentially biased discrepancies. Outcomes that were promoted in the review were more likely to be significant than if there was no discrepancy (relative risk 1.66 95% CI (1.10, 2.49), p = 0.02).

Conclusion

In a review, making changes after seeing the results for included studies can lead to biased and misleading interpretation if the importance of the outcome (primary or secondary) is changed on the basis of those results. Our assessment showed that reasons for discrepancies with the protocol are not reported in the review, demonstrating an under-recognition of the problem. Complete transparency in the reporting of changes in outcome specification is vital; systematic reviewers should ensure that any legitimate changes to outcome specification are reported with reason in the review.     

Long term efficacy of DOTS regimens for tuberculosis: systematic review

Objective To identify published studies assessing tuberculosis recurrence after successful treatment with standard short course regimens for six months to determine the strength and sufficiency of evidence to support current guidelines.Design Systematic review.Data sources Medline, Embase, Cochrane clinical trials register, specialist tuberculosis journals, and reference lists. Only English language publications were eligible.Review methods Studies were included irrespective of methodology or quality. Abstracted information included inclusion and exclusion criteria for participants, duration of follow-up, and definitions of treatment success and disease recurrence. The primary outcome was the proportion of successfully treated patients recorded with recurrent tuberculosis during the follow-up period.Results 17 study arms from 16 studies met the inclusion criteria; 10 were controlled clinical trials and six were either studies done under programmatic conditions or observational studies from functioning tuberculosis programmes. Although several clinical trials supported the use of daily treatment regimens, studies reporting tuberculosis recurrence after intermittent regimens were limited. Few studies carried out under routine programmatic conditions reported disease recurrence. Overall there was wide variation in recurrence after successful treatment, ranging from 0% to 14%. Considerable heterogeneity across studies precluded the systematic assessment of factors contributing to tuberculosis recurrence.Conclusions Despite DOTS (directly observed treatment, short course) being implemented for more than 10 years and millions of patients treated for tuberculosis, few studies have assessed the ability of standard DOTS regimens to result in lasting cure for patients treated under routine programmatic conditions.     

Methodological Rigour and Transparency of Clinical Practice Guidelines Developed by Neurology Professional Societies in Croatia

Background

Clinical practice guidelines are systematically created documents that summarize knowledge and assist in delivering high-quality medicine by identifying evidence that supports best clinical care. They are produced not only by international professional groups but also by local professionals to address locally-relevant clinical practice. We evaluated the methodological rigour and transparency of guideline development in neurology formulated by professionals in a local medical community.

Methods

We analyzed clinical guidelines in neurology publicly available at the web-site of the Physicians’ Assembly in Croatia in 2012: 6 guidelines developed by Croatian authors and 1 adapted from the European Federation of Neurological Societies. The quality was assessed by 2 independent evaluators using the AGREE II instrument. We also conducted a search of the Cochrane Library to identify potential changes in recommendation from Cochrane systematic reviews included in guideline preparation.

Results

The methodological quality of the guidelines greatly varied across different domains. „Scope and Purpose” and „Clarity of Presentation“ domains received high scores (100% [95% confidence interval (CI) 98.5–100] and 97% [77.9–100], respectively), the lowest scores were in “Stakeholder Involvement“ (19% [15.5–34.6]) and “Editorial Independence” (0% [0–19.2]). Conclusions of 3 guidelines based on Cochrane systematic reviews were confirmed in updated versions and one update provided new information on the effectiveness of another antidepressant. Two Cochrane reviews used in guidelines were withdrawn and split into new reviews and their findings are now considered to be out of date.

Conclusion

Neurological guidelines used in Croatia differ in structure and their methodological quality. We recommend to national societies and professional groups to develop a more systematic and rigorous approach to the development of the guidelines, timely inclusion of best evidences and an effort to involve target users and patients in the guideline development procedures.