Reporting Quality of Systematic Reviews and Meta-Analyses of Otorhinolaryngologic Articles Based on the PRISMA Statement

Background

Systematic reviews (SRs) and meta-analyses (MAs) provide the highest possible level of evidence. However, poor conduct or reporting of SRs and MAs may reduce their utility. The PRISMA Statement (Preferred Reporting Items for Systematic reviews and Meta-Analyses) was developed to help authors report their SRs and MAs adequately.

Objectives

Our objectives were to (1) evaluate the quality of reporting of SRs and MAs and their abstracts in otorhinolaryngologic literature using the PRISMA and PRISMA for Abstracts checklists, respectively, (2) compare the quality of reporting of SRs and MAs published in Ear Nose Throat (ENT) journals to the quality of SRs and MAs published in the ‘gold standard’ Cochrane Database of Systematic Reviews (CDSR), and (3) formulate recommendations to improve reporting of SRs and MAs in ENT journals.

Methods

On September 3, 2014, we searched the Pubmed database using a combination of filters to retrieve SRs and MAs on otorhinolaryngologic topics published in 2012 and 2013 in the top 5 ENT journals (ISI Web of Knowledge 2013) or CDSR and relevant articles were selected. We assessed how many, and which, PRISMA (for Abstracts) items were reported adequately per journal type.

Results

We identified large differences in the reporting of individual items between the two journal types with room for improvement. In general, SRs and MAs published in ENT journals (n = 31) reported a median of 54.4% of the PRISMA items adequately, whereas the 49 articles published in the CDSR reported a median of 100.0 adequately (difference statistically significant, p < 0.001). For abstracts, medians of 41.7% for ENT journals and 75.0% for the CDSR were found (p < 0.001).

Conclusion

The reporting of SRs and MAs in ENT journals leaves room for improvement and would benefit if the PRISMA Statement were endorsed by these journals.     

Reporting Quality of Systematic Reviews/Meta-Analyses of Acupuncture

Background

The QUOROM and PRISMA statements were published in 1999 and 2009, respectively, to improve the consistency of reporting systematic reviews (SRs)/meta-analyses (MAs) of clinical trials. However, not all SRs/MAs adhere completely to these important standards. In particular, it is not clear how well SRs/MAs of acupuncture studies adhere to reporting standards and which reporting criteria are generally ignored in these analyses.

Objectives

To evaluate reporting quality in SRs/MAs of acupuncture studies.

Methods

We performed a literature search for studies published prior to 2014 using the following public archives: PubMed, EMBASE, Web of Science, the Cochrane Database of Systematic Reviews (CDSR), the Chinese Biomedical Literature Database (CBM), the Traditional Chinese Medicine (TCM) database, the Chinese Journal Full-text Database (CJFD), the Chinese Scientific Journal Full-text Database (CSJD), and the Wanfang database. Data were extracted into pre-prepared Excel data-extraction forms. Reporting quality was assessed based on the PRISMA checklist (27 items).

Results

Of 476 appropriate SRs/MAs identified in our search, 203, 227, and 46 were published in Chinese journals, international journals, and the Cochrane Database, respectively. In 476 SRs/MAs, only 3 reported the information completely. By contrast, approximately 4.93% (1/203), 8.81% (2/227) and 0.00% (0/46) SRs/Mas reported less than 10 items in Chinese journals, international journals and CDSR, respectively. In general, the least frequently reported items (reported≤50%) in SRs/MAs were “protocol and registration”, “risk of bias across studies”, and “additional analyses” in both methods and results sections.

Conclusions

SRs/MAs of acupuncture studies have not comprehensively reported information recommended in the PRISMA statement. Our study underscores that, in addition to focusing on careful study design and performance, attention should be paid to comprehensive reporting standards in SRs/MAs on acupuncture studies.     

Design,Analysis, and Reporting of Crossover Trials for Inclusion in a Meta-Analysis

Randomized crossover trials are clinical experiments in which participants are assigned randomly to a sequence of treatments and each participant serves as his/her own control in estimating treatment effect. We need a better understanding of the validity of their results to enable recommendations as to which crossover trials can be included in meta-analysis and for development of reporting guidelines.

Objective

To evaluate the characteristics of the design, analysis, and reporting of crossover trials for inclusion in a meta-analysis of treatment for primary open-angle glaucoma and to provide empirical evidence to inform the development of tools to assess the validity of the results from crossover trials and reporting guidelines.

Methods

We searched MEDLINE, EMBASE, and Cochrane’s CENTRAL register for randomized crossover trials for a systematic review and network meta-analysis we are conducting. Two individuals independently screened the search results for eligibility and abstracted data from each included report.

Results

We identified 83 crossover trials eligible for inclusion. Issues affecting the risk of bias in crossover trials, such as carryover, period effects and missing data, were often ignored. Some trials failed to accommodate the within-individual differences in the analysis. For a large proportion of the trials, the authors tabulated the results as if they arose from a parallel design. Precision estimates properly accounting for the paired nature of the design were often unavailable from the study reports; consequently, to include trial findings in a meta-analysis would require further manipulation and assumptions.

Conclusions

The high proportion of poorly reported analyses and results has the potential to affect whether crossover data should or can be included in a meta-analysis. There is pressing need for reporting guidelines for crossover trials.     

Genomic Ancestry,Self-Rated Health and Its Association with Mortality in an Admixed Population: 10 Year Follow-Up of the Bambui-Epigen (Brazil) Cohort Study of Ageing

Background

Self-rated health (SRH) has strong predictive value for mortality in different contexts and cultures, but there is inconsistent evidence on ethnoracial disparities in SRH in Latin America, possibly due to the complexity surrounding ethnoracial self-classification.

Materials/Methods

We used 370,539 Single Nucleotide Polymorphisms (SNPs) to examine the association between individual genomic proportions of African, European and Native American ancestry, and ethnoracial self-classification, with baseline and 10-year SRH trajectories in 1,311 community dwelling older Brazilians. We also examined whether genomic ancestry and ethnoracial self-classification affect the predictive value of SRH for subsequent mortality.

Results

European ancestry predominated among participants, followed by African and Native American (median = 84.0%, 9.6% and 5.3%, respectively); the prevalence of Non-White (Mixed and Black) was 39.8%. Persons at higher levels of African and Native American genomic ancestry, and those self-identified as Non-White, were more likely to report poor health than other groups, even after controlling for socioeconomic conditions and an array of self-reported and objective physical health measures. Increased risks for mortality associated with worse SRH trajectories were strong and remarkably similar (hazard ratio ~3) across all genomic ancestry and ethno-racial groups.

Conclusions

Our results demonstrated for the first time that higher levels of African and Native American genomic ancestry—and the inverse for European ancestry—were strongly correlated with worse SRH in a Latin American admixed population. Both genomic ancestry and ethnoracial self-classification did not modify the strong association between baseline SRH or SRH trajectory, and subsequent mortality.     

An Investigation of the Shortcomings of the CONSORT 2010 Statement for the Reporting of Group Sequential Randomised Controlled Trials: A Methodological Systematic Review

Background

It can be argued that adaptive designs are underused in clinical research. We have explored concerns related to inadequate reporting of such trials, which may influence their uptake. Through a careful examination of the literature, we evaluated the standards of reporting of group sequential (GS) randomised controlled trials, one form of a confirmatory adaptive design.

Methods

We undertook a systematic review, by searching Ovid MEDLINE from the 1^st January 2001 to 23^rd September 2014, supplemented with trials from an audit study. We included parallel group, confirmatory, GS trials that were prospectively designed using a Frequentist approach. Eligible trials were examined for compliance in their reporting against the CONSORT 2010 checklist. In addition, as part of our evaluation, we developed a supplementary checklist to explicitly capture group sequential specific reporting aspects, and investigated how these are currently being reported.

Results

Of the 284 screened trials, 68(24%) were eligible. Most trials were published in “high impact” peer-reviewed journals. Examination of trials established that 46(68%) were stopped early, predominantly either for futility or efficacy. Suboptimal reporting compliance was found in general items relating to: access to full trials protocols; methods to generate randomisation list(s); details of randomisation concealment, and its implementation. Benchmarking against the supplementary checklist, GS aspects were largely inadequately reported. Only 3(7%) trials which stopped early reported use of statistical bias correction. Moreover, 52(76%) trials failed to disclose methods used to minimise the risk of operational bias, due to the knowledge or leakage of interim results. Occurrence of changes to trial methods and outcomes could not be determined in most trials, due to inaccessible protocols and amendments.

Discussion and Conclusions

There are issues with the reporting of GS trials, particularly those specific to the conduct of interim analyses. Suboptimal reporting of bias correction methods could potentially imply most GS trials stopping early are giving biased results of treatment effects. As a result, research consumers may question credibility of findings to change practice when trials are stopped early. These issues could be alleviated through a CONSORT extension. Assurance of scientific rigour through transparent adequate reporting is paramount to the credibility of findings from adaptive trials. Our systematic literature search was restricted to one database due to resource constraints.     

Bias in Spontaneous Reporting of Adverse Drug Reactions in Japan

Background

Attitudes of healthcare professionals regarding spontaneous reporting of adverse drug reactions (ADRs) in Japan are not well known, and Japan’s unique system of surveillance, called early post-marketing phase vigilance (EPPV), may affect these reporting attitudes. Our objectives were to describe potential effects of EPPV and to test whether ADR seriousness, prominence, and frequency are related to changes in reporting over time.

Methods

A manufacturer’s database of spontaneous ADR reports was used to extract data from individual case safety reports for 5 drugs subject to EPPV. The trend of reporting and the time lag between ADR onset and reporting to the manufacturer were examined. The following indices for ADRs occurring with each drug were calculated and analyzed to assess reporting trends: Serious:Non-serious ratio, High prominence:Low prominence ratio, and High frequency:Low frequency ratio.

Results

For all 5 drugs, the time lag between ADR onset and reporting to the manufacturer was shorter in the EPPV period than in the post-EPPV period. All drugs showed higher Serious:Non-serious ratios in the post-EPPV period. No specific patterns were observed for the High prominence:Low prominence ratio. The High frequency:Low frequency ratio for peginterferon alpha-2a and sevelamer hydrochloride decreased steadily throughout the study period.

Conclusions

Healthcare professionals may be more likely to report serious ADRs than to report non-serious ADRs, but the effect of event prominence on reporting trends is still unclear. Factors associated with ADR reporting attitude in Japan might be different from those in other countries because of EPPV and the involvement of medical representatives in the spontaneous reporting process. Pharmacovigilance specialists should therefore be cautious when comparing data between different time periods or different countries. Further studies are needed to elucidate the underlying mechanism of spontaneous ADR reporting in Japan.     

Strengthening Community-Based Vital Events Reporting for Real-Time Monitoring of Under-Five Mortality: Lessons Learned from the Balaka and Salima Districts in Malawi

Background

Malawi ratified a compulsory birth and death registration system in 2009. Until it captures complete coverage of vital events, Malawi relies on other data sources to calculate mortality estimates. We tested a community-based method to estimate annual under-five mortality rates (U5MR) through the Real-Time Monitoring of Under-Five Mortality (RMM) project in Malawi. We implemented RMM in two phases, and conducted an independent evaluation of phase one after 21 months of implementation. We present results of the phase two validation that covers the full project time span, and compare the results to those of the phase one validation.

Methods and Findings

We assessed the completeness of the counts of births and deaths and the accuracy of disaggregated U5MR from the community-based method against a retrospective full pregnancy history for rolling twelve-month periods after the independent evaluation. We used full pregnancy histories collected through household interviews carried out between November 2013 and January 2014 as the validation data source. Health Surveillance Agents (HSAs) across the 160 catchment areas submitted routine reports on pregnancies, births, and deaths consistently. However, for the 15-month implementation period post-evaluation, average completeness of birth event reporting was 76%, whereas average completeness of death event reporting was 67% relative to that expected from a comparable pregnancy history. HSAs underestimated the U5MR by an average of 21% relative to that estimated from a comparable pregnancy history.

Conclusions

On a medium scale, the community-based RMM method in Malawi produced substantial underestimates of annualized U5MR relative to those obtained from a full pregnancy history, despite the additional incentives and quality-control activities. We were not able to achieve an optimum level of incentive and support to make the system work while ensuring sustainability. Lessons learned from the implementation of RMM can inform programs supporting community-based interventions through HSAs in Malawi.     

The Assessment of the Quality of Reporting of Systematic Reviews/Meta-Analyses in Diagnostic Tests Published by Authors in China

Background

The quality of reporting in systematic reviews (SRs)/meta-analyses (MAs) of diagnostic tests published by authors in China has not been evaluated. The aims of present study are to evaluate the quality of reporting in diagnostic SRs/MAs using the PRISMA statement and determine the changes in the quality of reporting over time.

Methods

According to the inclusion and exclusion criteria, we searched five databases including Chinese Biomedical Literature Database, PubMed, EMBASE, the Cochrane Library, and Web of knowledge, to identify SRs/MAs on diagnostic tests. The searches were conducted on July 14, 2012 and the cut off for inclusion of the SRs/MAs was December 31^st 2011. The PRISMA statement was used to assess the quality of reporting. Analysis was performed using Excel 2003, RevMan 5.

Results

A total of 312 studies were included. Fifteen diseases systems were covered. According to the PRISMA checklist, there had been serious reporting flaws in following items: structured summary (item 2, 22.4%), objectives (item 4, 18.9%), protocol and registration (item 5, 2.6%), risk of bias across studies (item 15, 26.3%), funding (item 27, 28.8%). The subgroup analysis showed that there had been some statistically significant improvement in total compliance for 9 PRISMA items after the PRISMA was released, 6 items were statistically improved regarding funded articles, 3 items were statistically improved for CSCD articles, and there was a statistically significant increase in the proportion of reviews reporting on 22 items for SCI articles (P<0.050).

Conclusion

The numbers of diagnostic SRs/MAs is increasing annually. The quality of reporting has measurably been improved over the previous years. Unfortunately, there are still many deficiencies in the reporting including protocol and registration, search, risk of bias across studies, and funding. Future Chinese reviewers should address issues on these aspects.     

The Reporting of Observational Clinical Functional Magnetic Resonance Imaging Studies: A Systematic Review

Introduction

Complete reporting assists readers in confirming the methodological rigor and validity of findings and allows replication. The reporting quality of observational functional magnetic resonance imaging (fMRI) studies involving clinical participants is unclear.

Objectives

We sought to determine the quality of reporting in observational fMRI studies involving clinical participants.

Methods

We searched OVID MEDLINE for fMRI studies in six leading journals between January 2010 and December 2011.Three independent reviewers abstracted data from articles using an 83-item checklist adapted from the guidelines proposed by Poldrack et al. (Neuroimage 2008; 40: 409–14). We calculated the percentage of articles reporting each item of the checklist and the percentage of reported items per article.

Results

A random sample of 100 eligible articles was included in the study. Thirty-one items were reported by fewer than 50% of the articles and 13 items were reported by fewer than 20% of the articles. The median percentage of reported items per article was 51% (ranging from 30% to 78%). Although most articles reported statistical methods for within-subject modeling (92%) and for between-subject group modeling (97%), none of the articles reported observed effect sizes for any negative finding (0%). Few articles reported justifications for fixed-effect inferences used for group modeling (3%) and temporal autocorrelations used to account for within-subject variances and correlations (18%). Other under-reported areas included whether and how the task design was optimized for efficiency (22%) and distributions of inter-trial intervals (23%).

Conclusions

This study indicates that substantial improvement in the reporting of observational clinical fMRI studies is required. Poldrack et al.''s guidelines provide a means of improving overall reporting quality. Nonetheless, these guidelines are lengthy and may be at odds with strict word limits for publication; creation of a shortened-version of Poldrack''s checklist that contains the most relevant items may be useful in this regard.     

Cluster Randomised Trials in Cochrane Reviews: Evaluation of Methodological and Reporting Practice

Objective

Systematic reviews can include cluster-randomised controlled trials (C-RCTs), which require different analysis compared with standard individual-randomised controlled trials. However, it is not known whether review authors follow the methodological and reporting guidance when including these trials. The aim of this study was to assess the methodological and reporting practice of Cochrane reviews that included C-RCTs against criteria developed from existing guidance.

Methods

Criteria were developed, based on methodological literature and personal experience supervising review production and quality. Criteria were grouped into four themes: identifying, reporting, assessing risk of bias, and analysing C-RCTs. The Cochrane Database of Systematic Reviews was searched (2^nd December 2013), and the 50 most recent reviews that included C-RCTs were retrieved. Each review was then assessed using the criteria.

Results

The 50 reviews we identified were published by 26 Cochrane Review Groups between June 2013 and November 2013. For identifying C-RCTs, only 56% identified that C-RCTs were eligible for inclusion in the review in the eligibility criteria. For reporting C-RCTs, only eight (24%) of the 33 reviews reported the method of cluster adjustment for their included C-RCTs. For assessing risk of bias, only one review assessed all five C-RCT-specific risk-of-bias criteria. For analysing C-RCTs, of the 27 reviews that presented unadjusted data, only nine (33%) provided a warning that confidence intervals may be artificially narrow. Of the 34 reviews that reported data from unadjusted C-RCTs, only 13 (38%) excluded the unadjusted results from the meta-analyses.

Conclusions

The methodological and reporting practices in Cochrane reviews incorporating C-RCTs could be greatly improved, particularly with regard to analyses. Criteria developed as part of the current study could be used by review authors or editors to identify errors and improve the quality of published systematic reviews incorporating C-RCTs.     

Health Reporting in Print Media in Lebanon: Evidence,Quality and Role in Informing Policymaking

Background

Media plays a vital role in shaping public policies and opinions through disseminating health-related information. This study aims at exploring the role of media in informing health policies in Lebanon, identifying the factors influencing health reporting and investigating the role of evidence in health journalism and the quality of health reporting. It also identifies strategies to enhance the use of evidence in health journalism and improve the quality of health reporting.

Methods

Media analysis was conducted to assess the way media reports on health-related issues and the quality of reporting using a quality assessment tool. Semi-structured interviews were also conducted with 27 journalists, researchers and policymakers to explore their perception on the role of media in health policymaking and the factors influencing health reporting. In addition, a validation workshop was conducted.

Results

Out of 1,279 health-related news articles identified, 318 articles used certain type of evidence to report health issues 39.8% of which relied on experts’ opinions as their source of evidence while only 5.9% referenced peer-reviewed research studies. The quality of health reporting was judged to be low based on a quality assessment tool consisting of a set of ten criteria. Journalists raised concerns about issues impeding them from referring to evidence. Journalists also reported difficulties with the investigative health journalism. Policymakers and researchers viewed media as an important tool for evidence-informed health policies, however, serious concerns were voiced in terms of the current practice and capacities.

Conclusion

Our study provides a structured reflection on the role of media and the factors that influence health reporting including context-specific strategies that would enhance the quality and promote the use of evidence in health reporting. In the light of the political changes in many Middle Eastern countries, findings from this study can contribute to redefining the role of media in strengthening health systems.     

The Impact of WHO Essential Medicines Policies on Inappropriate Use of Antibiotics

Background

Inappropriate overuse of antibiotics contributes to antimicrobial resistance (AMR), yet policy implementation to reduce inappropriate antibiotic use is poor in low and middle-income countries.

Aims

To determine whether public sector inappropriate antibiotic use is lower in countries reporting implementation of selected essential medicines policies.

Materials and Methods

Results from independently conducted antibiotic use surveys in countries that did, and did not report implementation of policies to reduce inappropriate antibiotic prescribing, were compared. Survey data on four validated indicators of inappropriate antibiotic use and 16 self-reported policy implementation variables from WHO databases were extracted. The average difference for indicators between countries reporting versus not reporting implementation of specific policies was calculated. For 16 selected policies we regressed the four antibiotic use variables on the numbers of policies the countries reported implementing.

Results

Data were available for 55 countries. Of 16 policies studied, four (having a national Ministry of Health unit on promoting rational use of medicines, a national drug information centre and provincial and hospital drugs and therapeutics committees) were associated with statistically significant reductions in antibiotic use of ≥20% in upper respiratory infection (URTI). A national strategy to contain antibiotic resistance was associated with a 30% reduction in use of antibiotics in acute diarrheal illness. Policies seemed to be associated with greater effects in antibiotic use for URTI and diarrhea compared with antibiotic use in all patients. There were negative correlations between the numbers of policies reported implemented and the percentage of acute diarrhoea cases treated with antibiotics (r = -0.484, p = 0.007) and the percentage of URTI cases treated with antibiotics (r = -0.472, p = 0.005). Major study limitations were the reliance on self-reported policy implementation data and antibiotic use data from linited surveys.

Conclusions

Selected essential medicines policies were associated with lower antibiotic use in low and middle income countries.     

Gender Differentials in Self-Rated Health and Self-Reported Disability among Adults in India

Background

The extant literature on gender differentials in health in developed countries suggests that women outlive men at all ages, but women report poorer health than men. It is well established that Indian women live longer than men, but few studies have been conducted to understand the gender dimension in self-rated health and self-reported disability. The present study investigates gender differentials in self-rated health (SRH) and self-reported disability (SRD) among adults in India, using a nationally representative data.

Methods

Using data on 10,736 respondents aged 18 and older in the 2007 WHO Study on Global Ageing and Adult Health in India, prevalence estimates of SRH are calculated separately for men and women by socio-economic and demographic characteristics. The association of SRH with gender is tested using a multinomial logistic regression method. SRD is assessed using 20 activities of daily living (ADL). Further, gender differences in total life expectancy (TLE), disability life expectancy (DLE) and the proportion of life spent with a disability at various adult ages are measured.

Results

The relative risk of reporting poor health by women was significantly higher than men (relative risk ratio: 1.660; 95% confidence Interval (CI): 1.430–1.927) after adjusting for socio-economic and demographic characteristics. Women reported higher prevalence of severe and extreme disability than men in 14 measures out of a total20 ADL measures. Women aged less than 60 years reported two times more than men in SRD ≥ 5 ADLs. Finally, both DLE and proportion of life spent with a disability were substantially higher for women irrespective of their ages.

Conclusion

Indian women live longer but report poorer health than men. A substantial gender differential is found in self-reported disability. This makes for an urgent call to health researchers and policy makers for gender-sensitive programs.     

The Method Quality of Cross-Over Studies Involved in Cochrane Systematic Reviews

Background

It is possible that cross-over studies included in current systematic reviews are being inadequately assessed, because the current risk of bias tools do not consider possible biases specific to cross-over design. We performed this study to evaluate whether this was being done in cross-over studies included in Cochrane Systematic Reviews (CSRs).

Methods

We searched the Cochrane Library (up to 2013 issue 5) for CSRs that included at least one cross-over trial. Two authors independently undertook the study selection and data extraction. A random sample of the CSRs was selected and we evaluated whether the cross-over trials in these CSRs were assessed according to criteria suggested by the Cochrane handbook. In addition we reassessed the risk of bias of these cross-over trials by a checklist developed form the Cochrane handbook.

Results

We identified 688 CSRs that included one or more cross-over studies. We chose a random sample of 60 CSRs and these included 139 cross-over studies. None of these CSRs undertook a risk of bias assessment specific for cross-over studies. In fact items specific for cross-over studies were seldom considered anywhere in quality assessment of these CSRs. When we reassessed the risk of bias, including the 3 items specific to cross-over trials, of these 139 studies, a low risk of bias was judged for appropriate cross-over design in 110(79%), carry-over effects in 48(34%) and for reporting data in all stages of the trial in 114(82%).Assessment of biases in cross-over trials could affect the GRADE assessment of a review’s findings.

Conclusion

The current Cochrane risk of bias tool is not adequate to assess cross-over studies. Items specific to cross-over trials leading to potential risk of bias are generally neglected in CSRs. A proposed check list for the evaluation of cross-over trials is provided.     

Systematic Review of the Measurement Properties of Tools Used to Measure Behaviour Problems in Young Children with Autism

Background

Behaviour problems are common in young children with autism spectrum disorder (ASD). There are many different tools used to measure behavior problems but little is known about their validity for the population.

Objectives

To evaluate the measurement properties of behaviour problems tools used in evaluation of intervention or observational research studies with children with ASD up to the age of six years.

Methods

Behaviour measurement tools were identified as part of a larger, two stage, systematic review. First, sixteen major electronic databases, as well as grey literature and research registers were searched, and tools used listed and categorized. Second, using methodological filters, we searched for articles examining the measurement properties of the tools in use with young children with ASD in ERIC, MEDLINE, EMBASE, CINAHL, and PsycINFO. The quality of these papers was then evaluated using the COSMIN checklist.

Results

We identified twelve tools which had been used to measure behaviour problems in young children with ASD, and fifteen studies which investigated the measurement properties of six of these tools. There was no evidence available for the remaining six tools. Two questionnaires were found to be the most robust in their measurement properties, the Child Behavior Checklist and the Home Situations Questionnaire—Pervasive Developmental Disorders version.

Conclusions

We found patchy evidence on reliability and validity, for only a few of the tools used to measure behaviour problems in young children with ASD. More systematic research is required on measurement properties of tools for use in this population, in particular to establish responsiveness to change which is essential in measurement of outcomes of intervention.

PROSPERO Registration Number

CRD42012002223     

Risk Knowledge in Relapsing Multiple Sclerosis (RIKNO 1.0) - Development of an Outcome Instrument for Educational Interventions

Background

Adequate risk knowledge of patients is a prerequisite for shared decision making but few attempts have been made to develop assessment tools. Multiple Sclerosis (MS) is a chronic inflammatory disease of young adults with an increasing number of partially effective immunotherapies and therefore a paradigmatic disease to study patient involvement.

Objective/methods

Based on an item bank of MS risk knowledge items and patient feedback including perceived relevance we developed a risk knowledge questionnaire for relapsing remitting (RR) MS (RIKNO 1.0) which was a primary outcome measure in a patient education trial (192 early RRMS patients).

Results

Fourteen of the RIKNO 1.0 multiple-choice items were selected based on patient perceived relevance and item difficulty indices, and five on expert opinion. Mean item difficulty was 0.58, ranging from 0.14 to 0.79. Mean RIKNO 1.0 score increased after the educational intervention from 10.6 to 12.4 (p = 0.0003). Selected items were particularly difficult (e.g. those on absolute risk reductions of having a second relapse) and were answered correctly in only 30% of the patients, even after the intervention.

Conclusion

Despite its high difficulty, RIKNO 1.0 is a responsive instrument to assess risk knowledge in RRMS patients participating in educational interventions.     

Assessment of the Quality of Reporting in Abstracts of Randomized Controlled Trials Published in Five Leading Chinese Medical Journals

Background

Clear, transparent and sufficiently detailed abstracts of randomized trials (RCTs), published in journal articles are important because readers will often base their initial assessment of a trial on such information. However, little is known about the quality of reporting in abstracts of RCTs published in medical journals in China.

Methods

We identified RCTs abstracts from 5 five leading Chinese medical journals published between 1998 and 2007 and indexed in MEDLINE. We assessed the quality of reporting of these abstracts based on the Consolidated Standards of Reporting Trials (CONSORT) abstract checklist. We also sought to identify whether any differences exist in reporting between the Chinese and English language version of the same abstract.

Results

We identified 332 RCT abstracts eligible for examination. Overall, the abstracts we examined reported 0–8 items as designated in the CONSORT checklist. On average, three items were reported per abstract. Details of the interventions (288/332; 87%), the number of participants randomized (216/332; 65%) and study objectives (109/332; 33%) were the top three items reported. Only two RCT abstracts reported details of trial registration, no abstracts reported the method of allocation concealment and only one mentioned specifically who was blinded. In terms of the proportion of RCT abstracts fulfilling a criterion, the absolute difference (percentage points) between the Chinese and English abstracts was 10% (ranging from 0 to 25%) on average, per item.

Conclusions

The quality of reporting in abstracts of RCTs published in Chinese medical journals needs to be improved. We hope that the introduction and endorsement of the CONSORT for Abstracts guidelines by journals reporting RCTs will lead to improvements in the quality of reporting.     

Interpretation of Results of Studies Evaluating an Intervention Highlighted in Google Health News: A Cross-Sectional Study of News

Background

Mass media through the Internet is a powerful means of disseminating medical research. We aimed to determine whether and how the interpretation of research results is misrepresented by the use of “spin” in the health section of Google News. Spin was defined as specific way of reporting, from whatever motive (intentional or unintentional), to emphasize that the beneficial effect of the intervention is greater than that shown by the results.

Methods

We conducted a cross-sectional study of news highlighted in the health section of US, UK and Canada editions of Google News between July 2013 and January 2014. We searched for news items for 3 days a week (i.e., Monday, Wednesday, and Friday) during 6 months and selected a sample of 130 news items reporting a scientific article evaluating the effect of an intervention on human health.

Results

In total, 78% of the news did not provide a full reference or electronic link to the scientific article. We found at least one spin in 114 (88%) news items and 18 different types of spin in news. These spin were mainly related to misleading reporting (59%) such as not reporting adverse events that were reported in the scientific article (25%), misleading interpretation (69%) such as claiming a causal effect despite non-randomized study design (49%) and overgeneralization/misleading extrapolation (41%) of the results such as extrapolating a beneficial effect from an animal study to humans (21%). We also identified some new types of spin such as highlighting a single patient experience for the success of a new treatment instead of focusing on the group results.

Conclusions

Interpretation of research results was frequently misrepresented in the health section of Google News. However, we do not know whether these spin were from the scientific articles themselves or added in the news.