Quality and content of abstracts in papers reporting about drug exposures during pregnancy

BACKGROUND: Most clinicians read only the abstract of papers in scientific journals. Therefore, it is very important that abstracts contain as much information as possible, to summarize the data succinctly. Our objectives were to evaluate the quality of information in abstracts reporting human fetal outcomes following drug exposure during pregnancy. METHODS: We developed quality criteria based on previous work, modifying them for use with pregnancy outcomes. Quality scores were calculated as present/absent for all of the equally weighted criteria, then expressed as percentages (present/[present + absent]). We examined a random sample of 100 abstracts obtained through searches of MEDLINE, EMBASE, and the Web of Science databases from 1990 to 2005. Average quality scores were compared across designs (cohort, case-control, meta-analysis, and mixed design) Using Kruskal-Wallis ANOVA and structured/unstructured formats using Student's t test. RESULTS: The overall average quality was 59.2% +/- 14% (median, 61.5%; range, 15.4-83.3%). Quality was not significantly different across designs (P = .16) or between structured and unstructured abstracts (P = .44). Quality scores increased over time (Rho = 0.23, P = .02). Most frequently absent were baseline risk (94%), drug dose (91%), nonsignificant P values (72%), confounders (69%), significant P values (57%), and risk difference (48%). CONCLUSIONS: Abstracts provide insufficient information, particularly baseline risk values, for readers to make evidence-based decisions regarding drug use during pregnancy. Efforts need to be made to improve the quality of abstracts and include critical information such as baseline risk.     

Reporting of gender-related information in clinical trials of drug therapy for myocardial infarction

BACKGROUND: Concern has been expressed that women are not adequately represented in clinical trials evaluating treatments for medical conditions they commonly experience. This study was designed to assess the reporting of data on women in recently published trials of drug therapy for myocardial infarction, including those funded by an agency with a gender-related policy. METHODS: All randomized controlled trials and meta-analyses of drug therapies for myocardial infarction published in The New England Journal of Medicine, The Lancet, The Journal of the American Medical Association, the Annals of Internal Medicine and the British Medical Journal from January 1992 to December 1996 were evaluated. On preliminary review 102 articles met the inclusion criteria; these were reviewed in detail, and 59 were excluded. Two reviewers independently extracted gender-related information from the 43 articles; discrepancies were resolved by consensus. RESULTS: Women presented up to 48% of the trial participants (mean 24.1%). In the trials funded by an agency with a gender-related policy, only 16.8% of participants, on average, were women. Of the 43 articles in the sample, only 14 (32%) provided gender-related results. Funding from an agency with gender-related policy did not affect the reporting of gender-related information. Subgroup analyses were provided for 14 (32%) of the 43 trials, including 2 (29%) of 7 trials funded by an agency with a gender-related policy. Of the 12 trials that included interaction analyses (excluding the 2 trials in which secondary analyses were conducted specifically to identify differences between women and men), 7 (58%) conducted an interaction analysis to determine if women responded differently than men; for one of these the interaction analysis was for a secondary outcome measure (drug safety). Only 5 (12%) of the 43 articles mentioned the differences between men and women in the Discussion section; 2 of these were studies that used secondary analyses to examine sex differences. Of the 5, only 1 was funded by an agency with a gender-related policy. INTERPRETATION: Women were poorly represented in the randomized controlled trials in this sample, regardless of whether the trials were funded by an agency with a gender-related policy. Structured reporting of gender-related information for clinical trials may improve the quality of information available about women and therefore facilitate the application of research findings to the care of women.     

Assessment of the Quality of Reporting in Abstracts of Randomized Controlled Trials Published in Five Leading Chinese Medical Journals

Background

Clear, transparent and sufficiently detailed abstracts of randomized trials (RCTs), published in journal articles are important because readers will often base their initial assessment of a trial on such information. However, little is known about the quality of reporting in abstracts of RCTs published in medical journals in China.

Methods

We identified RCTs abstracts from 5 five leading Chinese medical journals published between 1998 and 2007 and indexed in MEDLINE. We assessed the quality of reporting of these abstracts based on the Consolidated Standards of Reporting Trials (CONSORT) abstract checklist. We also sought to identify whether any differences exist in reporting between the Chinese and English language version of the same abstract.

Results

We identified 332 RCT abstracts eligible for examination. Overall, the abstracts we examined reported 0–8 items as designated in the CONSORT checklist. On average, three items were reported per abstract. Details of the interventions (288/332; 87%), the number of participants randomized (216/332; 65%) and study objectives (109/332; 33%) were the top three items reported. Only two RCT abstracts reported details of trial registration, no abstracts reported the method of allocation concealment and only one mentioned specifically who was blinded. In terms of the proportion of RCT abstracts fulfilling a criterion, the absolute difference (percentage points) between the Chinese and English abstracts was 10% (ranging from 0 to 25%) on average, per item.

Conclusions

The quality of reporting in abstracts of RCTs published in Chinese medical journals needs to be improved. We hope that the introduction and endorsement of the CONSORT for Abstracts guidelines by journals reporting RCTs will lead to improvements in the quality of reporting.     

Sex Distribution of Study Samples Reported in American Society of Biomechanics Annual Meeting Abstracts

Background

Study samples should be appropriately selected to maximize generalizability of results. Excluding one sex from studies of conditions that affect both sexes is problematic and has received attention as a public policy issue in the United States, resulting in legislation and recommendations made by the National Institutes of Health to address this deficiency of study designs. It is unknown to what extent biomechanical studies have inappropriately excluded one sex. The objective of this study was to provide objective data on this question.

Methods

A retrospective review of random samples of abstracts presented at American Society of Biomechanics annual meetings from 1983 to 2013 was conducted to assess reporting of sex of study samples and whether the study samples were approximately balanced with respect to sex.

Findings

We did not find a statistically significant increasing trend in the percentage of abstracts reporting sex over time. However, increasing trends were noted in the percentage of abstracts including both sexes (p < 0.05) and percentage of abstracts having an “approximately balanced” study sample containing 50 ± 20% females (p > 0.05). In 2013 the percentage of abstracts reporting studies having approximately balanced study samples was only 28%, far from the ideal level of 100%.

Interpretation

While there has been modest change since 1983, there remains significant room for improvement in the reporting and composition of experimental studies reported at American Society of Biomechanics annual meetings.     

A systematic evaluation of the impact of STRICTA and CONSORT recommendations on quality of reporting for acupuncture trials

Background

We investigated whether there had been an improvement in quality of reporting for randomised controlled trials of acupuncture since the publication of the STRICTA and CONSORT statements. We conducted a before-and-after study, comparing ratings for quality of reporting following the publication of both STRICTA and CONSORT recommendations.

Methodology and Principal Findings

Ninety peer reviewed journal articles reporting the results of acupuncture trials were selected at random from a wider sample frame of 266 papers. Papers published in three distinct time periods (1994–1995, 1999–2000 and 2004–2005) were compared. Assessment criteria were developed directly from CONSORT and STRICTA checklists. Papers were independently assessed for quality of reporting by two assessors, one of whom was blind to information which could have introduced systematic bias (e.g. date of publication). We detected a statistically significant increase in the reporting of CONSORT items for papers published in each time period measured. We did not, however, find a difference between the number of STRICTA items reported in journal articles published before and 3 to 4 years following the introduction of STRICTA recommendations.

Conclusions and Significance

The results of this study suggest that general standards of reporting for acupuncture trials have significantly improved since the introduction of the CONSORT statement in 1996, but that quality in reporting details specific to acupuncture interventions has yet to change following the more recent introduction of STRICTA recommendations. Wider targeting and revision of the guidelines is recommended.     

Reporting on quality of life in randomised controlled trials: bibliographic study

ObjectivesTo examine the frequency and quality of reporting on quality of life in randomised controlled trials.DesignSearch of the Cochrane Controlled Trials Register 1980 to 1997 to identify trials from all disciplines, from oncology, and from cardiovascular medicine that reported on quality of life. Assessment of abstracts from articles published from 1993 to 1996. Assessment of a sample of full reports with a standardised instrument.ResultsDuring 1980-97 reporting on quality of life increased from 0.63% to 4.2% for trials from all disciplines, from 1.5% to 8.2% for cancer trials, and from 0.34% to 3.6% for cardiovascular trials. Of 364 abstracts, 65% reported on drug interventions. Of a sample of 67 full reports, authors of 48 (72%) used 62 established quality of life instruments. In 15 reports (22%) authors developed their own measures, and in 2 (3%) methods were unclear. Response rates were given in 38 (57%), and complete reporting on all items and scales occurred in 31 (46%).ConclusionsLess than 5% of all randomised controlled trials reported on quality of life, and this proportion was below 10% even for cancer trials. A plethora of instruments was used in different studies, and the reporting of methods and results was often inadequate. Standards for the measurement and reporting of quality of life in clinical trials research need to be developed.

Key messages

• We examined the reporting on quality of life in randomised controlled trials listed in the Cochrane Controlled Trials Register
• Although reporting on quality of life increased over time, fewer than 5% of trials overall and fewer than 10% of cancer trials included quality of life in 1997
• Among 67 articles selected at random for detailed examination, a wide range of established and self developed measures of quality of life were used
• Only about half of trials gave response rates, and less than half reported on all items and scales used
• Standards for assessing and reporting quality of life in clinical research trials need to be developed

     

Recommendations for improving the home office project licence abstracts

The Freedom of Information Act 2000 prompted the Government to evaluate how public understanding about the use of animals in scientific research can be improved and how animal experimentation is regulated within the UK. This resulted in the publication of anonymous project licence abstracts. Here, the abstracts published between December 2004 and October 2005 have been evaluated, by using a scoring system which takes into account their content and format, in order to assess whether they provide information that will facilitate the increased implementation of reduction, refinement and replacement strategies. The potential impact of freely-available Home Office project licence abstracts on public attitudes to animal testing and on the quality of animal-based research, is also discussed. Although some abstracts scored well, in general, the abstracts were found to be severely lacking in detail about the welfare of experimental animals. Thus, the abstracts tend to present a distorted picture of animal-based research. Many abstracts lack the details which would reveal how the licensing criteria had been met. It is concluded that, in their current form, the project abstracts are not sufficiently informative. A number of recommendations are made, which address this problem.     

Conflict of interest in spine research reporting

Background

Medical studies are more likely to report favorable findings when a conflict of interest is declared. We aim to quantify and determine the effect of author disclosure of conflict of interest on scientific reporting.

Methods

Abstracts from an international spine research meeting (North American Spine Society 2010) were selected that specifically evaluated a device, biologic, or proprietary procedure. They were then made anonymous to reviewers. An item of interest was established in each of the abstracts in order to standardize evaluation. Next, three blinded reviewers independently rated the abstracts as favorable, neutral, or unfavorable with regard to the item of interest. Additionally, the blinded reviewers attempted to predict whether a related disclosure was made. The meeting disclosure index was used to tabulate the minimum US dollar value attributable to disclosures.

Results

Of the 344 total abstracts, 76 met inclusion criteria. In 79%, a related conflict of interest was reported. The amount of the disclosure was incompletely reported in 30% of cases. Where available, it averaged a cumulative minimum of $219,634 USD per abstract. The results of the abstracts were judged to be favorable, neutral, and unfavorable in 63%, 32% and 5% of abstracts, respectively. There was no correlation between the presence of a related disclosure and the findings of the studies (p = 0.81), although interpretation of this is limited by a small sample size and an overall apparent bias to report favorable studies. Additionally, the blinded reviewers were unable to predict whether a related disclosure was made (p = 0.40).

Conclusion

No association existed between the presence of a related disclosure and the results of the studies. While the actual compliance with reporting a potential conflict of interest is unable to be determined, the value amount related to the disclosures made was inadequately reported according to meeting guidelines.     

The limitations of sweat electrolyte reference intervals for the diagnosis of cystic fibrosis: a systematic review

The sweat test has been used for more than 50 years for the diagnosis of cystic fibrosis (CF) and remains an important diagnostic test in the genomic era. The currently used reference intervals for sweat electrolytes are applied to all patients regardless of age or sex. We performed a systematic review to summarise the studies with published reference values of sweat electrolyte concentrations for the diagnosis of CF. The MEDLINE (from 1950), EMBASE (from 1980) and PubMed (from 1950) databases were searched for English language studies. An abstract was also found by hand-searching. The search generated 1136 articles that matched the search key terms. Of these, 17 studies that contained data on sweat electrolyte concentrations were included in the analysis. Among these, seven studies did not perform the sweat test in accordance with current international and Australian guidelines. Of the ten remaining studies, four reported both the sweat sodium and chloride concentrations and six reported sweat chloride concentration only. A major limitation of these studies was the subject selection. Most recruited patients with various medical conditions including respiratory diseases or undefined recruitment criteria, whilst some did not report the subjects’ age and some had small subject numbers. Only one study performed mutation analysis to determine carrier status. No study used appropriate statistical analysis to develop a sweat chloride reference interval. The literature review yielded no studies that reliably developed reference intervals for sweat electrolyte concentrations. The limitations of the studies highlight the need for reliable age-related reference intervals for sweat electrolyte concentrations in healthy subjects.     

Quality of pharmaceutical industry press releases based on original research

Background

Press releases are a popular vehicle to disseminate health information to the lay media. While the quality of press releases issued by scientific conferences and medical journals has been questioned, no efforts to assess pharmaceutical industry press releases have been made. Therefore, we sought to systematically examine pharmaceutical company press releases about original research for measures of quality.

Methodolgy/Principal Findings

Press releases issued by the ten top selling, international pharmaceutical companies in the year 2005 were selected for evaluation. A total of 1028 electronic press releases were issued and 235 were based on original research. More than half (59%) reported results presented at a scientific meeting. Twenty-one percent of releases were not explicit about the source of original data. While harms or adverse events were commonly cited (76%), study limitations were rarely noted (6%). Almost one-third (29%) of releases did not quantify study results. Studies presented in abstract form were subsequently published within at least 20 months in 53% of cases.

Conclusions

Pharmaceutical company press releases frequently report basic study details. However, readers should be cautioned by the preliminary nature of the data and lack of identified limitations. Methods to improve the reporting and interpretation of drug company press releases are desirable to prevent misleading media coverage.     

Strengthening the Reporting of Observational Studies in Epidemiology (STROBE): explanation and elaboration

Much medical research is observational. The reporting of observational studies is often of insufficient quality. Poor reporting hampers the assessment of the strengths and weaknesses of a study and the generalisability of its results. Taking into account empirical evidence and theoretical considerations, a group of methodologists, researchers, and editors developed the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) recommendations to improve the quality of reporting of observational studies. The STROBE Statement consists of a checklist of 22 items, which relate to the title, abstract, introduction, methods, results and discussion sections of articles. Eighteen items are common to cohort studies, case-control studies and cross-sectional studies and four are specific to each of the three study designs. The STROBE Statement provides guidance to authors about how to improve the reporting of observational studies and facilitates critical appraisal and interpretation of studies by reviewers, journal editors and readers. This explanatory and elaboration document is intended to enhance the use, understanding, and dissemination of the STROBE Statement. The meaning and rationale for each checklist item are presented. For each item, one or several published examples and, where possible, references to relevant empirical studies and methodological literature are provided. Examples of useful flow diagrams are also included. The STROBE Statement, this document, and the associated Web site (http://www.strobe-statement.org/) should be helpful resources to improve reporting of observational research.     

The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies

Much biomedical research is observational. The reporting of such research is often inadequate, which hampers the assessment of its strengths and weaknesses and of a study's generalisability. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Initiative developed recommendations on what should be included in an accurate and complete report of an observational study. We defined the scope of the recommendations to cover three main study designs: cohort, case-control, and cross-sectional studies. We convened a 2-day workshop in September 2004, with methodologists, researchers, and journal editors to draft a checklist of items. This list was subsequently revised during several meetings of the coordinating group and in e-mail discussions with the larger group of STROBE contributors, taking into account empirical evidence and methodological considerations. The workshop and the subsequent iterative process of consultation and revision resulted in a checklist of 22 items (the STROBE Statement) that relate to the title, abstract, introduction, methods, results, and discussion sections of articles. 18 items are common to all three study designs and four are specific for cohort, case-control, or cross-sectional studies. A detailed Explanation and Elaboration document is published separately and is freely available on the Web sites of PLoS Medicine, Annals of Internal Medicine, and Epidemiology. We hope that the STROBE Statement will contribute to improving the quality of reporting of observational studies.     

The Assessment of the Quality of Reporting of Systematic Reviews/Meta-Analyses in Diagnostic Tests Published by Authors in China

Background

The quality of reporting in systematic reviews (SRs)/meta-analyses (MAs) of diagnostic tests published by authors in China has not been evaluated. The aims of present study are to evaluate the quality of reporting in diagnostic SRs/MAs using the PRISMA statement and determine the changes in the quality of reporting over time.

Methods

According to the inclusion and exclusion criteria, we searched five databases including Chinese Biomedical Literature Database, PubMed, EMBASE, the Cochrane Library, and Web of knowledge, to identify SRs/MAs on diagnostic tests. The searches were conducted on July 14, 2012 and the cut off for inclusion of the SRs/MAs was December 31^st 2011. The PRISMA statement was used to assess the quality of reporting. Analysis was performed using Excel 2003, RevMan 5.

Results

A total of 312 studies were included. Fifteen diseases systems were covered. According to the PRISMA checklist, there had been serious reporting flaws in following items: structured summary (item 2, 22.4%), objectives (item 4, 18.9%), protocol and registration (item 5, 2.6%), risk of bias across studies (item 15, 26.3%), funding (item 27, 28.8%). The subgroup analysis showed that there had been some statistically significant improvement in total compliance for 9 PRISMA items after the PRISMA was released, 6 items were statistically improved regarding funded articles, 3 items were statistically improved for CSCD articles, and there was a statistically significant increase in the proportion of reviews reporting on 22 items for SCI articles (P<0.050).

Conclusion

The numbers of diagnostic SRs/MAs is increasing annually. The quality of reporting has measurably been improved over the previous years. Unfortunately, there are still many deficiencies in the reporting including protocol and registration, search, risk of bias across studies, and funding. Future Chinese reviewers should address issues on these aspects.     

Outlier Removal and the Relation with Reporting Errors and Quality of Psychological Research

Background

The removal of outliers to acquire a significant result is a questionable research practice that appears to be commonly used in psychology. In this study, we investigated whether the removal of outliers in psychology papers is related to weaker evidence (against the null hypothesis of no effect), a higher prevalence of reporting errors, and smaller sample sizes in these papers compared to papers in the same journals that did not report the exclusion of outliers from the analyses.

Methods and Findings

We retrieved a total of 2667 statistical results of null hypothesis significance tests from 153 articles in main psychology journals, and compared results from articles in which outliers were removed (N = 92) with results from articles that reported no exclusion of outliers (N = 61). We preregistered our hypotheses and methods and analyzed the data at the level of articles. Results show no significant difference between the two types of articles in median p value, sample sizes, or prevalence of all reporting errors, large reporting errors, and reporting errors that concerned the statistical significance. However, we did find a discrepancy between the reported degrees of freedom of t tests and the reported sample size in 41% of articles that did not report removal of any data values. This suggests common failure to report data exclusions (or missingness) in psychological articles.

Conclusions

We failed to find that the removal of outliers from the analysis in psychological articles was related to weaker evidence (against the null hypothesis of no effect), sample size, or the prevalence of errors. However, our control sample might be contaminated due to nondisclosure of excluded values in articles that did not report exclusion of outliers. Results therefore highlight the importance of more transparent reporting of statistical analyses.     

Individual patient data meta-analysis of acupuncture for chronic pain: protocol of the Acupuncture Trialists' Collaboration

Objectives

To evaluate the use and reporting of adjusted analysis in randomised controlled trials (RCTs) and compare the quality of reporting before and after the revision of the CONSORT Statement in 2001.

Design

Comparison of two cross sectional samples of published articles.

Data Sources

Journal articles indexed on PubMed in December 2000 and December 2006.

Study Selection

Parallel group RCTs with a full publication carried out in humans and published in English

Main outcome measures

Proportion of articles reported adjusted analysis; use of adjusted analysis; the reason for adjustment; the method of adjustment and the reporting of adjusted analysis results in the main text and abstract.

Results

In both cohorts, 25% of studies reported adjusted analysis (84/355 in 2000 vs 113/422 in 2006). Compared with articles reporting only unadjusted analyses, articles that reported adjusted analyses were more likely to specify primary outcomes, involve multiple centers, perform stratified randomization, be published in general medical journals, and recruit larger sample sizes. In both years a minority of articles explained why and how covariates were selected for adjustment (20% to 30%). Almost all articles specified the statistical methods used for adjustment (99% in 2000 vs 100% in 2006) but only 5% and 10%, respectively, reported both adjusted and unadjusted results as recommended in the CONSORT guidelines.

Conclusion

There was no evidence of change in the reporting of adjusted analysis results five years after the revision of the CONSORT Statement and only a few articles adhered fully to the CONSORT recommendations.     

Inconsistent Definitions for Intention-To-Treat in Relation to Missing Outcome Data: Systematic Review of the Methods Literature

Background

Authors of randomized trial reports seem to hold a variety of views regarding the relationship between missing outcome data (MOD) and intention to treat (ITT). The objectives of this study were to systematically investigate how authors of methodology articles define ITT in the presence of MOD, how they recommend handling MOD under ITT, and to make a proposal for potential improvement in the definition and use of ITT in relation to MOD.

Methods and Findings

We systematically searched MEDLINE in February 2009 for methodological articles written in English that devoted at least one paragraph to ITT and two other paragraphs to either ITT or MOD. We excluded original trial reports, observational studies, and clinical systematic reviews. Working in teams of two, we independently extracted relevant information from each eligible article. Of 1007 titles and abstracts reviewed, 66 articles met eligibility criteria. Five (8%) did not provide a definition of ITT; 25 (38%) mentioned MOD but did not discuss its relationship to ITT; and 36 (55%) discussed the relationship of MOD with ITT. These 36 articles described one or more of three statements: complete follow-up is required for ITT (58%); ITT and MOD are separate issues (17%); and ITT requires a specific strategy for handling MOD (78%); 17 (47%) endorsed more than one relationship. The most frequently mentioned strategies for handling MOD within ITT were: using the last outcome carried forward (50%); sensitivity analysis (50%); and use of available data to impute missing data (46%).

Conclusion

We found that there is no consensus on the definition of ITT in relation to MOD. For conceptual clarity, we suggest that both reports of randomized trials and systematic reviews separately consider and describe how they deal with participants with complete data and those with MOD.     

Reliability of a structured method of selecting abstracts for a plastic surgical scientific meeting

There is no generally accepted method for assessing abstracts that are submitted for a medical scientific meeting. This article describes the development and prospective evaluation of such a method applied to the 220 abstracts submitted for the 2000 Annual Meeting of the European Association of Plastic Surgeons. Structured abstracts were evaluated in three categories: aesthetic surgery, basic research, and clinical study. Each anonymous abstract was assessed separately by 10 reputable European plastic surgeons. These reviewers used a structured rating questionnaire which resulted in a score given by each reviewer to each abstract between -6 and +6. The scores of all 10 reviewers were added for each abstract, and the papers were accepted in each of the three categories on the basis of this abridged score. To evaluate the reliability of this structured method of selection, the interrater agreement among the reviewers was tested by means of kappa analysis and the Cronbach alpha coefficient. The kappa values for agreement among reviewers regarding acceptability of abstracts were low, but the alpha coefficient indicated an acceptable degree of reliability of the average reviewers' ratings for all categories. Using a structured questionnaire can be helpful in the objective assessment of abstracts for a scientific meeting and may facilitate comparison of abstracts. Meritocratic dichotomy of abstracts by the reviewers is advocated to further improve reliability of the rating. Even though reliability generally increases with the number of reviewers, the annual increase of submitted abstracts may necessitate a decrease in the number of reviewers for each abstract.