Intervention description is not enough: evidence from an in-depth multiple case study on the untold role and impact of context in randomised controlled trials of seven complex interventions

ABSTRACT: BACKGROUND: A number of single case reports have suggested that the context within which intervention studies take place may challenge the assumptions that underpin RCTs. However, the diverse ways in which context may challenge the central tenets of the RCT, and the degree to which this information is known to researchers and/or subsequently reported, has received much less attention. In this paper we explore these issues by focussing on 7 RCTs of interventions ranging in type and degree of complexity, and across diverse contexts. METHODS: This in-depth multiple case study using interviews, focus groups and documentary analysis was conducted in two phases. In phase one, a RCT of a nurse-led intervention provided a single exploratory case and informed the design, sampling and data collection within the main study. Phase two consisted of a multiple explanatory case study covering a range of trials of different types of complex intervention. A total of 84 data sources across the 7 trials were accessed. RESULTS: We present consistent empirical evidence across all trials to indicate that four key elements of context (personal, organisational, trial and problem context) are crucial to understanding how a complex intervention works and to enable both assessments of internal validity and likely generalizability to other settings. The ways in which context challenged trial operation was often complex, idiosyncratic, and subtle; often falling outside of current trial reporting formats. However, information on such issues appeared to be available via first hand "insider accounts" of each trial suggesting that improved reporting on the role of context is possible. CONCLUSIONS: Sufficient detail about context needs to be understood and reported in RCTs of complex interventions, in order for the transferability of complex interventions to be assessed. Improved reporting formats that require and encourage the clarification of both general and project-specific threats to the likely internal and external validity need to be developed. In addition a cultural change is required in which the open and honest reporting of such issues is seen as an indicator of study strength and researcher integrity, rather than a symbol of a poor quality study or investigator ability.     

Reporting on quality of life in randomised controlled trials: bibliographic study

ObjectivesTo examine the frequency and quality of reporting on quality of life in randomised controlled trials.DesignSearch of the Cochrane Controlled Trials Register 1980 to 1997 to identify trials from all disciplines, from oncology, and from cardiovascular medicine that reported on quality of life. Assessment of abstracts from articles published from 1993 to 1996. Assessment of a sample of full reports with a standardised instrument.ResultsDuring 1980-97 reporting on quality of life increased from 0.63% to 4.2% for trials from all disciplines, from 1.5% to 8.2% for cancer trials, and from 0.34% to 3.6% for cardiovascular trials. Of 364 abstracts, 65% reported on drug interventions. Of a sample of 67 full reports, authors of 48 (72%) used 62 established quality of life instruments. In 15 reports (22%) authors developed their own measures, and in 2 (3%) methods were unclear. Response rates were given in 38 (57%), and complete reporting on all items and scales occurred in 31 (46%).ConclusionsLess than 5% of all randomised controlled trials reported on quality of life, and this proportion was below 10% even for cancer trials. A plethora of instruments was used in different studies, and the reporting of methods and results was often inadequate. Standards for the measurement and reporting of quality of life in clinical trials research need to be developed.

Key messages

• We examined the reporting on quality of life in randomised controlled trials listed in the Cochrane Controlled Trials Register
• Although reporting on quality of life increased over time, fewer than 5% of trials overall and fewer than 10% of cancer trials included quality of life in 1997
• Among 67 articles selected at random for detailed examination, a wide range of established and self developed measures of quality of life were used
• Only about half of trials gave response rates, and less than half reported on all items and scales used
• Standards for assessing and reporting quality of life in clinical research trials need to be developed

     

Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta-epidemiological study

Objective To examine whether the association of inadequate or unclear allocation concealment and lack of blinding with biased estimates of intervention effects varies with the nature of the intervention or outcome.Design Combined analysis of data from three meta-epidemiological studies based on collections of meta-analyses.Data sources 146 meta-analyses including 1346 trials examining a wide range of interventions and outcomes.Main outcome measures Ratios of odds ratios quantifying the degree of bias associated with inadequate or unclear allocation concealment, and lack of blinding, for trials with different types of intervention and outcome. A ratio of odds ratios <1 implies that inadequately concealed or non-blinded trials exaggerate intervention effect estimates.Results In trials with subjective outcomes effect estimates were exaggerated when there was inadequate or unclear allocation concealment (ratio of odds ratios 0.69 (95% CI 0.59 to 0.82)) or lack of blinding (0.75 (0.61 to 0.93)). In contrast, there was little evidence of bias in trials with objective outcomes: ratios of odds ratios 0.91 (0.80 to 1.03) for inadequate or unclear allocation concealment and 1.01 (0.92 to 1.10) for lack of blinding. There was little evidence for a difference between trials of drug and non-drug interventions. Except for trials with all cause mortality as the outcome, the magnitude of bias varied between meta-analyses.Conclusions The average bias associated with defects in the conduct of randomised trials varies with the type of outcome. Systematic reviewers should routinely assess the risk of bias in the results of trials, and should report meta-analyses restricted to trials at low risk of bias either as the primary analysis or in conjunction with less restrictive analyses.     

A prospective multiple case study of the impact of emerging scientific evidence on established colorectal cancer screening programs: a study protocol

In this editorial, we reflect on the arguments for starting a scientific society focused on research on how to improve healthcare. This society would take an inclusive approach to what constitutes healthcare. For instance, it should include mental health healthcare, treatment for substance abuse, the work of allied health professions, and preventive healthcare. The society would be open to researchers from all traditions. Thus, we take an inclusive approach to what constitutes scientific research, as long as it uses rigorous methods, is focused on improving healthcare, and aims at knowledge that can be transferred across settings. The society would primarily target scientific researchers but would invite others with an interest in this area of research, regardless of their discipline, position, field of application, or group affiliation (e.g., improvement science, behavioral medicine, knowledge translation). A society would need fruitful collaboration with related societies and organizations, which may include having combined meetings. Special links may be developed with one or more journals. A website to provide information on relevant resources, events, and training opportunities is another key activity. It would also provide a voice for the field at funding agencies, political arenas, and similar institutions. An organizational structure and financial resources are required to develop and run these activities. Our aim is to start an international debate, to discover if we can establish a shared vision across academics and stakeholders engaged with creating scientific knowledge on how to improve healthcare. We invite readers to express their views in the online questionnaire accessed by following the URL link provided at the end of the editorial.     

Foliicolous lichens from Valdivian temperate rain forest of Chile and Argentina: evidence of an austral element, with the description of seven new taxa

Aim To describe the composition, diversity and biogeographical affinities of the foliicolous lichen mycobiota in Valdivian temperate rain forest in southern South America. Location Seven localities in Chile (IX to XI region) and Argentina (Neuquén province). Methods Opportunistic sampling of leaf substrates, identification of taxa, and assignment to distribution types. Results Thirty‐seven species, including three lichenicolous fungi, were found in the studied collections, increasing the number of foliicolous taxa known from Valdivian temperate rain forest to 55. New records for the area include Arthonia cyanea, Byssoloma marginatum, B. subdiscordans, Fellhanera bouteillei, F. dispersa, F. subfuscatula, Gyalectidium caucasicum, G. ciliatum, Logilvia gilva, Porina thaxteri, Strigula nitidula and the lichenicolous A. microsticta, Helicobolomyces lichenicola and Opegrapha sipmanii. Seven taxa are described as new: Enterographa falcata, Gyalectidium chilense, G. plicatum, Gyalideopsis choshuencensis, Porina fulvelloides, Strigula wandae and Trichothelium meridionale ssp. austroamericanum. Conclusions There are unexpected floristic affinities in the foliicolous lichen mycobiota of Valdivian temperate rain forest with those of New Zealand and Tasmania. Three typically foliicolous species clearly belong to an austral element: Caprettia setifera and Badimiella pteridophila, both known previously from New Zealand and Tasmania; and Kantvilasia hians, known already from Tasmania and Valdivian temperate rain forest. Other Southern Hemisphere lichens, such as Parmeliella nigrocincta, P. thysanota and Psoroma caliginosum, are also found commonly on leaves. On the other hand, specific affinities of the foliicolous lichen mycobiota of Valdivian temperate rain forest with the Neotropics are absent: most of the species shared between the two regions belong to a cosmopolitan‐tropical or a circumpacific element. Thus far, nine taxa are endemic to Valdivia. The Valdivian temperate rain forest foliicolous lichen mycobiota is therefore regarded as one of six distinctive regions in the world.     

A randomised controlled trial of an exercise plus behaviour change intervention in people with multiple sclerosis: the step it up study protocol

Background

Exercise has consistently yielded short-term, positive effects on health outcomes in people with multiple sclerosis (MS). However, these effects have not been maintained in the long-term. Behaviour change interventions aim to promote long-term positive lifestyle change. This study, namely, “Step it Up” will compare the effect of an exercise plus Social Cognitive Theory (SCT)-based behaviour change intervention with an exercise plus control education intervention on walking mobility among people with MS.

Methods/design

People with a diagnosis of MS who walk independently, score of 0–3 on the Patient Determined Disease Steps, who have not experienced an MS relapse or change in their MS medication in the last 12 weeks and who are physically inactive will be randomised to one of two study conditions. The experimental group will undergo a 10-week exercise plus SCT-based behavioural change intervention. The control group will undergo a 10-week exercise plus education intervention to control for contact. Participants will be assessed at weeks 1, 12, 24 and 36. The primary outcome will be walking mobility. Secondary outcomes will include: aerobic capacity, lower extremity muscle strength, participant adherence to the exercise programme, self-report exercise intensity, self-report enjoyment of exercise, exercise self-efficacy, outcome expectations for exercise, goal-setting for exercise, perceived benefits and barriers to exercise, perceptions of social support, physical and psychological impact of MS and fatigue. A qualitative evaluation of Step it Up will be completed among participants post-intervention.

Discussion

This randomised controlled trial will examine the effectiveness of an exercise plus SCT-based behaviour change intervention on walking mobility among people with MS. To this end, Step it Up will serve to inform future directions of research and clinical practice with regard to sustainable exercise interventions for people with MS.

Trial registration

ClinicalTrials.gov, https://clinicaltrials.gov/ct2/show/NCT02301442

     

Conflicts of interest at medical journals: the influence of industry-supported randomised trials on journal impact factors and revenue - cohort study

Background

Transparency in reporting of conflict of interest is an increasingly important aspect of publication in medical journals. Publication of large industry-supported trials may generate many citations and journal income through reprint sales and thereby be a source of conflicts of interest for journals. We investigated industry-supported trials'' influence on journal impact factors and revenue.

Methods and Findings

We sampled six major medical journals (Annals of Internal Medicine, Archives of Internal Medicine, BMJ, JAMA, The Lancet, and New England Journal of Medicine [NEJM]). For each journal, we identified randomised trials published in 1996–1997 and 2005–2006 using PubMed, and categorized the type of financial support. Using Web of Science, we investigated citations of industry-supported trials and the influence on journal impact factors over a ten-year period. We contacted journal editors and retrieved tax information on income from industry sources. The proportion of trials with sole industry support varied between journals, from 7% in BMJ to 32% in NEJM in 2005–2006. Industry-supported trials were more frequently cited than trials with other types of support, and omitting them from the impact factor calculation decreased journal impact factors. The decrease varied considerably between journals, with 1% for BMJ to 15% for NEJM in 2007. For the two journals disclosing data, income from the sales of reprints contributed to 3% and 41% of the total income for BMJ and The Lancet in 2005–2006.

Conclusions

Publication of industry-supported trials was associated with an increase in journal impact factors. Sales of reprints may provide a substantial income. We suggest that journals disclose financial information in the same way that they require them from their authors, so that readers can assess the potential effect of different types of papers on journals'' revenue and impact. Please see later in the article for the Editors'' Summary     

Why zygosity of multiple births is not always obvious: an examination of zygosity testing requests from twins or their parents.

This paper examines why parents of twins or adult twins themselves request zygosity testing. Of 405 multiples including 8 sets of triplets, the majority (93%) were monozygotic. Age of testing ranged from 0 days to 73 years. About 50% of requests came from parents or twins who were curious about, or expressed a need to be certain of, their zygosity. Other reasons included health concerns (current or future), other twins in the family, and misinformation about zygosity, frequently because of the erroneous assumption that all dichorionic twins are dizygotic. Parents of monozygotic twins may expect their twins to be 'identical' and believe their twins to be dizygotic because of minor phenotypic differences between them. Dizygotic twins like other siblings may share a phenotypic resemblance. Health professionals should be aware that zygosity of multiples may not always be obvious to parents and that accurate knowledge of zygosity may be justified.     

The effects of an acute bout of exercise on neural activity in alcohol and cocaine craving: study protocol for a randomised controlled trial

Background

Numerous studies suggest that exercise may be an effective adjunct treatment for substance use disorders. It has been suggested that exercise-induced improvements in inhibitory control may reduce craving for the substance of abuse. However, this potential mechanism has seldom been researched.

Objectives

The aim of the ExAlCo Study is to examine how acute bouts of exercise, at varying intensities, impact on craving for cocaine or alcohol. Cerebral haemodynamic responses during cognitive tests of inhibitory control, and exposure to substance-related cue imagery, will also be assessed using functional near-infrared spectroscopy.

Design

The study is a crossover randomised controlled trial. Participants will be recruited from inpatient and outpatient psychiatric treatment centres, on the approval of their treating physician. A healthy control group will be recruited using online advertising. All participants will undergo each of three conditions in randomised order: 20?min of cycle ergometry at 50–60% of maximum heart rate; 20?min of exercise at 70–80% of maximum heart rate; and 20?min of quiet reading. Immediately before and after each condition, participants will be asked to complete a computerised Stroop test, watch a film containing substance-related images and self-report craving levels. During the Stroop test and film viewing, participants’ neural activity will be measured via functional near-infrared spectroscopy.

Outcomes

The primary outcome measures are self-reported craving, inhibitory control and cerebral haemodynamic response to the Stroop test and a substance-related film. It is hoped that the findings from this study will shed more light on the role of exercise in the treatment of substance use disorders, particularly its scope in preventing relapse through reduced craving severity.

Trial registration

ClinicalTrials.gov, NCT03502486. Registered retrospectively on 5 April 2018.

     

Integrating human health impact from indoor emissions into an LCA: a case study evaluating the significance of the use stage

Purpose

Indoor emissions of toxic substances from products can have a negative effect on human health. These are typically not considered in a life cycle assessment (LCA), potentially underestimating the importance of the use stage. The purpose of this paper is to develop a method that, based on a set of measured emission rates, calculates the impact on human health during the use stage of products that are used indoors and that emit volatile organic compounds (VOCs).

Methods

Emissions from a product are measured in a test chamber and reported as a set of emission rates (microgrammes per hour) at specific points in time (hour/day). Constrained non-linear regression (CNLR) analysis is then used to determine parameters for three emission models, and a model is selected based on goodness of fit with the measured emission rates (R ² and expert judgement). The emission model is integrated over a defined time period to estimate the total use stage emissions per functional unit (FU). The total emissions are subsequently integrated in a homogeneously mixed one-box model within the USEtox model. Intake fraction (iF) is calculated based on size of residential home, inhalation rate, exposure time, ventilation rate, mixing factor and number of people exposed.

Results and discussion

The method is tested in a case study of a chair, with the results showing that the impacts in the use stage are in most cases significantly higher than from the production and disposal stages combined. The sensitivity to parameter variations is evaluated. Intake fraction (factor of 761), replacement frequency (factor of 70) and emission model (factor of 24) are found to be the most important model parameters. Limiting early exposure (>14 % of emissions may occur in the first month and >50 % in the first year) and replacing furniture less frequently will reduce exposure.

Conclusions

The case study shows that the impact on human health from indoor emissions can be of significance, when compared to the impact on human health from total outdoor emissions. Without specific exposure data (e.g. ventilation rates) the uncertainty will be high. The developed method is applicable to all products that emit VOCs, provided that the emission rate can be modelled using an exponential decay model and that the product amount is related to a meaningful functional unit. It is recommended that when performing an LCA of products that emit VOCs, the indoor use stage is included in the life cycle impact assessment.     

The impact of crystallization conditions on structure‐based drug design: A case study on the methylene blue/acetylcholinesterase complex

Structure‐based drug design utilizes apoprotein or complex structures retrieved from the PDB. >57% of crystallographic PDB entries were obtained with polyethylene glycols (PEGs) as precipitant and/or as cryoprotectant, but <6% of these report presence of individual ethyleneglycol oligomers. We report a case in which ethyleneglycol oligomers' presence in a crystal structure markedly affected the bound ligand's position. Specifically, we compared the positions of methylene blue and decamethonium in acetylcholinesterase complexes obtained using isomorphous crystals precipitated with PEG200 or ammonium sulfate. The ligands' positions within the active‐site gorge in complexes obtained using PEG200 are influenced by presence of ethyleneglycol oligomers in both cases bound to W84 at the gorge's bottom, preventing interaction of the ligand's proximal quaternary group with its indole. Consequently, both ligands are ～3.0Å further up the gorge than in complexes obtained using crystals precipitated with ammonium sulfate, in which the quaternary groups make direct π‐cation interactions with the indole. These findings have implications for structure‐based drug design, since data for ligand‐protein complexes with polyethylene glycol as precipitant may not reflect the ligand's position in its absence, and could result in selecting incorrect drug discovery leads. Docking methylene blue into the structure obtained with PEG200, but omitting the ethyleneglycols, yields results agreeing poorly with the crystal structure; excellent agreement is obtained if they are included. Many proteins display features in which precipitants might lodge. It will be important to investigate presence of precipitants in published crystal structures, and whether it has resulted in misinterpreting electron density maps, adversely affecting drug design.     

The use of islands and cluster-randomized trials to investigate vector control interventions: a case study on the Bijagós archipelago,Guinea-Bissau

Vector-borne diseases threaten the health of populations around the world. While key interventions continue to provide protection from vectors, there remains a need to develop and test new vector control tools. Cluster-randomized trials, in which the intervention or control is randomly allocated to clusters, are commonly selected for such evaluations, but their design must carefully consider cluster size and cluster separation, as well as the movement of people and vectors, to ensure sufficient statistical power and avoid contamination of results. Island settings present an opportunity to conduct these studies. Here, we explore the benefits and challenges of conducting intervention studies on islands and introduce the Bijagós archipelago of Guinea-Bissau as a potential study site for interventions intended to control vector-borne diseases.This article is part of the theme issue ‘Novel control strategies for mosquito-borne diseases''.