Intervention description is not enough: evidence from an in-depth multiple case study on the untold role and impact of context in randomised controlled trials of seven complex interventions

ABSTRACT: BACKGROUND: A number of single case reports have suggested that the context within which intervention studies take place may challenge the assumptions that underpin RCTs. However, the diverse ways in which context may challenge the central tenets of the RCT, and the degree to which this information is known to researchers and/or subsequently reported, has received much less attention. In this paper we explore these issues by focussing on 7 RCTs of interventions ranging in type and degree of complexity, and across diverse contexts. METHODS: This in-depth multiple case study using interviews, focus groups and documentary analysis was conducted in two phases. In phase one, a RCT of a nurse-led intervention provided a single exploratory case and informed the design, sampling and data collection within the main study. Phase two consisted of a multiple explanatory case study covering a range of trials of different types of complex intervention. A total of 84 data sources across the 7 trials were accessed. RESULTS: We present consistent empirical evidence across all trials to indicate that four key elements of context (personal, organisational, trial and problem context) are crucial to understanding how a complex intervention works and to enable both assessments of internal validity and likely generalizability to other settings. The ways in which context challenged trial operation was often complex, idiosyncratic, and subtle; often falling outside of current trial reporting formats. However, information on such issues appeared to be available via first hand "insider accounts" of each trial suggesting that improved reporting on the role of context is possible. CONCLUSIONS: Sufficient detail about context needs to be understood and reported in RCTs of complex interventions, in order for the transferability of complex interventions to be assessed. Improved reporting formats that require and encourage the clarification of both general and project-specific threats to the likely internal and external validity need to be developed. In addition a cultural change is required in which the open and honest reporting of such issues is seen as an indicator of study strength and researcher integrity, rather than a symbol of a poor quality study or investigator ability.     

Reporting on quality of life in randomised controlled trials: bibliographic study

ObjectivesTo examine the frequency and quality of reporting on quality of life in randomised controlled trials.DesignSearch of the Cochrane Controlled Trials Register 1980 to 1997 to identify trials from all disciplines, from oncology, and from cardiovascular medicine that reported on quality of life. Assessment of abstracts from articles published from 1993 to 1996. Assessment of a sample of full reports with a standardised instrument.ResultsDuring 1980-97 reporting on quality of life increased from 0.63% to 4.2% for trials from all disciplines, from 1.5% to 8.2% for cancer trials, and from 0.34% to 3.6% for cardiovascular trials. Of 364 abstracts, 65% reported on drug interventions. Of a sample of 67 full reports, authors of 48 (72%) used 62 established quality of life instruments. In 15 reports (22%) authors developed their own measures, and in 2 (3%) methods were unclear. Response rates were given in 38 (57%), and complete reporting on all items and scales occurred in 31 (46%).ConclusionsLess than 5% of all randomised controlled trials reported on quality of life, and this proportion was below 10% even for cancer trials. A plethora of instruments was used in different studies, and the reporting of methods and results was often inadequate. Standards for the measurement and reporting of quality of life in clinical trials research need to be developed.

Key messages

• We examined the reporting on quality of life in randomised controlled trials listed in the Cochrane Controlled Trials Register
• Although reporting on quality of life increased over time, fewer than 5% of trials overall and fewer than 10% of cancer trials included quality of life in 1997
• Among 67 articles selected at random for detailed examination, a wide range of established and self developed measures of quality of life were used
• Only about half of trials gave response rates, and less than half reported on all items and scales used
• Standards for assessing and reporting quality of life in clinical research trials need to be developed

     

Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta-epidemiological study

Objective To examine whether the association of inadequate or unclear allocation concealment and lack of blinding with biased estimates of intervention effects varies with the nature of the intervention or outcome.Design Combined analysis of data from three meta-epidemiological studies based on collections of meta-analyses.Data sources 146 meta-analyses including 1346 trials examining a wide range of interventions and outcomes.Main outcome measures Ratios of odds ratios quantifying the degree of bias associated with inadequate or unclear allocation concealment, and lack of blinding, for trials with different types of intervention and outcome. A ratio of odds ratios <1 implies that inadequately concealed or non-blinded trials exaggerate intervention effect estimates.Results In trials with subjective outcomes effect estimates were exaggerated when there was inadequate or unclear allocation concealment (ratio of odds ratios 0.69 (95% CI 0.59 to 0.82)) or lack of blinding (0.75 (0.61 to 0.93)). In contrast, there was little evidence of bias in trials with objective outcomes: ratios of odds ratios 0.91 (0.80 to 1.03) for inadequate or unclear allocation concealment and 1.01 (0.92 to 1.10) for lack of blinding. There was little evidence for a difference between trials of drug and non-drug interventions. Except for trials with all cause mortality as the outcome, the magnitude of bias varied between meta-analyses.Conclusions The average bias associated with defects in the conduct of randomised trials varies with the type of outcome. Systematic reviewers should routinely assess the risk of bias in the results of trials, and should report meta-analyses restricted to trials at low risk of bias either as the primary analysis or in conjunction with less restrictive analyses.     

A prospective multiple case study of the impact of emerging scientific evidence on established colorectal cancer screening programs: a study protocol

In this editorial, we reflect on the arguments for starting a scientific society focused on research on how to improve healthcare. This society would take an inclusive approach to what constitutes healthcare. For instance, it should include mental health healthcare, treatment for substance abuse, the work of allied health professions, and preventive healthcare. The society would be open to researchers from all traditions. Thus, we take an inclusive approach to what constitutes scientific research, as long as it uses rigorous methods, is focused on improving healthcare, and aims at knowledge that can be transferred across settings. The society would primarily target scientific researchers but would invite others with an interest in this area of research, regardless of their discipline, position, field of application, or group affiliation (e.g., improvement science, behavioral medicine, knowledge translation). A society would need fruitful collaboration with related societies and organizations, which may include having combined meetings. Special links may be developed with one or more journals. A website to provide information on relevant resources, events, and training opportunities is another key activity. It would also provide a voice for the field at funding agencies, political arenas, and similar institutions. An organizational structure and financial resources are required to develop and run these activities. Our aim is to start an international debate, to discover if we can establish a shared vision across academics and stakeholders engaged with creating scientific knowledge on how to improve healthcare. We invite readers to express their views in the online questionnaire accessed by following the URL link provided at the end of the editorial.     

A randomised controlled trial of an exercise plus behaviour change intervention in people with multiple sclerosis: the step it up study protocol

Background

Exercise has consistently yielded short-term, positive effects on health outcomes in people with multiple sclerosis (MS). However, these effects have not been maintained in the long-term. Behaviour change interventions aim to promote long-term positive lifestyle change. This study, namely, “Step it Up” will compare the effect of an exercise plus Social Cognitive Theory (SCT)-based behaviour change intervention with an exercise plus control education intervention on walking mobility among people with MS.

Methods/design

People with a diagnosis of MS who walk independently, score of 0–3 on the Patient Determined Disease Steps, who have not experienced an MS relapse or change in their MS medication in the last 12 weeks and who are physically inactive will be randomised to one of two study conditions. The experimental group will undergo a 10-week exercise plus SCT-based behavioural change intervention. The control group will undergo a 10-week exercise plus education intervention to control for contact. Participants will be assessed at weeks 1, 12, 24 and 36. The primary outcome will be walking mobility. Secondary outcomes will include: aerobic capacity, lower extremity muscle strength, participant adherence to the exercise programme, self-report exercise intensity, self-report enjoyment of exercise, exercise self-efficacy, outcome expectations for exercise, goal-setting for exercise, perceived benefits and barriers to exercise, perceptions of social support, physical and psychological impact of MS and fatigue. A qualitative evaluation of Step it Up will be completed among participants post-intervention.

Discussion

This randomised controlled trial will examine the effectiveness of an exercise plus SCT-based behaviour change intervention on walking mobility among people with MS. To this end, Step it Up will serve to inform future directions of research and clinical practice with regard to sustainable exercise interventions for people with MS.

Trial registration

ClinicalTrials.gov, https://clinicaltrials.gov/ct2/show/NCT02301442

     

Conflicts of interest at medical journals: the influence of industry-supported randomised trials on journal impact factors and revenue - cohort study

Background

Transparency in reporting of conflict of interest is an increasingly important aspect of publication in medical journals. Publication of large industry-supported trials may generate many citations and journal income through reprint sales and thereby be a source of conflicts of interest for journals. We investigated industry-supported trials'' influence on journal impact factors and revenue.

Methods and Findings

We sampled six major medical journals (Annals of Internal Medicine, Archives of Internal Medicine, BMJ, JAMA, The Lancet, and New England Journal of Medicine [NEJM]). For each journal, we identified randomised trials published in 1996–1997 and 2005–2006 using PubMed, and categorized the type of financial support. Using Web of Science, we investigated citations of industry-supported trials and the influence on journal impact factors over a ten-year period. We contacted journal editors and retrieved tax information on income from industry sources. The proportion of trials with sole industry support varied between journals, from 7% in BMJ to 32% in NEJM in 2005–2006. Industry-supported trials were more frequently cited than trials with other types of support, and omitting them from the impact factor calculation decreased journal impact factors. The decrease varied considerably between journals, with 1% for BMJ to 15% for NEJM in 2007. For the two journals disclosing data, income from the sales of reprints contributed to 3% and 41% of the total income for BMJ and The Lancet in 2005–2006.

Conclusions

Publication of industry-supported trials was associated with an increase in journal impact factors. Sales of reprints may provide a substantial income. We suggest that journals disclose financial information in the same way that they require them from their authors, so that readers can assess the potential effect of different types of papers on journals'' revenue and impact. Please see later in the article for the Editors'' Summary     

Why zygosity of multiple births is not always obvious: an examination of zygosity testing requests from twins or their parents.

This paper examines why parents of twins or adult twins themselves request zygosity testing. Of 405 multiples including 8 sets of triplets, the majority (93%) were monozygotic. Age of testing ranged from 0 days to 73 years. About 50% of requests came from parents or twins who were curious about, or expressed a need to be certain of, their zygosity. Other reasons included health concerns (current or future), other twins in the family, and misinformation about zygosity, frequently because of the erroneous assumption that all dichorionic twins are dizygotic. Parents of monozygotic twins may expect their twins to be 'identical' and believe their twins to be dizygotic because of minor phenotypic differences between them. Dizygotic twins like other siblings may share a phenotypic resemblance. Health professionals should be aware that zygosity of multiples may not always be obvious to parents and that accurate knowledge of zygosity may be justified.     

The effects of an acute bout of exercise on neural activity in alcohol and cocaine craving: study protocol for a randomised controlled trial

Background

Numerous studies suggest that exercise may be an effective adjunct treatment for substance use disorders. It has been suggested that exercise-induced improvements in inhibitory control may reduce craving for the substance of abuse. However, this potential mechanism has seldom been researched.

Objectives

The aim of the ExAlCo Study is to examine how acute bouts of exercise, at varying intensities, impact on craving for cocaine or alcohol. Cerebral haemodynamic responses during cognitive tests of inhibitory control, and exposure to substance-related cue imagery, will also be assessed using functional near-infrared spectroscopy.

Design

The study is a crossover randomised controlled trial. Participants will be recruited from inpatient and outpatient psychiatric treatment centres, on the approval of their treating physician. A healthy control group will be recruited using online advertising. All participants will undergo each of three conditions in randomised order: 20?min of cycle ergometry at 50–60% of maximum heart rate; 20?min of exercise at 70–80% of maximum heart rate; and 20?min of quiet reading. Immediately before and after each condition, participants will be asked to complete a computerised Stroop test, watch a film containing substance-related images and self-report craving levels. During the Stroop test and film viewing, participants’ neural activity will be measured via functional near-infrared spectroscopy.

Outcomes

The primary outcome measures are self-reported craving, inhibitory control and cerebral haemodynamic response to the Stroop test and a substance-related film. It is hoped that the findings from this study will shed more light on the role of exercise in the treatment of substance use disorders, particularly its scope in preventing relapse through reduced craving severity.

Trial registration

ClinicalTrials.gov, NCT03502486. Registered retrospectively on 5 April 2018.

     

Integrating human health impact from indoor emissions into an LCA: a case study evaluating the significance of the use stage

Purpose

Indoor emissions of toxic substances from products can have a negative effect on human health. These are typically not considered in a life cycle assessment (LCA), potentially underestimating the importance of the use stage. The purpose of this paper is to develop a method that, based on a set of measured emission rates, calculates the impact on human health during the use stage of products that are used indoors and that emit volatile organic compounds (VOCs).

Methods

Emissions from a product are measured in a test chamber and reported as a set of emission rates (microgrammes per hour) at specific points in time (hour/day). Constrained non-linear regression (CNLR) analysis is then used to determine parameters for three emission models, and a model is selected based on goodness of fit with the measured emission rates (R ² and expert judgement). The emission model is integrated over a defined time period to estimate the total use stage emissions per functional unit (FU). The total emissions are subsequently integrated in a homogeneously mixed one-box model within the USEtox model. Intake fraction (iF) is calculated based on size of residential home, inhalation rate, exposure time, ventilation rate, mixing factor and number of people exposed.

Results and discussion

The method is tested in a case study of a chair, with the results showing that the impacts in the use stage are in most cases significantly higher than from the production and disposal stages combined. The sensitivity to parameter variations is evaluated. Intake fraction (factor of 761), replacement frequency (factor of 70) and emission model (factor of 24) are found to be the most important model parameters. Limiting early exposure (>14 % of emissions may occur in the first month and >50 % in the first year) and replacing furniture less frequently will reduce exposure.

Conclusions

The case study shows that the impact on human health from indoor emissions can be of significance, when compared to the impact on human health from total outdoor emissions. Without specific exposure data (e.g. ventilation rates) the uncertainty will be high. The developed method is applicable to all products that emit VOCs, provided that the emission rate can be modelled using an exponential decay model and that the product amount is related to a meaningful functional unit. It is recommended that when performing an LCA of products that emit VOCs, the indoor use stage is included in the life cycle impact assessment.     

Effect of peginterferon beta-1a on MRI measures and achieving no evidence of disease activity: results from a randomized controlled trial in relapsing-remitting multiple sclerosis

Background

Subcutaneous peginterferon beta-1a provided clinical benefits at Year 1 (placebo-controlled period) of the 2-Year Phase 3 ADVANCE study in relapsing-remitting multiple sclerosis (RRMS). Here we report the effect of peginterferon beta-1a on brain magnetic resonance imaging (MRI) lesions, and no evidence of disease activity (NEDA; absence of clinical [relapses and 12-week confirmed disability progression] and MRI [gadolinium-enhancing, and new or newly-enlarging T2 hyperintense lesions] disease activity) during Year 1.

Methods

RRMS patients (18–65 years; Expanded Disability Status Scale score ≤5) were randomized to double-blind placebo or peginterferon beta-1a 125 μg every 2 or 4 weeks. Sensitivity analyses of last observation carried forward and composite disease activity (using minimal MRI allowance definitions) were conducted.

Results

1512 patients were randomized and dosed (placebo n?=?500; peginterferon beta-1a every 2 [n?=?512] or 4 [n?=?500] weeks). Every 2 week dosing significantly reduced, versus placebo and every 4 week dosing, the number of new or newly-enlarging T2 hyperintense lesions at Weeks 24 (by 61% and 51%, respectively) and 48 (secondary endpoint; by 67% and 54%, respectively); all p?<?0.0001. Every 2 week dosing also provided significant reductions versus placebo and every 4 week dosing in the number of new T1 hypointense, gadolinium-enhancing, and new active (gadolinium-enhancing plus non-enhancing new T2) lesions (all p?<?0.0001), as well as the volume of T2 and T1 lesions (p?<?0.05) at Weeks 24 and 48. Significantly more patients dosed every 2 weeks had NEDA versus placebo and every 4 weeks (all p?<?0.01) from baseline to Week 48 (33.9% versus 15.1% and 21.5%, respectively [odds ratios, ORs: 2.89 and 1.87]), from baseline to Week 24 (41.0% versus 21.9% and 30.7%, [ORs: 2.47 and 1.57]) and from Week 24 to Week 48 (60.2% versus 28.9% and 36.6%, [ORs: 3.71 and 2.62]). Consistent results were seen when allowing for minimal MRI activity.

Conclusion

During Year 1 of ADVANCE, significantly more RRMS patients receiving peginterferon beta-1a every 2 weeks had NEDA, and early and sustained improvements in all MRI endpoints, versus placebo and every 4 week dosing. NEDA sensitivity analyses align with switch strategies in clinical practice settings and provide insight into future responders/non-responders.

Trial registration

ClinicalTrials.gov: https://clinicaltrials.gov/ct2/show/NCT00906399

     

The impact of data precision on the effectiveness of alien plant control programmes: a case study from a protected area

Successful long-term invasive alien plant control programmes rely on alien plant distribution and abundance data to assess, prioritise, implement and monitor the efficacy of the programme. Here we assess the impact of data accuracy using the alien plant programme in Table Mountain National Park, South Africa. A systematic plot-based survey method was carried out to assess the distribution of alien plants in the park at a fine scale (systematic sampling). Alien plant richness, total area invaded and the degree of spatial overlap in species’ presence were compared between the systematic sample and a protected area (PA) managers’ dataset (collated from collective observations by park visitors, rangers and managers) and Working for Water (WfW) project data (data collected for the planning and implementation of the alien plant clearing programme) using a range of confusion matrix-based statistics to assess similarity and error rates between the datasets. A total of 106 alien plant taxa were detected across the three datasets, 12 in PA manager’s data, 23 in WfW data and 101 in the systematic survey. Overall, there was substantive disagreement between the datasets on the distribution of alien plants. For example both management datasets estimated species’ hectare coverage at orders of magnitude greater than indicated by systematic sampling. The inaccuracy of manager data has direct negative implications for funding allocation, which currently appears to be in excess of what is required. We recommend that contrary to perception, fine-scale surveys are a cost-effective way to inform long-term monitoring programmes and improve programme effectiveness.     

Exosome-producing follicle associated epithelium is not involved in uptake of PrPd from the gut of sheep (Ovis aries): an ultrastructural study

In natural or experimental oral scrapie infection of sheep, disease associated prion protein (PrP(d)) often first accumulates in Peyer's patch (PP) follicles. The route by which infectivity reaches the follicles is unknown, however, intestinal epithelial cells may participate in intestinal antigenic presentation by delivering exosomes as vehicles of luminal antigens. In a previous study using an intestinal loop model, following inoculation of scrapie brain homogenate, inoculum associated PrP(d) was detected by light microscopy shortly (15 minutes to 3.5 hours) after inoculation in the villous lacteals and sub-mucosal lymphatics. No PrP(d) was located within the follicle-associated epithelium (FAE), sub-FAE domes or the PP follicles. To evaluate this gut loop model and the transportation routes in more detail, we used electron microscopy (EM) to study intestinal tissues exposed to scrapie or control homogenates for 15 minutes to 10 days. In addition, immuno-EM was used to investigate whether exosomes produced in the FAE may possess small amounts of PrP(d) that were not detectable by light microscopy. This study showed that the integrity of the intestinal epithelium was sustained in the intestinal loop model. Despite prominent transcytotic activity and exosome release from the FAE of the ileal PP in sheep, these structures were not associated with transportation of PrP(d) across the mucosa. The study did not determine how infectivity reaches the follicles of PPs. The possibility that the infectious agent is transported across the FAE remains a possibility if it occurs in a form that is undetectable by the methods used in this study. Infectivity may also be transported via lymph to the blood and further to all other lymphoid tissues including the PP follicles, but the early presence of PrP(d) in the PP follicles during scrapie infection argues against such a mechanism.