Outcomes for patients with the same disease treated inside and outside of randomized trials: a systematic review and meta-analysis

Background:

It is unclear whether participation in a randomized controlled trial (RCT), irrespective of assigned treatment, is harmful or beneficial to participants. We compared outcomes for patients with the same diagnoses who did (“insiders”) and did not (“outsiders”) enter RCTs, without regard to the specific therapies received for their respective diagnoses.

Methods:

By searching the MEDLINE (1966–2010), Embase (1980–2010), CENTRAL (1960–2010) and PsycINFO (1880–2010) databases, we identified 147 studies that reported the health outcomes of “insiders” and a group of parallel or consecutive “outsiders” within the same time period. We prepared a narrative review and, as appropriate, meta-analyses of patients’ outcomes.

Results:

We found no clinically or statistically significant differences in outcomes between “insiders” and “outsiders” in the 23 studies in which the experimental intervention was ineffective (standard mean difference in continuous outcomes −0.03, 95% confidence interval CI] −0.1 to 0.04) or in the 7 studies in which the experimental intervention was effective and was received by both “insiders” and “outsiders” (mean difference 0.04, 95% CI −0.04 to 0.13). However, in 9 studies in which an effective intervention was received only by “insiders,” the “outsiders” experienced significantly worse health outcomes (mean difference −0.36, 95% CI −0.61 to −0.12).

Interpretation:

We found no evidence to support clinically important overall harm or benefit arising from participation in RCTs. This conclusion refutes earlier claims that trial participants are at increased risk of harm.When people are asked to participate in a randomized controlled trial (RCT), it is natural for them to ask several questions in return. How safe are these treatments? How many extra visits and tests must I undergo? Will the researchers keep my family doctor informed about what’s going on? What outcomes are to be measured, and do they include ones that are of interest to me as a patient?These multiple questions can be summarized as follows: Would I fare better being treated within the trial (as an “insider”) or in routine clinical care outside it (as an “outsider”)? Patients may ask this question in 1 of 2 ways. The first is highly specific: “Am I better off receiving this specific treatment as an insider or as an outsider?” Alternatively, they might ask a more general question: “Am I better off having my illness managed, regardless of the specific treatment I would receive, as an insider or as an outsider?” These questions are highly appropriate, and both deserve to be asked and answered,^1,2 especially given that nonsystematic reviews have suggested a possible “inclusion benefit” from participating in trials.³These 2 specific patient questions are analogous to those posed by researchers asking whether treatments do more good than harm when applied under “ideal” circumstances (in explanatory trials) or in the “real world” of routine health care (in pragmatic trials). Vist and colleagues answered the explanatory question when their earlier review⁴ found no advantage or disadvantage from receiving the same treatment inside or outside an RCT. Left unanswered, however, was the broader, more pragmatic question. In our experience, trial participants are often offered new, as-yet-untested treatments that would not be available to them outside the trial. This review looks at the dilemma faced by these patients, which needs to be addressed before general conclusions can be drawn about trial safety.