Developing core outcome sets for clinical trials: issues to consider

ABSTRACT: Selection of appropriate outcomes or domains is crucial when designing clinical trials to compare directly the effects of different interventions in ways that minimise bias. If the findings are to influence policy and practice then the chosen outcomes need to be relevant and important to key stakeholders including patients and the public, health care professionals and others making decisions about health care. There is a growing recognition that insufficient attention has been paid to the outcomes measured in clinical trials. These issues could be addressed through the development and use of an agreed standardised collection of outcomes, known as a core outcome set, which should be measured and reported, as a minimum, in all trials for a specific clinical area. Accumulating work in this area has identified the need for general guidance on the development of core outcome sets. Key issues to consider in the development of a core outcome set include its scope, the stakeholder groups to be involved, choice of consensus method and the achievement of a consensus.     

Core Outcomes for Colorectal Cancer Surgery: A Consensus Study

BackgroundColorectal cancer (CRC) is a major cause of worldwide morbidity and mortality. Surgical treatment is common, and there is a great need to improve the delivery of such care. The gold standard for evaluating surgery is within well-designed randomized controlled trials (RCTs); however, the impact of RCTs is diminished by a lack of coordinated outcome measurement and reporting. A solution to these issues is to develop an agreed standard “core” set of outcomes to be measured in all trials to facilitate cross-study comparisons, meta-analysis, and minimize outcome reporting bias. This study defines a core outcome set for CRC surgery.ConclusionThis study used robust consensus methodology to develop a core outcome set for use in colorectal cancer surgical trials. It is now necessary to validate the use of this set in research practice.     

Evaluating Outcomes Used in Cardiothoracic Surgery Interventional Research: A Systematic Review of Reviews to Develop a Core Outcome Set

Background

When planning clinical trials, it is a key element to choose appropriate outcomes that ensure the comparability of effects of interventions in ways that minimise bias. We hypothesise that outcome measures in cardiothoracic surgical trials are inconsistent and without standard. Therefore, comparing the relative effectiveness of interventions across studies is problematic. We surmise that cardiothoracic research has focused habitually on the identification of risk factors and on the reduction of adverse outcomes with less consideration of factors that contribute to well being and positive health outcomes (salutogenesis).

Methods and Findings

We conducted a systematic review of reviews to determine both the type and number of outcomes reported in current cardiothoracic surgery interventional research, in order to identify a list of potential outcomes for a minimum core outcome set (COS). Special focus was placed on outcomes that emphasise salutogenesis. We interpreted salutogenic outcomes as those relating to optimum and/or positive health and well being. We searched Issue 7 (July 2014) of the Cochrane Database of Systematic Reviews. Systematic reviews of randomised trials on non-minimal-invasive off- or on-pump cardiothoracic surgery (elective and emergency, excluding transplants) investigating pre-, intra- or postsurgical interventions related to the outcome of the procedure were eligible for inclusion. We excluded protocols and withdrawn systematic reviews. Two review authors extracted outcome data independently. Unique lists of salutogenically and non-salutogenically focused outcomes were established. 15 systematic reviews involving 371 randomized trials and 58,253 patients were included in this review. Applied definitions of single and composite endpoints varied significantly, and patient-centred, salutogenically focused outcomes were seldom reported. One third of included reviews did not assess patient-centred outcomes at all; all other reviews were unable to perform meta-analyses due to an absence of data or heterogeneity in outcome measures. This compares to 36 non-salutogenically focused outcome domains representing 121 individual non-salutogenically focused outcomes, whereof 50% were assessed only once. Measures of mortality, cerebrovascular complications and hospitalisation were reported most frequently. Two reviews chose a composite endpoint as primary outcome. Pooled analysis of composite endpoints was not possible, as the required data was not reported per patient in all components.

Conclusion

In cardiothoracic surgical trials, choice and definition of non-salutogenically focused single and composite outcomes are inconsistent. There is an absence of patient centred, salutogenically focused outcome parameters in cardiac trials. We recommend the development of a core outcome set of salutogenically focused and non-salutogenically focused outcomes for cardiothoracic surgical research.     

Self-help materials for the prevention of smoking relapse: study protocol for a randomized controlled trial

Background

In clinical trials in childhood asthma, outcomes reflecting short-term disease activity are frequently measured, whilst functional status, quality of life (QoL), and long-term treatment effects are rarely assessed. There is also non-uniformity across studies in the selection and measurement of outcomes within these domains. The development of a core outcome set has the potential to reduce heterogeneity between trials, lead to research that is more likely to have measured relevant outcomes, and enhance the value of evidence synthesis by reducing the risk of outcome reporting bias and ensuring that all trials contribute usable information.

Methods

Paediatricians and specialist nurses, identified through the British Paediatric Respiratory Society, completed a two-round Delphi survey. Separate cohorts of parents of children younger than 18?years, recruited in clinics, participated in each round. Young people with asthma, aged at least 13?years, participated in the first round. Outcomes were identified separately for preschool and school-aged children. We identified outcomes considered important in routine clinical assessment by clinicians and parents/young people. In round 1, 46 clinicians suggested outcomes they considered important when deciding whether to adjust a child??s asthma therapy regime, and 49 parents/young people were asked, using open questions, how they judged whether their child??s (for young people, their own) asthma therapy was appropriate. Two researchers independently classified responses into appropriate, corresponding outcomes. In round 2, 43 clinicians and 50 parents scored, from 0?C4, the importance of each outcome suggested by at least 10?% of round 1 responders and selected the three most important.

Results

The most important outcomes, when making shared decisions about regular therapies for school-aged and preschool children with asthma, were daytime and nocturnal symptoms, exacerbations, QoL, and mortality. Results from parents and clinicians were generally concordant, but parents placed more emphasis on long-term treatment effects.

Conclusions

We have developed a methodology to identify outcomes of most relevance to clinicians, parents, and young people when evaluating regularly administered therapies for asthma. Daytime and nocturnal symptoms, exacerbations, QoL, and mortality are particularly important outcomes that should be measured and reported in all clinical trials of regular therapies for children with asthma.     

Clinical outcomes and outcome measurement tools reported in randomised controlled trials of treatment for snakebite envenoming: A systematic review

BackgroundSnakebite is a priority neglected tropical disease and causes a range of complications that vary depending on the snake species. Randomised clinical trials have used varied outcome measures that do not allow results to be compared or combined. In accordance with the Core Outcomes Measurements in Effectiveness Trials (COMET) initiative, this systematic review aims to support the development of a globally relevant core outcome set for snakebite.MethodsAll randomised controlled trials, secondary analyses of randomised controlled trials and study protocols investigating the efficacy of therapeutics for human snakebite envenoming were eligible for inclusion. Study screening and data extraction were conducted in duplicate by two independent reviewers. All primary and secondary outcome measures were extracted and compiled, as were adverse event outcome measures. Similar outcome measures were grouped into domains. The study was prospectively registered with PROSPERO: CRD42020196160.ResultsThis systematic review included 43 randomised controlled trials, two secondary analyses and 13 study protocols. A total of 382 outcome measures were extracted and, after duplicates were merged, there were 153 unique outcomes. The most frequently used outcome domain (‘venom antigenaemia’) was included in less than one third of the studies. The unique outcomes were classified into 60 outcome domains. Patient-centred outcomes were used in only three of the studies.DiscussionSignificant heterogeneity in outcome measures exists in snakebite clinical trials. Consensus is needed to select outcome measures that are valid, reliable, patient-centred and feasible. The results of this systematic review strongly support the development of a core outcome set for use in snakebite clinical trials.     

Development of a core outcome set for clinical trials in rosacea: study protocol for a systematic review of the literature and identification of a core outcome set using a Delphi survey

BackgroundRosacea is a chronic inflammatory disorder affecting millions of individuals worldwide. Diagnosis is based on signs and symptoms with management and treatment aimed to suppress inflammatory lesions, erythema, and telangiectasia. While many clinical trials of rosacea exist, the lack of consensus in outcome reporting across all trials poses a concern. Proper evaluation and comparison of treatment modalities is challenging. In order to address the inconsistencies present, this project aims to determine a core set of outcomes which should be evaluated in all clinical trials of rosacea.Methods/designThis project will utilize a methodology similar to previous core outcome set research. A long list of outcomes will be extracted over four phases: (1) systematic literature review, (2) patient interviews, (3) other published sources, and (4) stakeholder involvement. Potential outcomes will be examined by the Steering Committee to provide further insight. The Delphi process will then be performed to prioritize and condense the list of outcomes generated. Two homogenous groups of physicians and patients will participate in two consecutive rounds of Delphi surveys. A consensus meeting, composed of physicians, patients, and stakeholders, will be conducted after the Delphi exercise to further select outcomes, taking into account participant scores. By the end of the meeting, members will vote and decide on a final recommended set of core outcomes. For the duration of the study, we will be in collaboration with both the Core Outcome Measures in Effectiveness Trials (COMET) and Cochrane Skin Group - Core Outcome Set Initiative (CSG-COUSIN).DiscussionThis study aims to develop a core outcome set to guide assessment in clinical trials of rosacea. The end-goal is to improve the reliability and consistency of outcome reporting, thereby allowing sufficient evaluation of treatment effectiveness and patient satisfaction.

Electronic supplementary material

The online version of this article (doi:10.1186/s13063-016-1554-3) contains supplementary material, which is available to authorized users.     

The CROWN Initiative: journal editors invite researchers to develop core outcomes in women’s health

Clinical trials, systematic reviews and guidelines compare beneficial and non-beneficial outcomes following interventions. Often, however, various studies on a particular topic do not address the same outcomes, making it difficult to draw clinically useful conclusions when a group of studies is looked at as a whole. This problem was recently thrown into sharp focus by a systematic review of interventions for preterm birth prevention, which found that among 103 randomised trials, no fewer than 72 different outcomes were reported. There is a growing recognition among clinical researchers that this variability undermines consistent synthesis of the evidence, and that what is needed is an agreed standardised collection of outcomes - a "core outcomes set" - for all trials in a specific clinical area. Recognising that the current inconsistency is a serious hindrance to progress in our specialty, the editors of over 50 journals related to women's health have come together to support The CROWN (CoRe Outcomes in WomeN's health) Initiative.     

New trends in assessing the outcomes of mental health interventions

Assessing the outcomes of interventions in mental health care is both important and challenging. The aim of this paper is to advance the field of outcomes research by proposing a taxonomy of the decisions that clinicians and researchers need to consider when evaluating outcomes. Our taxonomy has eight components, framed as decisions: Whose outcome will be considered? Which scientific stage is being investigated? What outcome domain(s) matter? What level of assessment will be used? Will clinical and/or recovery outcomes be assessed? Whose perspective will be considered? Will deficits and/or strengths be the focus? Will invariant or individualized measures be preferred? We propose a future focus on understanding what matters most to people using mental health services, and on the use of measures rated by service users as the primary approach to evaluating outcome.     

Pharmacology of traumatic brain injury: where is the "golden bullet"?

Traumatic brain injury (TBI) represents a major health care problem and a significant socioeconomic challenge worldwide. In the United States alone, approximately 1.5 million patients are affected each year, and the mortality of severe TBI remains as high as 35%-40%. These statistics underline the urgent need for efficient treatment modalities to improve posttraumatic morbidity and mortality. Despite advances in basic and clinical research as well as improved neurological intensive care in recent years, no specific pharmacological therapy for TBI is available that would improve the outcome of these patients. Understanding of the cellular and molecular mechanisms underlying the pathophysiological events after TBI has resulted in the identification of new potential therapeutic targets. Nevertheless, the extrapolation from basic research data to clinical application in TBI patients has invariably failed, and results from prospective clinical trials are disappointing. We review the published prospective clinical trials on pharmacological treatment modalities for TBI patients and outline future promising therapeutic avenues in the field.     

Development of a core outcome set for studies involving patients undergoing major lower limb amputation for peripheral arterial disease: study protocol for a systematic review and identification of a core outcome set using a Delphi survey

Background

The development of a standardised reporting set is important to ensure that research is directed towards the most important outcomes and that data is comparable. To ensure validity, the set must be agreed by a consensus of stakeholders including patients, healthcare professionals and lay representatives. There is currently no agreed core outcome set for patients undergoing major lower limb amputation for peripheral arterial disease (PAD) for either short- or medium-term research outcomes. By developing these sets we aim to rationalise future trial outcomes, facilitate meta-analysis and improve the quality and applicability of amputation research.

Methods/design

We will undertake a comprehensive systematic review of studies of patients undergoing major lower limb amputation for PAD. Data regarding all primary and secondary outcomes reported in relevant studies will be extracted and summarised as outcome domains. We will then undertake focus groups with key stakeholders (patients, carers, health and social care workers) to collect qualitative data to identify the main short- and medium-term research outcomes for patients undergoing major lower limb amputation. Results of the systematic review and focus groups will be combined to create a comprehensive list of potential key outcomes. Stakeholders (patients, researchers and health and social care workers) will then be polled to determine which of the outcomes are considered to be important in a general context using a three-phase Delphi process. After preliminary analysis, results will be presented at a face-to-face meeting of key stakeholders for discussion and voting on the final set of core outcomes. This project is being run in parallel with a feasibility trial assessing perineural catheters in patients undergoing lower limb amputation (the PLACEMENT trial). Full ethical approval has been granted for the study (Wales REC 3 reference number 16/WA/0353).

Discussion

Core outcome sets will be developed for short- and medium-term outcomes of research involving patients undergoing major lower limb amputation for PAD. This will help with the design of future trials and facilitate meta-analyses of trial data.

Trial registration

PROSPERO: CRD42017059329. Registered on 30 March 2017.COMET: 975. Registered on 5 April 2017.

     

Outcomes in Clinical Trials of Inhaled Corticosteroids for Children with Asthma Are Narrowly Focussed on Short Term Disease Activity

Background

Little work has been done to determine which outcomes should be measured in randomised controlled trials (RCTs) in children with asthma. Drug regulatory authorities require that short term disease activity is measured, but other outcome domains are not mandatory for licensing and marketing purposes. We aimed to identify whether any domains were underrepresented in RCTs of regular therapies for children with asthma over a 20 year period, and to examine what consistency there was between RCTs in the outcomes used to assess the domains.

Methodology/Principal Findings

By searching the Cochrane Central Register of Controlled Trials in January 2008, we identified all parallel-group RCTs, published between January 1988 and December 2007, which assessed inhaled corticosteroids (ICS) as regular therapy for children with asthma. We evaluated how frequently RCTs measured the following pre-defined domains: disease activity; disease damage; functional status; quality of life; health resource utilisation; and adverse effects of therapy. Our initial search identified 1668 abstracts, of which 412 were retrieved in full. 159 RCTs, of which 115 involved only children and 44 involved children and adults, were included in the review. Disease activity was measured in 157 RCTs, adverse effects of ICS in 135, functional status in 25, quality of life in 21, and health resource utilisation in 17. No RCT measured long term disease damage, although two used FEV1 as a measure of ‘lung growth’. RCTs were inconsistent in the outcomes used to measure the domains.

Conclusions

Short term disease activity is the most frequently measured outcome domain in RCTs in children with asthma. Effects of regular therapies on functional status, quality of life, and long term consequences of asthma are infrequently assessed. A core set of outcomes, developed using consensus techniques, would standardise the measurement of appropriate outcomes in these RCTs. Involving patients would identify outcomes which are most relevant from their perspective.     

Reforming the NHS reforms.

Rather than improving efficiency, the reforms imposed on the NHS have increased bureaucracy, reduced patient choice, limited the range of core services, and led to inequity of treatment. In this paper I examine how the medical profession might help to solve these problems. Priorities must be set for health care since no government can afford all the possibilities offered by medical science. It is essential to forge a consensus of patients, carers, professionals, the public, and government if a system of priorities is to be equitable and just. We also need to be able to measure quality of outcome in health care. This requires consensus on what is the desired outcome and the development of appropriate guidelines, audit, and performance review. This is primarily a task for the health professions supported by management and by adequate investment. Basically, the government must reinstate the three traditional values of the NHS--equity, consensus, and regard for representative professional advice.     

Targets and outcomes of psychotherapies for mental disorders: an overview

It is not yet clear what mental disorders are and what are the causal pathways that lead to them. That makes it difficult to decide what the targets and outcomes of psychotherapies should be. In this paper, the main types of targets and outcomes of psychotherapies are described, and a brief overview is provided of some of the main results of research on these types. These include symptom reduction, personal targets and outcomes from the patient's perspective, improvement of quality of life, intermediate outcomes depending on the theoretical framework of the therapist, negative outcomes to be avoided, and economic outcomes. In line with the dominance of the DSM and ICD systems for diagnoses, most research has been focused on symptom reduction. This considerable body of research, with hundreds of randomized trials, has shown that for most mental disorders effective psychotherapies are available. There is also research showing that psychotherapies can result in improvement of quality of life in most mental disorders. However, relatively little research is available on patient‐defined outcomes, intermediate outcomes, negative outcomes and economic outcomes. Patients, relatives, therapists, employers, health care providers and society at large each have their own perspectives on targets and outcomes of psychotherapies. The perspective of patients should have more priority in research, and a standardization of outcome measures across trials is much needed.     

What Are the Health Benefits of Active Travel? A Systematic Review of Trials and Cohort Studies

Background

Increasing active travel (primarily walking and cycling) has been widely advocated for reducing obesity levels and achieving other population health benefits. However, the strength of evidence underpinning this strategy is unclear. This study aimed to assess the evidence that active travel has significant health benefits.

Methods

The study design was a systematic review of (i) non-randomised and randomised controlled trials, and (ii) prospective observational studies examining either (a) the effects of interventions to promote active travel or (b) the association between active travel and health outcomes. Reports of studies were identified by searching 11 electronic databases, websites, reference lists and papers identified by experts in the field. Prospective observational and intervention studies measuring any health outcome of active travel in the general population were included. Studies of patient groups were excluded.

Results

Twenty-four studies from 12 countries were included, of which six were studies conducted with children. Five studies evaluated active travel interventions. Nineteen were prospective cohort studies which did not evaluate the impact of a specific intervention. No studies were identified with obesity as an outcome in adults; one of five prospective cohort studies in children found an association between obesity and active travel. Small positive effects on other health outcomes were found in five intervention studies, but these were all at risk of selection bias. Modest benefits for other health outcomes were identified in five prospective studies. There is suggestive evidence that active travel may have a positive effect on diabetes prevention, which may be an important area for future research.

Conclusions

Active travel may have positive effects on health outcomes, but there is little robust evidence to date of the effectiveness of active transport interventions for reducing obesity. Future evaluations of such interventions should include an assessment of their impacts on obesity and other health outcomes.     

Osteopathic manipulative treatment for pneumonia

The pneumonias due to infection continue to be a meaningful threat to the health and viability of persons, particularly those in high risk groups: children, the aged and the debilitated. Noll and colleagues provide us with the results of a well-designed and well-executed multi-institutional controlled clinical trial to evaluate the efficacy of osteopathic manipulative treatment (OMT) in the treatment of pneumonia. The data obtained indicate that by intention-to-treat analysis, the addition of OMT to conventional care did not improve the designated outcomes when compared to conventional care only. A disappointing but important finding. However, by per-protocol analysis, the addition of OMT or of light touch decreased length of hospital stay, the duration of intravenous antibiotics and the incidence of respiratory failure and death relative to conventional care only. Further study is called for to explain these surprising results. Meeting the need for randomized clinical trials of the role and efficacy of OMT is a responsibility of high priority for the osteopathic profession in this age of evidence-based medicine. The American Osteopathic Association (AOA) needs to consider reinstating a dues-generated financial set-aside both to increase its support of osteopathic research and to initiate a program of physician-investigator career development awards to recruit and help establish osteopathic clinical investigators in a career in translational and clinical research.     

Patient-centered outcomes of high-velocity, low-amplitude spinal manipulation for low back pain: A systematic review

Low back pain (LBP) is a well-recognized public health problem with no clear gold standard medical approach to treatment. Thus, those with LBP frequently turn to treatments such as spinal manipulation (SM). Many clinical trials have been conducted to evaluate the efficacy or effectiveness of SM for LBP. The primary objective of this paper was to describe the current literature on patient-centered outcomes following a specific type of commonly used SM, high-velocity low-amplitude (HVLA), in patients with LBP. A systematic search strategy was used to capture all LBP clinical trials of HVLA using our predefined patient-centered outcomes: visual analogue scale, numerical pain rating scale, Roland-Morris Disability Questionnaire, and the Oswestry Low Back Pain Disability Index. Of the 1294 articles identified by our search, 38 met our eligibility criteria. Like previous SM for LBP systematic reviews, this review shows a small but consistent treatment effect at least as large as that seen in other conservative methods of care. The heterogeneity and inconsistency in reporting within the studies reviewed makes it difficult to draw definitive conclusions. Future SM studies for LBP would benefit if some of these issues were addressed by the scientific community before further research in this area is conducted.     

Including Ethnic Minorities in Mental Health Intervention Research: A Practical Approach to a Long-Standing Problem

This paper examines a controversy that arose while developing a supplement to Mental Health: A Report of the Surgeon General that was focused on ethnic minority mental health. The controversy involved whether and how to make recommendations about ethnic minorities seeking mental health care. We found that few studies provided information on outcomes of mental health care for ethnic minorities. In this paper, we discuss outcomes of mental health care for ethnic minorities and how to proceed in developing an evidence base for understanding mental health care and minorities. We conclude that entering representative (based on population) numbers of ethnic minorities in efficacy trials is unlikely to produce useful information on outcomes of care because the numbers will be too small to produce reliable findings. We also conclude that while conducting randomized efficacy trials for all mental health interventions for each ethnic group would be impractical, innovative and theoretically informed studies that focus on specific cultural groups are needed to advance the knowledge base. We call for theory-driven research focused on mental health disparities that has the potential for understanding disparities and improving outcomes for ethnic minority populations.     

Quality of life measures in health care. I: Applications and issues in assessment.

Many clinicians remain unsure of the relevance of measuring quality of life to their clinical practice. In health economics quality of life measures have become the standard means of assessing the results of health care interventions and, more controversially, the means of prioritising funding; but they have many other applications. This article--the first of three on measuring quality of life--reviews the instruments available and their application in screening programmes, audit, health care research, and clinical trials. Using the appropriate instrument is essential if outcome measures are to be valid and clinically meaningful.     

Paradigms of expected failure

This study explores the outsourcing and offshoring of clinical trials and how they interconnect with the dynamics of drug development and regulation in the United States. I focus on the activities of United States-based contract research organizations, which make up a specialized global clinical trials industry focusing on the recruitment of human subjects and investigators. Tracking this industry’s activities in eastern Europe and Latin America, two clinical trial market ‘growth regions,’ I address the strategies of evidence-making that inform clinical trial offshoring. I also show how aspects of the clinical trial model—in which failures to predict safety outcomes or a paradigm of expected failure—are being exported along with the offshored trial. The clinical trials industry is a crucial, highly mobile, and profitable arm of the global pharmaceutical industry. Where state agencies furnish limited or no health care, drug developers claim that trial expansion and experiments have become social goods in themselves. But questions remain: How is drug value and research integrity maintained? And how do the results of clinical trials strengthen or undermine the delivery of affordable and effective interventions? As this essay shows, clinical trials are not only hypothesis-testing instruments; they are operative environments redistributing resources and occasioning tense medical and social fields. In highlighting the inefficiencies and uncertainties of global drug development, this study points to problems in the operational model of drug development and in systems of human protection. It also considers new forms of accountability at the nexus of private sector science and public health.