Internal and external validity of cluster randomised trials: systematic review of recent trials

Objectives To assess aspects of the internal validity of recently published cluster randomised trials and explore the reporting of information useful in assessing the external validity of these trials.Design Review of 34 cluster randomised trials in primary care published in 2004 and 2005 in seven journals (British Medical Journal, British Journal of General Practice, Family Practice, Preventive Medicine, Annals of Internal Medicine, Journal of General Internal Medicine, Pediatrics).Data sources National Library of Medicine (Medline) via PubMed.Data extraction To assess aspects of internal validity we extracted data on appropriateness of sample size calculations and analyses, methods of identifying and recruiting individual participants, and blinding. To explore reporting of information useful in assessing external validity we extracted data on cluster eligibility, cluster inclusion and retention, cluster generalisability, and the feasibility and acceptability of the intervention to health providers in clusters.Results 21 (62%) trials accounted for clustering in sample size calculations and 30 (88%) in the analysis; about a quarter were potentially biased because of procedures surrounding recruitment and identification of patients; individual participants were blind to allocation status in 19 (56%) and outcome assessors were blind in 15 (44%). In almost half the reports, information relating to generalisability of clusters was poorly reported, and in two fifths there was no information about the feasibility and acceptability of the intervention.Conclusions Cluster randomised trials are essential for evaluating certain types of interventions. Issues affecting their internal validity, such as appropriate sample size calculations and analysis, have been widely disseminated and are now better addressed by researchers. Blinding of those identifying and recruiting patients to allocation status is recommended but is not always carried out. There may be fewer barriers to internal validity in trials in which individual participants are not recruited. External validity seems poorly addressed in many trials, yet is arguably as important as internal validity in judging quality as a basis for healthcare intervention.     

Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta-epidemiological study

Objective To examine whether the association of inadequate or unclear allocation concealment and lack of blinding with biased estimates of intervention effects varies with the nature of the intervention or outcome.Design Combined analysis of data from three meta-epidemiological studies based on collections of meta-analyses.Data sources 146 meta-analyses including 1346 trials examining a wide range of interventions and outcomes.Main outcome measures Ratios of odds ratios quantifying the degree of bias associated with inadequate or unclear allocation concealment, and lack of blinding, for trials with different types of intervention and outcome. A ratio of odds ratios <1 implies that inadequately concealed or non-blinded trials exaggerate intervention effect estimates.Results In trials with subjective outcomes effect estimates were exaggerated when there was inadequate or unclear allocation concealment (ratio of odds ratios 0.69 (95% CI 0.59 to 0.82)) or lack of blinding (0.75 (0.61 to 0.93)). In contrast, there was little evidence of bias in trials with objective outcomes: ratios of odds ratios 0.91 (0.80 to 1.03) for inadequate or unclear allocation concealment and 1.01 (0.92 to 1.10) for lack of blinding. There was little evidence for a difference between trials of drug and non-drug interventions. Except for trials with all cause mortality as the outcome, the magnitude of bias varied between meta-analyses.Conclusions The average bias associated with defects in the conduct of randomised trials varies with the type of outcome. Systematic reviewers should routinely assess the risk of bias in the results of trials, and should report meta-analyses restricted to trials at low risk of bias either as the primary analysis or in conjunction with less restrictive analyses.     

Evidence for risk of bias in cluster randomised trials: review of recent trials published in three general medical journals

Objective To examine the prevalence of a risk of bias associated with the design and conduct of cluster randomised controlled trials among a sample of recently published studies.Design Retrospective review of cluster randomised trials published in the BMJ, Lancet, and New England Journal of Medicine from January 1997 to October 2002.Main outcome measures Prevalence of secure randomisation of clusters, identification of participants before randomisation (to avoid foreknowledge of allocation), differential recruitment between treatment arms, differential application of inclusion and exclusion criteria, and differential attrition.Results Of the 36 trials identified, 24 were published in the BMJ,11 in the Lancet, and a single trial in the New England Journal of Medicine. At the cluster level, 15 (42%) trials provided evidence for secure allocation and 25 (69%) used stratified allocation. Few trials showed evidence of imbalance at the cluster level. However, some evidence of susceptibility to risk of bias at the individual level existed in 14 (39%) studies.Conclusions Some recently published cluster randomised trials may not have taken adequate precautions to guard against threats to the internal validity of their design.     

What is meant by intention to treat analysis? Survey of published randomised controlled trials

ObjectivesTo assess the methodological quality of intention to treat analysis as reported in randomised controlled trials in four large medical journals.DesignSurvey of all reports of randomised controlled trials published in 1997 in the BMJ, Lancet, JAMA, and New England Journal of Medicine.Results119 (48%) of the reports mentioned intention to treat analysis. Of these, 12 excluded any patients who did not start the allocated intervention and three did not analyse all randomised subjects as allocated. Five reports explicitly stated that there were no deviations from random allocation. The remaining 99 reports seemed to analyse according to random allocation, but only 34 of these explicitly stated this. 89 (75%) trials had some missing data on the primary outcome variable. The methods used to deal with this were generally inadequate, potentially leading to a biased treatment effect. 29 (24%) trials had more than 10% of responses missing for the primary outcome, the methods of handling the missing responses were similar in this subset.ConclusionsThe intention to treat approach is often inadequately described and inadequately applied. Authors should explicitly describe the handling of deviations from randomised allocation and missing responses and discuss the potential effect of any missing response. Readers should critically assess the validity of reported intention to treat analyses.

Key messages

• Intention to treat gives a pragmatic estimate of the benefit of a change in treatment policy rather than of potential benefit in patients who receive treatment exactly as planned
• Full application of intention to treat is possible only when complete outcome data are available for all randomised subjects
• About half of all published reports of randomised controlled trials stated that intention to treat was used, but handling of deviations from randomised allocation varied widely
• Many trials had some missing data on the primary outcome variable, and methods used to deal with this were generally inadequate, potentially leading to bias
• Intention to treat analyses are often inadequately described and inadequately applied

     

Multifactorial assessment and targeted intervention for preventing falls and injuries among older people in community and emergency care settings: systematic review and meta-analysis

Objective To evaluate the effectiveness of multifactorial assessment and intervention programmes to prevent falls and injuries among older adults recruited to trials in primary care, community, or emergency care settings.Design Systematic review of randomised and quasi-randomised controlled trials, and meta-analysis.Data sources Six electronic databases (Medline, Embase, CENTRAL, CINAHL, PsycINFO, Social Science Citation Index) to 22 March 2007, reference lists of included studies, and previous reviews.Review methods Eligible studies were randomised or quasi-randomised trials that evaluated interventions to prevent falls that were based in emergency departments, primary care, or the community that assessed multiple risk factors for falling and provided or arranged for treatments to address these risk factors.Data extraction Outcomes were number of fallers, fall related injuries, fall rate, death, admission to hospital, contacts with health services, move to institutional care, physical activity, and quality of life. Methodological quality assessment included allocation concealment, blinding, losses and exclusions, intention to treat analysis, and reliability of outcome measurement. Results 19 studies, of variable methodological quality, were included. The combined risk ratio for the number of fallers during follow-up among 18 trials was 0.91 (95% confidence interval 0.82 to 1.02) and for fall related injuries (eight trials) was 0.90 (0.68 to 1.20). No differences were found in admissions to hospital, emergency department attendance, death, or move to institutional care. Subgroup analyses found no evidence of different effects between interventions in different locations, populations selected for high risk of falls or unselected, and multidisciplinary teams including a doctor, but interventions that actively provide treatments may be more effective than those that provide only knowledge and referral.Conclusions Evidence that multifactorial fall prevention programmes in primary care, community, or emergency care settings are effective in reducing the number of fallers or fall related injuries is limited. Data were insufficient to assess fall and injury rates.     

The methodological quality of 176,620 randomized controlled trials published between 1966 and 2018 reveals a positive trend but also an urgent need for improvement

Many randomized controlled trials (RCTs) are biased and difficult to reproduce due to methodological flaws and poor reporting. There is increasing attention for responsible research practices and implementation of reporting guidelines, but whether these efforts have improved the methodological quality of RCTs (e.g., lower risk of bias) is unknown. We, therefore, mapped risk-of-bias trends over time in RCT publications in relation to journal and author characteristics. Meta-information of 176,620 RCTs published between 1966 and 2018 was extracted. The risk-of-bias probability (random sequence generation, allocation concealment, blinding of patients/personnel, and blinding of outcome assessment) was assessed using a risk-of-bias machine learning tool. This tool was simultaneously validated using 63,327 human risk-of-bias assessments obtained from 17,394 RCTs evaluated in the Cochrane Database of Systematic Reviews (CDSR). Moreover, RCT registration and CONSORT Statement reporting were assessed using automated searches. Publication characteristics included the number of authors, journal impact factor (JIF), and medical discipline. The annual number of published RCTs substantially increased over 4 decades, accompanied by increases in authors (5.2 to 7.8) and institutions (2.9 to 4.8). The risk of bias remained present in most RCTs but decreased over time for allocation concealment (63% to 51%), random sequence generation (57% to 36%), and blinding of outcome assessment (58% to 52%). Trial registration (37% to 47%) and the use of the CONSORT Statement (1% to 20%) also rapidly increased. In journals with a higher impact factor (>10), the risk of bias was consistently lower with higher levels of RCT registration and the use of the CONSORT Statement. Automated risk-of-bias predictions had accuracies above 70% for allocation concealment (70.7%), random sequence generation (72.1%), and blinding of patients/personnel (79.8%), but not for blinding of outcome assessment (62.7%). In conclusion, the likelihood of bias in RCTs has generally decreased over the last decades. This optimistic trend may be driven by increased knowledge augmented by mandatory trial registration and more stringent reporting guidelines and journal requirements. Nevertheless, relatively high probabilities of bias remain, particularly in journals with lower impact factors. This emphasizes that further improvement of RCT registration, conduct, and reporting is still urgently needed.

Many randomized controlled trials (RCTs) are biased and difficult to reproduce due to methodological flaws and poor reporting. Analysis of 176,620 RCTs published between 1966 and 2018 reveals that the risk of bias in RCTs generally decreased. Nevertheless, relatively high probabilities of bias remain, showing that further improvement of RCT registration, conduct, and reporting is still urgently needed.     

Developing a register of randomised controlled trials in primary care.

OBJECTIVE--To determine the number, nature, site of publication, and feasibility of identifying randomised controlled trials relevant to primary care. DESIGN--Review of literature using three strategies: approaching journal editors, Medline search, and manual search of individual journals. SETTING--Journals containing publications of studies based in primary care. MAIN OUTCOME MEASURES--The number, site of publication, and subject of trials identified. RESULTS--No journal had a system which enabled identification of all the randomised controlled trials it published. 266 trials relevant to primary care were identified from 110 different journals during 1987-91 by Medline. Of these, only 62 trials were published in primary care journals. Hand searching of seven major primary care research journals showed that between 13% and 38% of the trials had been missed by the Medline search. Of the trials identified, 47 (18%) were concerned with mental disease (including neuroses, tobacco misuse and alcohol misuse) and 43 (16%) were concerned with hypertension. CONCLUSION--Given the diversity of publication sources and topics, this supports the need for a centrally based register of randomised controlled trials that may be relevant to primary care overviews in the future.     

Corticosteroids in acute traumatic brain injury: systematic review of randomised controlled trials.

OBJECTIVE: To quantify the effectiveness and safety of corticosteroids in the treatment of acute traumatic brain injury. DESIGN: Systematic review of randomised controlled trials of corticosteroids in acute traumatic brain injury. Summary odds ratios were estimated as an inverse variance weighted average of the odds ratios for each study. SETTING: Randomised trials available by March 1996. SUBJECTS: The included trials with outcome data comprised 2073 randomised participants. RESULTS: The effect of corticosteroids on the risk of death was reported in 13 included trials. The pooled odds ratio for the 13 trials was 0.91 (95% confidence interval 0.74 to 1.12). Pooled absolute risk reduction was 1.8% (-2.5% to 5.7%). For the 10 trials that reported death or disability the pooled odds ratio was 0.90 (0.72 to 1.11). For infections of any type the pooled odds ratio was 0.92 (0.69 to 1.23) and for the seven trials reporting gastrointestinal bleeding it was 1.05 (0.44 to 2.52). With only those trials with the best quality of concealment of allocation, the pooled odds ratio estimates for death and death or disability became closer to unity. CONCLUSIONS: This systematic review of randomised controlled trials of corticosteroids in acute traumatic brain injury shows that there remains considerable uncertainty over their effects. Neither moderate benefits nor moderate harmful effects can be excluded. The widely practicable nature of the drugs and the importance of the health problem suggest that large simple trials are feasible and worth while to establish whether there are any benefits from use of corticosteroids in this setting.     

Reporting on quality of life in randomised controlled trials: bibliographic study

ObjectivesTo examine the frequency and quality of reporting on quality of life in randomised controlled trials.DesignSearch of the Cochrane Controlled Trials Register 1980 to 1997 to identify trials from all disciplines, from oncology, and from cardiovascular medicine that reported on quality of life. Assessment of abstracts from articles published from 1993 to 1996. Assessment of a sample of full reports with a standardised instrument.ResultsDuring 1980-97 reporting on quality of life increased from 0.63% to 4.2% for trials from all disciplines, from 1.5% to 8.2% for cancer trials, and from 0.34% to 3.6% for cardiovascular trials. Of 364 abstracts, 65% reported on drug interventions. Of a sample of 67 full reports, authors of 48 (72%) used 62 established quality of life instruments. In 15 reports (22%) authors developed their own measures, and in 2 (3%) methods were unclear. Response rates were given in 38 (57%), and complete reporting on all items and scales occurred in 31 (46%).ConclusionsLess than 5% of all randomised controlled trials reported on quality of life, and this proportion was below 10% even for cancer trials. A plethora of instruments was used in different studies, and the reporting of methods and results was often inadequate. Standards for the measurement and reporting of quality of life in clinical trials research need to be developed.

Key messages

• We examined the reporting on quality of life in randomised controlled trials listed in the Cochrane Controlled Trials Register
• Although reporting on quality of life increased over time, fewer than 5% of trials overall and fewer than 10% of cancer trials included quality of life in 1997
• Among 67 articles selected at random for detailed examination, a wide range of established and self developed measures of quality of life were used
• Only about half of trials gave response rates, and less than half reported on all items and scales used
• Standards for assessing and reporting quality of life in clinical research trials need to be developed

     

Parachute use to prevent death and major trauma related to gravitational challenge: systematic review of randomised controlled trials

Objectives To determine whether parachutes are effective in preventing major trauma related to gravitational challenge.Design Systematic review of randomised controlled trials.Data sources: Medline, Web of Science, Embase, and the Cochrane Library databases; appropriate internet sites and citation lists.Study selection: Studies showing the effects of using a parachute during free fall.Main outcome measure Death or major trauma, defined as an injury severity score > 15.Results We were unable to identify any randomised controlled trials of parachute intervention.Conclusions As with many interventions intended to prevent ill health, the effectiveness of parachutes has not been subjected to rigorous evaluation by using randomised controlled trials. Advocates of evidence based medicine have criticised the adoption of interventions evaluated by using only observational data. We think that everyone might benefit if the most radical protagonists of evidence based medicine organised and participated in a double blind, randomised, placebo controlled, crossover trial of the parachute.     

The unpredictability paradox: review of empirical comparisons of randomised and non-randomised clinical trials

Objective To summarise comparisons of randomised clinical trials and non-randomised clinical trials, trials with adequately concealed random allocation versus inadequately concealed random allocation, and high quality trials versus low quality trials where the effect of randomisation could not be separated from the effects of other methodological manoeuvres.Design Systematic review.Selection criteria Cohorts or meta-analyses of clinical trials that included an empirical assessment of the relation between randomisation and estimates of effect.Data sources Cochrane Review Methodology Database, Medline, SciSearch, bibliographies, hand searching of journals, personal communication with methodologists, and the reference lists of relevant articles.Main outcome measures Relation between randomisation and estimates of effect.Results Eleven studies that compared randomised controlled trials with non-randomised controlled trials (eight for evaluations of the same intervention and three across different interventions), two studies that compared trials with adequately concealed random allocation and inadequately concealed random allocation, and five studies that assessed the relation between quality scores and estimates of treatment effects, were identified. Failure to use random allocation and concealment of allocation were associated with relative increases in estimates of effects of 150% or more, relative decreases of up to 90%, inversion of the estimated effect and, in some cases, no difference. On average, failure to use randomisation or adequate concealment of allocation resulted in larger estimates of effect due to a poorer prognosis in non-randomly selected control groups compared with randomly selected control groups.Conclusions Failure to use adequately concealed random allocation can distort the apparent effects of care in either direction, causing the effects to seem either larger or smaller than they really are. The size of these distortions can be as large as or larger than the size of the effects that are to be detected.

Key messages

• Empirical studies support using random allocation in clinical trials and ensuring that the allocation process is concealed—that is, that assignment is impervious to any influence by the people making the allocation
• The effect of not using concealed random allocation can be as large or larger than the effects of worthwhile interventions
• On average, failure to use concealed random allocation results in overestimates of effect due to a poorer prognosis in non-randomly selected control groups compared with randomly selected control groups, but it can result in underestimates of effect, reverse the direction of effect, mask an effect, or give similar estimates of effect
• The adequacy of allocation concealment may be a more sensitive measure of bias in clinical trials than scales used to assess the quality of clinical trials
• It is a paradox that the unpredictability of randomisation is the best protection against the unpredictability of the extent and direction of bias in clinical trials that are not properly randomised

     

Meta-analysis of efficacy of quinine for treatment of nocturnal leg cramps in elderly people.

OBJECTIVE--To assess quantitatively the efficacy of quinine (as quinine sulphate) compared with placebo in the treatment of nocturnal leg cramps. DESIGN--A meta-analysis of six randomised, double blind, crossover trials. SETTING--Randomised trials that were available as of April 1994. SUBJECTS--A total of 107 general ambulatory patients who suffered from regular nocturnal leg cramps from six clinical trials. RESULTS--Data from individual patients were used to calculate point estimates and 95% confidence intervals for each of the outcome measures reported by these studies. Treatment with quinine resulted in a significant reduction in the number of cramps for a four week period compared with placebo (8.83 fewer cramps; 95% confidence interval 4.16 to 13.49). Treatment with quinine reduced the number of nights with cramps by 27.4% (24.0% to 30.8%) compared with placebo. Treatment did not produce a significant change in the severity or duration of individual nocturnal leg cramps. Side effects were uncommon. CONCLUSIONS--The results indicate that quinine can prevent nocturnal leg cramps in general ambulatory populations. Given the possible serious side effects of treatment with quinine, the benefits and risks in patients taking this drug should be closely monitored.     

African HIV/AIDS trials are more likely to report adequate allocation concealment and random generation than North American trials

Background

Adherence to good methodological quality is necessary to minimise bias in randomised conrolled trials (RCTs). Specific trial characteristics are associated with better trial quality, but no studies to date are specific to HIV/AIDS or African trials. We postulated that location may negatively impact on trial quality in regions where resources are scarce.

Methods

1) To compare the methodological quality of all HIV/AIDS RCTs conducted in Africa with a random sample of similar trials conducted in North America; 2) To assess whether location is predictive of trial quality. We searched MEDLINE, EMBASE, CENTRAL and LILACS. Eligible trials were 1) randomized, 2) evaluations of preventive or treatment interventions for HIV/AIDS, 3) reported before 2004, and 4) conducted wholly or partly (if multi-centred) in Africa or North America. We assessed adequacy of random generation, allocation concealment and masking of assessors. Using univariate and multivariate logistic regression analyses we evaluated the association between location (Africa versus North America) and these domains.

Findings

The African search yielded 12,815 records, from which 80 trials were identified. The North American search yielded 13,158 records from which 785 trials were identified and a random sample of 114 selected for analysis. African trials were three times more likely than North American trials to report adequate allocation concealment (OR = 3.24; 95%CI: 1.59 to 6.59; p<0.01) and twice as likely to report adequate generation of the sequence (OR = 2.36; 95%CI: 1.20 to 4.67; p = 0.01), after adjusting for other confounding factors. Additional significant factors positively associated with quality were an a priori sample size power calculation, restricted randomization and inclusion of a flow diagram detailing attrition. We did not detect an association between location and outcome assessor masking.

Conclusions

The higher quality of reporting of methodology in African trials is noteworthy. Most African trials are externally funded, and it is possible that stricter agency requirements when leading trials in other countries and greater experience and training of principal investigators of an international stature, may account for this difference.     

Randomized controlled trials of HIV/AIDS prevention and treatment in Africa: results from the Cochrane HIV/AIDS Specialized Register

Introduction

To effectively address HIV/AIDS in Africa, evidence on preventing new infections and providing effective treatment is needed. Ideally, decisions on which interventions are effective should be based on evidence from randomized controlled trials (RCTs). Our previous research described African RCTs of HIV/AIDS reported between 1987 and 2003. This study updates that analysis with RCTs published between 2004 and 2008.

Objectives

To describe RCTs of HIV/AIDS conducted in Africa and reported between 2004 and 2008.

Methods

We searched the Cochrane HIV/AIDS Specialized Register in September 2009. Two researchers independently evaluated studies for inclusion and extracted data using standardized forms. Details included location of trials, interventions, methodological quality, location of principal investigators and funders.

Results

Our search identified 834 RCTs, with 68 conducted in Africa. Forty-three assessed prevention-interventions and 25 treatment-interventions. Fifteen of the 43 prevention RCTs focused on preventing mother-to-child HIV transmission. Thirteen of the 25 treatment trials focused on opportunistic infections. Trials were conducted in 16 countries with most in South Africa (20), Zambia (12) and Zimbabwe (9). The median sample size was 628 (range 33-9645). Methods used for the generation of the allocation sequence and allocation concealment were adequate in 38 and 32 trials, respectively, and 58 reports included a CONSORT recommended flow diagram. Twenty-nine principal investigators resided in the United States of America (USA) and 18 were from African countries. Trials were co-funded by different agencies with most of the funding obtained from USA governmental and non-governmental agencies. Nineteen pharmaceutical companies provided partial funding to 15 RCTs and African agencies co-funded 17 RCTs. Ethical approval was reported in 65 trials and informed consent in 61 trials.

Conclusion

Prevention trials dominate the trial landscape in Africa. Of note, few principal investigators and funders are from Africa. These findings mirror our previous work and continue to indicate a need for strengthening trial research capacity in Africa.     

Nonadherence to treatment protocol in published randomised controlled trials: a review

ABSTRACT: This review aimed to ascertain the extent to which nonadherence to treatment protocol is reported and addressed in a cohort of published analyses of randomised controlled trials (RCTs). One hundred publications of RCTs, randomly selected from those published in BMJ, New England Journal of Medicine, the Journal of the American Medical Association and The Lancet during 2008, were reviewed to determine the extent and nature of reported nonadherence to treatment protocol, and whether statistical methods were used to examine the effect of such nonadherence on both benefit and harms analyses. We also assessed the quality of trial reporting of treatment protocol nonadherence and the quality of reporting of the statistical analysis methods used to investigate such nonadherence. Nonadherence to treatment protocol was reported in 98 of the 100 trials, but reporting on such nonadherence was often vague or incomplete. Forty-two publications did not state how many participants started their randomised treatment. Reporting of treatment initiation and completeness was judged to be inadequate in 64% of trials with short-term interventions and 89% of trials with long-term interventions. More than half (51) of the 98 trials with treatment protocol nonadherence implemented some statistical method to address this issue, most commonly based on per protocol analysis (46) but often labelled as intention to treat (ITT) or modified ITT (23 analyses in 22 trials). The composition of analysis sets for their benefit outcomes were not explained in 57% of trials, and 62% of trials that presented harms analyses did not define harms analysis populations. The majority of defined harms analysis populations (18 out of 26 trials, 69%) were based on actual treatment received, while the majority of trials with undefined harms analysis populations (31 out of 43 trials, 72%) appeared to analyse harms using the ITT approach. Adherence to randomised intervention is poorly considered in the reporting and analysis of published RCTs. The majority of trials are subject to various forms of nonadherence to treatment protocol, and though trialists deal with this nonadherence using a variety of statistical methods and analysis populations, they rarely consider the potential for bias introduced. There is a need for increased awareness of more appropriate causal methods to adjust for departures from treatment protocol, as well as guidance on the appropriate analysis population to use for harms outcomes in the presence of such nonadherence.     

Stratification,Blinding and Placebo Effect in a Randomized,Double Blind Placebo-controlled Clinical Trial of Gold Bead Implantation in Dogs with Hip Dysplasia

The purpose of this study was to investigate the need for and choice of stratification factors, and the effects of blinding and placebo in a clinical experiment. Eighty dogs with canine hip dysplasia (CHD) were included in a randomized, placebo-controlled and double blind clinical trial with stratified parallel group design, in which body weight and degree of CHD were used as stratification factors. Thirty-eight dogs were allocated to gold bead implantation and 42 to placebo. After six months, 33 of the 42 placebo-treated dogs received gold bead implantation in an open study lasting a further 18 months. The main outcome variable in the study was change in pain signs of CHD as assessed by the owner. No significant difference in the main outcome variable, regardless of the treatment given, could be detected in the two chosen stratification factors. The only factor to influence the main outcome variable significantly was age. The blinding procedure used in the study, in which 60% of the owners correctly guessed the treatment given, was found sufficient. Of those who guessed the treatment erroneously, 88% believed the treatment given was gold bead implantation. The treatment efficacy after six months in the blinded treatment group was found to be significantly larger compared to the efficacy obtained in the open study. A significant placebo effect was therefore detected. Conclusion and Clinical Relevance: The age of the dogs influenced the outcome of the CHD treatment, and is recommended as a stratification factor. A significant placebo effect has to be expected and an optimal blinding procedure is necessary in similar clinical studies.