Guidelines for Guidelines: Are They Up to the Task? A Comparative Assessment of Clinical Practice Guideline Development Handbooks

Objectives

We conducted a comparative review of clinical practice guideline development handbooks. We aimed to identify the main guideline development tasks, assign weights to the importance of each task using expert opinions and identify the handbooks that provided a comprehensive coverage of the tasks.

Methods

We systematically searched and included handbooks published (in English language) by national, international or professional bodies responsible for evidenced-based guideline development. We reviewed the handbooks to identify the main guideline development tasks and scored each handbook for each task from 0 (the handbook did not mention the task) to 2 (the task suitably addressed and explained), and calculated a weighted score for each handbook. The tasks included in over 75% of the handbooks were considered as ‘necessary’ tasks.

Result

Nineteen guideline development handbooks and twenty seven main tasks were identified. The guideline handbooks’ weighted scores ranged from 100 to 220. Four handbooks scored over 80% of the maximum possible score, developed by the National Institute for Health and Clinical Excellence, Swiss Centre for International Health, Scottish Intercollegiate Guidelines Network and World Health Organization. Necessary tasks were: selecting the guideline topic, determining the guideline scope, identifying relevant existing guidelines, involving the consumers, forming guideline development group,, developing clinical questions, systematic search for evidence, selecting relevant evidence, appraising identifies research evidence, making group decision, grading available evidence, creating recommendations, final stakeholder consultation, guideline implementation strategies, updating recommendations and correcting potential errors.

Discussion

Adequate details for evidence based development of guidelines were still lacking from many handbooks. The tasks relevant to ethical issues and piloting were missing in most handbooks. The findings help decision makers in identifying the necessary tasks for guideline development, provide an updated comparative list of guideline development handbooks, and provide a checklist to assess the comprehensiveness of guideline development processes.     

Developing clinical practice guidelines: target audiences, identifying topics for guidelines, guideline group composition and functioning and conflicts of interest

ABSTRACT: Clinical practice guidelines are one of the foundations of efforts to improve health care. In 1999, we authored a paper about methods to develop guidelines. Since it was published, the methods of guideline development have progressed both in terms of methods and necessary procedures and the context for guideline development has changed with the emergence of guideline clearing houses and large scale guideline production organisations (such as the UK National Institute for Health and Clinical Excellence). It therefore seems timely to, in a series of three articles, update and extend our earlier paper. In this first paper we discuss: the target audience(s) for guidelines and their use of guidelines; identifying topics for guidelines; guideline group composition (including consumer involvement) and the processes by which guideline groups function and the important procedural issue of managing conflicts of interest in guideline development.     

Integrating guideline development and implementation: Analysis of guideline development manual instructions for generating implementation advice

ABSTRACT: BACKGROUND: Guidelines are important tools that inform health care delivery based on best available research evidence. Guideline use is in part based on quality of the guidelines, which varies. Others hypothesized this is due to limited instructions in guideline development manuals. The purpose of this study was to examine manual instructions for implementation advice. METHODS: We used a directed and summative content analysis approach based on an established framework of guideline implementability. Six manuals identified by another research group were examined to enumerate implementability domains and elements. RESULTS: Manuals were similar in content but lacked sufficient detail in particular domains. Most frequently this was Accomodation, which includes information that would help guideline users anticipate and/or overcome organizational and system level barriers. In more than one manual information was also lacking for Communicability, information that would educate patients or facilitate their involvement in shared decision making, and Applicability, or clinical parameters to help clinicians tailor recommendations for individual patients. DISCUSSION: Most manuals that direct guideline development lack complete information about incorporating implementation advice. These findings can be used by those who developed the manuals to consider expanding their content in these domains. It can also be used by guideline developers as they plan the content and implementation of their guidelines so that the two are integrated. New approaches for guideline development and implementation may need to be developed. Use of guidelines might be improved if they included implementation advice, but this must be evaluated through ongoing research.     

Developing clinical practice guidelines: reviewing, reporting, and publishing guidelines; updating guidelines; and the emerging issues of enhancing guideline implementability and accounting for comorbid conditions in guideline development

ABSTRACT: Clinical practice guidelines are one of the foundations of efforts to improve health care. In 1999, we authored a paper about methods to develop guidelines. Since it was published, the methods of guideline development have progressed both in terms of methods and necessary procedures and the context for guideline development has changed with the emergence of guideline clearing houses and large scale guideline production organisations (such as the UK National Institute for Health and Clinical Excellence). It therefore seems timely to, in a series of three articles, update and extend our earlier paper. In this third paper we discuss the issues of: reviewing, reporting, and publishing guidelines; updating guidelines; and the two emerging issues of enhancing guideline implementability and how guideline developers should approach dealing with the issue of patients who will be the subject of guidelines having co-morbid conditions.     

Managing Conflicts of Interest in the UK National Institute for Health and Care Excellence (NICE) Clinical Guidelines Programme: Qualitative Study

Background

There is international concern that conflicts of interest (COI) may bias clinical guideline development and render it untrustworthy. Guideline COI policies exist with the aim of reducing this bias but it is not known how such policies are interpreted and used by guideline producing organisations. This study sought to determine how conflicts of interest (COIs) are disclosed and managed by a national clinical guideline developer (NICE: the UK National Institute for Health and Care Excellence).

Methods

Qualitative study using semi-structured telephone interviews with 14 key informants: 8 senior staff of NICE’s guideline development centres and 6 chairs of guideline development groups (GDGs). We conducted a thematic analysis.

Results

Participants regard the NICE COI policy as comprehensive leading to transparent and independent guidance. The application of the NICE COI policy is, however, not straightforward and clarity could be improved. Disclosure of COI relies on self reporting and guideline developers have to take “on trust” the information they receive, certain types of COI (non-financial) are difficult to categorise and manage and disclosed COI can impact on the ability to recruit clinical experts to GDGs. Participants considered it both disruptive and stressful to exclude members from GDG meetings when required by the COI policy. Nonetheless the impact of this disruption can be minimised with good group chairing skills.

Conclusions

We consider that the successful implementation of a COI policy in clinical guideline development requires clear policies and procedures, appropriate training of GDG chairs and an evaluation of how the policy is used in practice.     

The value of juvenile animal studies: a Japanese industry perspective

Pharmaceuticals have been used on adults and children; however, they were previously investigated only by adult human clinical studies and adult animal nonclinical studies. The US FDA finalized the guidance of juvenile animal toxicity studies in 2006, and EMEA was finalized in 2008. At that point, juvenile animal toxicity studies were encouraged to investigate the safety of the pediatric population. In Japan, the awareness of the development of pediatric drugs is increasing, and many scientific meetings about juvenile animal studies are being held. A Japanese guideline for juvenile animal toxicity studies has been long awaited by many Japanese pharmaceutical companies because concrete directionality has not been available in Japan thus far. The Ministry of Health, Labour, and Welfare started to prepare the guideline for nonclinical safety studies in juvenile animals since October 2010. After completion of the Japanese guideline, guidelines would exist in the three regions: Japan, US, and Europe. Then, global development of pediatric pharmaceuticals would be accelerated effectively.     

Model requirements for Biobank Software Systems

Biobanks are essential tools in diagnostics and therapeutics research and development related to personalized medicine. Several international recommendations, standards and guidelines exist that discuss the legal, ethical, technological, and management requirements of biobanks. Today's biobanks are much more than just collections of biospecimens. They also store a huge amount of data related to biological samples which can be either clinical data or data coming from biochemical experiments. A well-designed biobank software system also provides the possibility of finding associations between stored elements. Modern research biobanks are able to manage multicenter sample collections while fulfilling all requirements of data protection and security. While developing several biobanks and analyzing the data stored in them, our research group recognized the need for a well-organized, easy-to-check requirements guideline that can be used to develop biobank software systems. International best practices along with relevant ICT standards were integrated into a comprehensive guideline: The Model Requirements for the Management of Biological Repositories (BioReq), which covers the full range of activities related to biobank development. The guideline is freely available on the Internet for the research community. AVAILABILITY: The database is available for free at http://bioreq.astridbio.com/bioreq_v2.0.pdf.     

三色迷彩油添加避蚊胺对白纹伊蚊驱避效果的实验室评价

参照中华人民共和国国家标准GB/T 17322.10-1998和《军队特需药品研发勤务技术指导原则(试行)》规定的测试方法,测试了避蚊胺和水溶性二氧化钛处理迷彩油(CFP)对白纹伊蚊Aedes albopictus的驱避效果,为研发我军专用的防蚊虫迷彩油提供依据。结果表明:在3种颜色迷彩油中加入DEET后,不影响DEET的驱蚊效果;添加不同浓度DEET对蚊虫的驱避效果差异显著,20%DEET和25%DEET+迷彩油对白纹伊蚊的平均防护时间均达到或超过10 h,达到《军队特需药品研发勤务技术指导原则(试行)》规定的标准,15%DEET+迷彩油平均防护时间不足10 h,未达《军队特需药品研发勤务技术指导原则(试行)》规定的标准。研究表明:将适量DEET与迷彩油混合涂抹于皮肤可能作为演习或战时的伪装和防蚊的一种有效的综合保护措施。     

Leidraad voor medisch wetenschappelijk onderzoek bij ouderen

This paper describes the development of a guideline on medical research with older subjects. Although our society is aging, evidence on health care for older persons is lacking on many topics, because these subjects are underrepresented in most drug and non-drug trials, while these services are used many older persons, and result in many adverse reactions and unplanned hospitalisation. Part of the reasons for this underrepresentation is the multimorbidity, often leading to exclusion, but also the lack of appropriate research methods plays a major role. Therefore, this paper describes the methods and results of the development of a multidisciplinary, evidence based guideline on how to conduct medical research in older persons. The recent changes in European and Dutch legislation on medical research were another reason to start this guideline project. We conducted two systematic reviews (on informed consent and recruitment) and conducted surveys and focus groups on the other four topics covered by the paper: proportionality, resistance, drop out, and societal relevance. In total we formulated 45 recommendations, all agreed on in consensus meetings, in which older persons’ representatives played a major role. This Guideline on medical research in older persons, will be implemented via the ethical review boards, the medical scientific committees, and the Ministry of Health in the Netherlands, who commissioned the guideline work. We hope the guideline stimulates quality and quantity of research on older adults to answer the increasing number of societal and scientific questions with regard to this populations.     

Conflict of Interest Disclosures for Clinical Practice Guidelines in the National Guideline Clearinghouse

Background

Conflict of interest (COI) is an important potential source of bias in the development of clinical practice guidelines (CPGs) and high rates of COI among guideline authors have been reported in the past. Our objective was to report current rates of disclosure and specific author COI across a broad range of CPGs and to examine whether CPG characteristics were associated with the presence of disclosures and of conflicts.

Methods and Findings

We selected a random sample of 250 CPGs listed in the National Guideline Clearinghouse on November 22, 2010, representing approximately a 10% sample of guidelines listed in the NGC on that date. We abstracted information on author COI from each CPG and examined predictors of the disclosures and COI using a logistic generalized estimating equation regression model. 87% of organizations developing guidelines had a CPG-specific policy, however, 40% of CPGs did not indicate that they had collected disclosures from guideline authors. In addition, 42% of organizations that did collect author disclosures did not have those disclosures available in the public domain. Of CPGs where we had disclosures for all authors, 60% had one or more authors with a conflict. On average, 28% of the authors of CPGs with available disclosures had a COI. Guidelines that were published in journals with an impact factor greater than 5.0 were more likely to have one or more authors with a COI than guidelines not published in journals.

Conclusions

Rates of disclosure of author COI and the public availability of that information are unacceptably low, however rates of COI among guideline authors may have decreased in recent years. Continued efforts are needed to establish and enforce optimal COI policies in clinical practice guideline development in order to minimize the risk of bias associated with those conflicts.     

Exercise-based cardiac rehabilitation in patients with coronary heart disease: a practice guideline

Background

To improve the quality of exercise-based cardiac rehabilitation (CR) in patients with coronary heart disease (CHD) the CR guideline from the Dutch Royal Society for Physiotherapists (KNGF) has been updated. This guideline can be considered an addition to the 2011 Dutch Multidisciplinary CR guideline, as it includes several novel topics.

Methods

A systematic literature search was performed to formulate conclusions on the efficacy of exercise-based interventions during all CR phases in patients with CHD. Evidence was graded (1–4) according the Dutch evidence-based guideline development (EBRO) criteria. In case of insufficient scientific evidence, recommendations were based on expert opinion. This guideline comprised a structured approach including assessment, treatment and evaluation.

Results

Recommendations for exercise-based CR were formulated covering the following topics: preoperative physiotherapy, mobilisation during the clinical phase, aerobic exercise, strength training, and relaxation therapy during the outpatient rehabilitation phase, and adoption and monitoring of a physically active lifestyle after outpatient rehabilitation.

Conclusions

There is strong evidence for the effectiveness of exercise-based CR during all phases of CR. The implementation of this guideline in clinical practice needs further evaluation as well as the maintenance of an active lifestyle after supervised rehabilitation.     

Designing an automated clinical decision support system to match clinical practice guidelines for opioid therapy for chronic pain

Background

Opioid prescribing for chronic pain is common and controversial, but recommended clinical practices are followed inconsistently in many clinical settings. Strategies for increasing adherence to clinical practice guideline recommendations are needed to increase effectiveness and reduce negative consequences of opioid prescribing in chronic pain patients.

Methods

Here we describe the process and outcomes of a project to operationalize the 2003 VA/DOD Clinical Practice Guideline for Opioid Therapy for Chronic Non-Cancer Pain into a computerized decision support system (DSS) to encourage good opioid prescribing practices during primary care visits. We based the DSS on the existing ATHENA-DSS. We used an iterative process of design, testing, and revision of the DSS by a diverse team including guideline authors, medical informatics experts, clinical content experts, and end-users to convert the written clinical practice guideline into a computable algorithm to generate patient-specific recommendations for care based upon existing information in the electronic medical record (EMR), and a set of clinical tools.

Results

The iterative revision process identified numerous and varied problems with the initially designed system despite diverse expert participation in the design process. The process of operationalizing the guideline identified areas in which the guideline was vague, left decisions to clinical judgment, or required clarification of detail to insure safe clinical implementation. The revisions led to workable solutions to problems, defined the limits of the DSS and its utility in clinical practice, improved integration into clinical workflow, and improved the clarity and accuracy of system recommendations and tools.

Conclusions

Use of this iterative process led to development of a multifunctional DSS that met the approval of the clinical practice guideline authors, content experts, and clinicians involved in testing. The process and experiences described provide a model for development of other DSSs that translate written guidelines into actionable, real-time clinical recommendations.

     

Exercise-based cardiac rehabilitation in patients with chronic heart failure: a Dutch practice guideline

Rationale

To improve the quality of exercise-based cardiac rehabilitation (CR) in patients with chronic heart failure (CHF) a practice guideline from the Dutch Royal Society for Physiotherapy (KNGF) has been developed.

Guideline development

A systematic literature search was performed to formulate conclusions on the efficacy of exercise-based intervention during all CR phases in patients with CHF. Evidence was graded (1–4) according the Dutch evidence-based guideline development criteria.

Clinical and research recommendations

Recommendations for exercise-based CR were formulated covering the following topics: mobilisation and treatment of pulmonary symptoms (if necessary) during the clinical phase, aerobic exercise, strength training (inspiratory muscle training and peripheral muscle training) and relaxation therapy during the outpatient CR phase, and adoption and monitoring training after outpatient CR.

Applicability and implementation issues

This guideline provides the physiotherapist with an evidence-based instrument to assist in clinical decision-making regarding patients with CHF. The implementation of the guideline in clinical practice needs further evaluation.

Conclusion

This guideline outlines best practice standards for physiotherapists concerning exercise-based CR in CHF patients. Research is needed on strategies to improve monitoring and follow-up of the maintenance of a physical active lifestyle after supervised CR.     

Establishing a knowledge trail from molecular experiments to clinical trials

During the development cycle of a new antibody therapy, the therapeutic agent will be tested on subsequently more biologically complex models. New experiments' designs are based upon data gathered from prior models. New researchers who inherit the data and researchers from groups with different cultures or expertise are often called upon to interpret these data. Experiments which are not recorded consistently or employ ambiguous terminology can make interpreting these results difficult. The researcher who had originally collected the data may not be at hand to correct any misunderstanding or offer clarification and data can be unknowingly misused. This introduces an element of risk into the therapy development process. We have developed a reporting guideline for recording therapy experiments. This guideline consists of a checklist of data to be recorded from antibody therapy experiments performed in molecular, cellular, animal and clinical model.     

Transforming scientific evidence into better consumer choices

Translational research using evidence-based and comparative effectiveness research continues to evolve, becoming a useful tool in improving informed consent and decision-making in the clinical setting. While in development, emerging technologies, including cellular and molecular biology, are leading to establishing evidence-based dental practices. One emerging technology, which conjoins bench proteomic findings to clinical decision-making for treatment intervention, is the Translational Evidence Mechanism. This mechanism was developed to be a foundation for a compact between researcher, translational researcher, clinician, and patient. The output of such a mechanism is the clinical practice guideline (CPG), an interactive tool for dentists and patients to game evidence in reaching optimum clinical decisions that correspond to individual patient preferences and values. As such, the clinical practice guideline requires the vesting of decision, utility, and cost best evidence. Evidence-based research provides decision data, a first attempt at supporting decision-making by providing best outcome data. Since then comparative effectiveness research has emerged, using systematic review analysis to compare similar treatments or procedures in maximizing the choice of the most effective cost/benefit option within the context of best evidence. With innovation in the clinical practice guideline for optimizing efficacy and comparative effectiveness research, evidence-based practices will shape a new approach to health-based systems that adhere to shared decision-making between bench scientists, healthcare providers and patients.     

Transforming scientific evidence into better consumer choices

Translational research using evidence-based and comparative effectiveness research continues to evolve, becoming a useful tool in improving informed consent and decision-making in the clinical setting. While in development, emerging technologies, including cellular and molecular biology, are leading to establishing evidence-based dental practices. One emerging technology, which conjoins bench proteomic findings to clinical decision-making for treatment intervention, is the Translational Evidence Mechanism. This mechanism was developed to be a foundation for a compact between researcher, translational researcher, clinician, and patient. The output of such a mechanism is the clinical practice guideline (CPG), an interactive tool for dentists and patients to game evidence in reaching optimum clinical decisions that correspond to individual patient preferences and values. As such, the clinical practice guideline requires the vesting of decision, utility, and cost best evidence. Evidence-based research provides decision data, a first attempt at supporting decision-making by providing best outcome data. Since then comparative effectiveness research has emerged, using systematic review analysis to compare similar treatments or procedures in maximizing the choice of the most effective cost/benefit option within the context of best evidence. With innovation in the clinical practice guideline for optimizing efficacy and comparative effectiveness research, evidence-based practices will shape a new approach to health-based systems that adhere to shared decision-making between bench scientists, healthcare providers and patients.     

Economic Evaluation of Active Implementation versus Guideline Dissemination for Evidence-Based Care of Acute Low-Back Pain in a General Practice Setting

Introduction

The development and publication of clinical practice guidelines for acute low-back pain has resulted in evidence-based recommendations that have the potential to improve the quality and safety of care for acute low-back pain. Development and dissemination of guidelines may not, however, be sufficient to produce improvements in clinical practice; further investment in active implementation of guideline recommendations may be required. Further research is required to quantify the trade-off between the additional upfront cost of active implementation of guideline recommendations for low-back pain and any resulting improvements in clinical practice.

Methods

Cost-effectiveness analysis alongside the IMPLEMENT trial from a health sector perspective to compare active implementation of guideline recommendations via the IMPLEMENT intervention (plus standard dissemination) against standard dissemination alone.

Results

The base-case analysis suggests that delivery of the IMPLEMENT intervention dominates standard dissemination (less costly and more effective), yielding savings of $135 per x-ray referral avoided (-$462.93/3.43). However, confidence intervals around point estimates for the primary outcome suggest that – irrespective of willingness to pay (WTP) – we cannot be at least 95% confident that the IMPLEMENT intervention differs in value from standard dissemination.

Conclusions

Our findings demonstrate that moving beyond development and dissemination to active implementation entails a significant additional upfront investment that may not be offset by health gains and/or reductions in health service utilization of sufficient magnitude to render active implementation cost-effective.     

American Association of Clinical Endocrinology Clinical Practice Guideline for the Diagnosis and Management of Nonalcoholic Fatty Liver Disease in Primary Care and Endocrinology Clinical Settings: Co-Sponsored by the American Association for the Study of Liver Diseases (AASLD)

ObjectiveTo provide evidence-based recommendations regarding the diagnosis and management of nonalcoholic fatty liver disease (NAFLD) and nonalcoholic steatohepatitis (NASH) to endocrinologists, primary care clinicians, health care professionals, and other stakeholders.MethodsThe American Association of Clinical Endocrinology conducted literature searches for relevant articles published from January 1, 2010, to November 15, 2021. A task force of medical experts developed evidence-based guideline recommendations based on a review of clinical evidence, expertise, and informal consensus, according to established American Association of Clinical Endocrinology protocol for guideline development.Recommendation SummaryThis guideline includes 34 evidence-based clinical practice recommendations for the diagnosis and management of persons with NAFLD and/or NASH and contains 385 citations that inform the evidence base.ConclusionNAFLD is a major public health problem that will only worsen in the future, as it is closely linked to the epidemics of obesity and type 2 diabetes mellitus. Given this link, endocrinologists and primary care physicians are in an ideal position to identify persons at risk on to prevent the development of cirrhosis and comorbidities. While no U.S. Food and Drug Administration-approved medications to treat NAFLD are currently available, management can include lifestyle changes that promote an energy deficit leading to weight loss; consideration of weight loss medications, particularly glucagon-like peptide-1 receptor agonists; and bariatric surgery, for persons who have obesity, as well as some diabetes medications, such as pioglitazone and glucagon-like peptide-1 receptor agonists, for those with type 2 diabetes mellitus and NASH. Management should also promote cardiometabolic health and reduce the increased cardiovascular risk associated with this complex disease.     

Wastewater reuse in agriculture and the risk of intestinal nematode infection

The increasing use of wastewater for crop irrigation in arid and semi-arid zones means that wastewater parasitology is becoming on important research area. The World Health Organization's (WHO) current guideline states that only treated wastewater containing no more than one human intestinal nematode egg per litre should be used for irrigation. This guideline was based on very limited epidemiological evidence. In this article Rachel Ayres, Rebecca Stott, Duncan Mara and Donald Lee describe their recent work in Yorkshire and northeast Brazil and suggest that their results may be the first evidence for relaxing the WHO guideline for restricted irrigation to no more than ten intestinal nematode eggs per litre.     

Evaluating the Good Ontology Design Guideline (GoodOD) with the Ontology Quality Requirements and Evaluation Method and Metrics (OQuaRE)

Objective

To (1) evaluate the GoodOD guideline for ontology development by applying the OQuaRE evaluation method and metrics to the ontology artefacts that were produced by students in a randomized controlled trial, and (2) informally compare the OQuaRE evaluation method with gold standard and competency questions based evaluation methods, respectively.

Background

In the last decades many methods for ontology construction and ontology evaluation have been proposed. However, none of them has become a standard and there is no empirical evidence of comparative evaluation of such methods. This paper brings together GoodOD and OQuaRE. GoodOD is a guideline for developing robust ontologies. It was previously evaluated in a randomized controlled trial employing metrics based on gold standard ontologies and competency questions as outcome parameters. OQuaRE is a method for ontology quality evaluation which adapts the SQuaRE standard for software product quality to ontologies and has been successfully used for evaluating the quality of ontologies.

Methods

In this paper, we evaluate the effect of training in ontology construction based on the GoodOD guideline within the OQuaRE quality evaluation framework and compare the results with those obtained for the previous studies based on the same data.

Results

Our results show a significant effect of the GoodOD training over developed ontologies by topics: (a) a highly significant effect was detected in three topics from the analysis of the ontologies of untrained and trained students; (b) both positive and negative training effects with respect to the gold standard were found for five topics.

Conclusion

The GoodOD guideline had a significant effect over the quality of the ontologies developed. Our results show that GoodOD ontologies can be effectively evaluated using OQuaRE and that OQuaRE is able to provide additional useful information about the quality of the GoodOD ontologies.