Improving benchmarking by using an explicit framework for the development of composite indicators: an example using pediatric quality of care

Background

The measurement of healthcare provider performance is becoming more widespread. Physicians have been guarded about performance measurement, in part because the methodology for comparative measurement of care quality is underdeveloped. Comprehensive quality improvement will require comprehensive measurement, implying the aggregation of multiple quality metrics into composite indicators.

Objective

To present a conceptual framework to develop comprehensive, robust, and transparent composite indicators of pediatric care quality, and to highlight aspects specific to quality measurement in children.

Methods

We reviewed the scientific literature on composite indicator development, health systems, and quality measurement in the pediatric healthcare setting. Frameworks were selected for explicitness and applicability to a hospital-based measurement system.

Results

We synthesized various frameworks into a comprehensive model for the development of composite indicators of quality of care. Among its key premises, the model proposes identifying structural, process, and outcome metrics for each of the Institute of Medicine's six domains of quality (safety, effectiveness, efficiency, patient-centeredness, timeliness, and equity) and presents a step-by-step framework for embedding the quality of care measurement model into composite indicator development.

Conclusions

The framework presented offers researchers an explicit path to composite indicator development. Without a scientifically robust and comprehensive approach to measurement of the quality of healthcare, performance measurement will ultimately fail to achieve its quality improvement goals.

     

Measuring and improving the quality of mental health care: a global perspective

Mental disorders are common worldwide, yet the quality of care for these disorders has not increased to the same extent as that for physical conditions. In this paper, we present a framework for promoting quality measurement as a tool for improving quality of mental health care. We identify key barriers to this effort, including lack of standardized information technology‐based data sources, limited scientific evidence for mental health quality measures, lack of provider training and support, and cultural barriers to integrating mental health care within general health environments. We describe several innovations that are underway worldwide which can mitigate these barriers. Based on these experiences, we offer several recommendations for improving quality of mental health care. Health care payers and providers will need a portfolio of validated measures of patient‐centered outcomes across a spectrum of conditions. Common data elements will have to be developed and embedded within existing electronic health records and other information technology tools. Mental health outcomes will need to be assessed more routinely, and measurement‐based care should become part of the overall culture of the mental health care system. Health care systems will need a valid way to stratify quality measures, in order to address potential gaps among subpopulations and identify groups in most need of quality improvement. Much more attention should be devoted to workforce training in and capacity for quality improvement. The field of mental health quality improvement is a team sport, requiring coordination across different providers, involvement of consumer advocates, and leveraging of resources and incentives from health care payers and systems.     

Improving quality of care: focus on liaison old age psychiatry

Introduction Elderly patients occupy up to 65% of acute hospital beds and a significant proportion of them present with a comorbid psychiatric condition such as depression, delirium or dementia. Liaison old age psychiatry (LOAP) services have been developed to provide psychiatric consultation in medical and surgical settings, improving at the same time the knowledge and expertise of general ward staff.Objective The aim of this study is to evaluate clinical characteristics across different psychiatric disorders among elderly patients in medical wards.Method A prospective observational study was developed between October 2011 and January 2013, which involved 107 subjects aged 65 years or older that were hospitalised in the Department of Internal Medicine and referred to the LOAP service. Psychiatric diagnostic was assessed using the Confusion Assessment Method, the Geriatric Depression Scale, the Mini-Mental State Examination and the Clinical Global Impression Scale.Results Delirium (40.6%), depression (22.4%) and dementia (20.4%) were the most common psychiatric diagnoses. Patients with delirium were significantly older, had more severe psychiatric symptomatology (mean CGI = 5.35) and presented infectious processes as acute medical conditions more frequently than the other patients.Conclusion Psychiatric disturbances occurring in elderly inpatients in medical wards are highly prevalent and complex. A LOAP service may play an important role in effectively reducing the overutilisation and consumption of health resources through early recognition of these conditions, effective management and prevention of adverse outcomes, and effective communication with out-patient clinics, community mental health teams and day-care centres.     

Improving the quality of care for infants: a cluster randomized controlled trial

Background

We developed and tested a new method, called the Evidence-based Practice for Improving Quality method, for continuous quality improvement.

Methods

We used cluster randomization to assign 6 neonatal intensive care units (ICUs) to reduce nosocomial infection (infection group) and 6 ICUs to reduce bronchopulmonary dysplasia (pulmonary group). We included all infants born at 32 or fewer weeks gestation. We collected baseline data for 1 year. Practice change interventions were implemented using rapid-change cycles for 2 years.

Results

The difference in incidence trends (slopes of trend lines) between the ICUs in the infection and pulmonary groups was − 0.0020 (95% confidence interval [CI] − 0.0007 to 0.0004) for nosocomial infection and − 0.0006 (95% CI − 0.0011 to − 0.0001) for bronchopulmonary dysplasia.

Interpretation

The results suggest that the Evidence-based Practice for Improving Quality method reduced bronchopulmonary dysplasia in the neonatal ICU and that it may reduce nosocomial infection.Although methods for continuous quality improvement have been used to improve outcomes,^1–3 some, such as the National Institutes of Child Health and Human Development Quality Collaborative,⁴ have reported little or no effect in neonatal intensive care units (ICUs). These methods have been criticized for being based on intuition and anecdotes rather than on evidence.⁵ To address these concerns, researchers have developed methods aimed at improving the use of evidence in quality improvement. Tarnow-Mordi and colleagues,⁶ Sankaran and colleagues⁷ and others^8–10 have used benchmarking instruments^6,8,11 to show risk-adjusted variations in outcomes in neonatal ICUs. Synnes and colleagues¹² reported that variations in the rates of intraventricular hemorrhage could be attributed to practice differences. MacNab and colleagues¹³ showed how multilevel modelling methods can be used to identify practice differences associated with variations in outcomes for targeted interventions and to quantify their attributable risks.Building on these results, we developed the Evidence-based Practice for Improving Quality method for continuous quality improvement. This method is based on 3 pillars: the use of evidence from published literature; the use of data from participating hospitals to identify hospital-specific practices for targeted intervention; and the use of a national network to share expertise. By selectively targeting hospital-specific practices for intervention, this method reduces the reliance on intuition and anecdotes that are associated with existing quality-improvement methods.Our objective was to evaluate the efficacy of the Evidence-based Practice for Improving Quality method by conducting a prospective cluster randomized controlled trial to reduce nosocomial infection and bronchopulmonary dysplasia among infants born at 32 or fewer weeks’ gestation and admitted to 12 Canadian Neonatal Network hospitals¹⁴ over a 36-month period. We hypothesized that the incidence of nosocomial infection would be reduced among infants in ICUs randomized to reduce infection but not among those in ICUs randomized to reduce bronchopulmonary dysplasia. We also hypothesized that the incidence of bronchopulmonary dysplasia would be reduced among infants in the ICUs randomized to reduce this outcome but not among those in ICUs randomized to reduce infections.     

Improving real-time forecasting of water quality indicators with combination of process-based models and data assimilation technique

Water quality indicators can be used to characterize the status and quantify and qualify the change of aquatic ecosystems under different disturbance regimes. Although many studies have been done to develop and assess indicators and discuss interactions among them, few studies have focused on how to improve the predicted indicators and explore their variations in receiving water bodies. Accurate and effective predictions of ecological indictors are critical to better understand changes of water quality in aquatic ecosystems, especially for the real-time forecasting. Process-based water quality models can predict the spatiotemporal variations of the water quality indicators and provide useful information for policy-makers on sound management of water resources. Given their inherent constraints, however, such process models alone cannot actually guarantee perfect results since water quality models generally have a large number of parameters and involve many processes which are too complex to be efficiently calibrated. To overcome these limitations and explore a fast and efficient forecasting method for the change of water quality indictors, we proposed a new framework which combines the process-based models and data assimilation technique. Unlike most traditional approaches in which only the model parameters or initial conditions are updated or corrected and the models are run online, this framework allows the information extracted from observations and outputs of process models to be directly used in a data-driven local/modified local model. The results from the data-driven model are then assimilated into the original process model to further improve its forecasting ability. This approach can be efficiently run offline to directly correct and update the output of water quality models. We applied this framework in a real case study in Singapore. Two of the water quality indicators, namely salinity and oxygen were selected and tested against the observations, suggesting that a good performance of improving the model results and reducing computation time can be obtained. This approach is simple and efficient, especially suitable for real-time forecasting systems. Thus, it can enhance forecasting of water quality indictors and thereby facilitate the effective management of water resources.     

Data quality issues impede comparability of hospital treatment delay performance indicators

Aim

To assess the comparability of five performance indicator scores for treatment delay among patients diagnosed with ST-segment elevation myocardial infarction (STEMI) undergoing primary percutaneous coronary intervention in relation to the quality of the underlying data.

Methods

Secondary analyses were performed on data from 1017 patients in seven Dutch hospitals. Data were collected using standardised forms for patients discharged in 2012. Comparability was assessed as the number of occasions the indicator threshold was reached for each hospital.

Results

Hospitals recorded different time points based on different interpretations of the definitions. This led to substantial differences in indicator scores, ranging from 57 to 100 % of the indictor threshold being reached. Some hospitals recorded all the required data elements for calculating the performance indicators but none of the data elements could be retrieved in a fully automated way. Moreover, recording accessibility and completeness of time points varied widely within and between hospitals.

Conclusion

Hospitals use different definitions for treatment delay and vary greatly in the extent to which the necessary data are available, accessible and complete, impeding comparability between hospitals. Indicator developers, users and hospitals providing data should be aware of these issues and aim to improve data quality in order to facilitate comparability of performance indicators.