Application of Balanced Scorecard in the Evaluation of a Complex Health System Intervention: 12 Months Post Intervention Findings from the BHOMA Intervention: A Cluster Randomised Trial in Zambia

Introduction

In many low income countries, the delivery of quality health services is hampered by health system-wide barriers which are often interlinked, however empirical evidence on how to assess the level and scope of these barriers is scarce. A balanced scorecard is a tool that allows for wider analysis of domains that are deemed important in achieving the overall vision of the health system. We present the quantitative results of the 12 months follow-up study applying the balanced scorecard approach in the BHOMA intervention with the aim of demonstrating the utility of the balanced scorecard in evaluating multiple building blocks in a trial setting.

Methods

The BHOMA is a cluster randomised trial that aims to strengthen the health system in three rural districts in Zambia. The intervention aims to improve clinical care quality by implementing practical tools that establish clear clinical care standards through intensive clinic implementations. This paper reports the findings of the follow-up health facility survey that was conducted after 12 months of intervention implementation. Comparisons were made between those facilities in the intervention and control sites. STATA version 12 was used for analysis.

Results

The study found significant mean differences between intervention(I) and control (C) sites in the following domains: Training domain (Mean I:C; 87.5.vs 61.1, mean difference 23.3, p = 0.031), adult clinical observation domain (mean I:C; 73.3 vs.58.0, mean difference 10.9, p = 0.02 ) and health information domain (mean I:C; 63.6 vs.56.1, mean difference 6.8, p = 0.01. There was no gender differences in adult service satisfaction. Governance and motivation scores did not differ between control and intervention sites.

Conclusion

This study demonstrates the utility of the balanced scorecard in assessing multiple elements of the health system. Using system wide approaches and triangulating data collection methods seems to be key to successful evaluation of such complex health intervention.

Trial number

ClinicalTrials.gov {"type":"clinical-trial","attrs":{"text":"NCT01942278","term_id":"NCT01942278"}}NCT01942278     

Health facility characteristics and their relationship to coverage of PMTCT of HIV services across four African countries: the PEARL study

Background

Health facility characteristics associated with effective prevention of mother-to-child transmission of HIV (PMTCT) coverage in sub-Saharan are poorly understood.

Methodology/Principal Findings

We conducted surveys in health facilities with active PMTCT services in Cameroon, Cote d''Ivoire, South Africa, and Zambia. Data was compiled via direct observation and exit interviews. We constructed composite scores to describe provision of PMTCT services across seven topical areas: antenatal quality, PMTCT quality, supplies available, patient satisfaction, patient understanding of medication, and infrastructure quality. Pearson correlations and Generalized Estimating Equations (GEE) to account for clustering of facilities within countries were used to evaluate the relationship between the composite scores, total time of visit and select individual variables with PMTCT coverage among women delivering.Between July 2008 and May 2009, we collected data from 32 facilities; 78% were managed by the government health system. An opt-out approach for HIV testing was used in 100% of facilities in Zambia, 63% in Cameroon, and none in Côte d''Ivoire or South Africa. Using Pearson correlations, PMTCT coverage (median of 55%, (IQR: 33–68) was correlated with PMTCT quality score (rho = 0.51; p = 0.003); infrastructure quality score (rho = 0.43; p = 0.017); time spent at clinic (rho = 0.47; p = 0.013); patient understanding of medications score (rho = 0.51; p = 0.006); and patient satisfaction quality score (rho = 0.38; p = 0.031). PMTCT coverage was marginally correlated with the antenatal quality score (rho = 0.304; p = 0.091). Using GEE adjustment for clustering, the, antenatal quality score became more strongly associated with PMTCT coverage (p<0.001) and the PMTCT quality score and patient understanding of medications remained marginally significant.

Conclusions/Results

We observed a positive relationship between an antenatal quality score and PMTCT coverage but did not identify a consistent set of variables that predicted PMTCT coverage.     

Measuring Health System Strengthening: Application of the Balanced Scorecard Approach to Rank the Baseline Performance of Three Rural Districts in Zambia

Introduction

There is growing interest in health system performance and recently WHO launched a report on health systems strengthening emphasising the need for close monitoring using system-wide approaches. One recent method is the balanced scorecard system. There is limited application of this method in middle- and low-income countries. This paper applies the concept of balanced scorecard to describe the baseline status of three intervention districts in Zambia.

Methodology

The Better Health Outcome through Mentoring and Assessment (BHOMA) project is a randomised step-wedged community intervention that aims to strengthen the health system in three districts in the Republic of Zambia. To assess the baseline status of the participating districts we used a modified balanced scorecard approach following the domains highlighted in the MOH 2011 Strategic Plan.

Results

Differences in performance were noted by district and residence. Finance and service delivery domains performed poorly in all study districts. The proportion of the health workers receiving training in the past 12 months was lowest in Kafue (58%) and highest in Luangwa district (77%). Under service capacity, basic equipment and laboratory capacity scores showed major variation, with Kafue and Luangwa having lower scores when compared to Chongwe. The finance domain showed that Kafue and Chongwe had lower scores (44% and 47% respectively). Regression model showed that children''s clinical observation scores were negatively correlated with drug availability (coeff −0.40, p = 0.02). Adult clinical observation scores were positively association with adult service satisfaction score (coeff 0.82, p = 0.04) and service readiness (coeff 0.54, p = 0.03).

Conclusion

The study applied the balanced scorecard to describe the baseline status of 42 health facilities in three districts of Zambia. Differences in performance were noted by district and residence in most domains with finance and service delivery performing poorly in all study districts. This tool could be valuable in monitoring and evaluation of health systems.     

Molecular characterization of Theileria orientalis causing fatal infection in crossbred adult bovines of South India

The disease condition attributed to have been caused by Theileria orientalis is generally benign. However, it is also thought that the parasite, at least some strains of it, can cause fatal disease. The present communication deals with the clinical signs, postmortem lesions and diagnosis of a fatal disease due to T. orientalis which caused mortality in crossbred adult bovines of South India. High body temperature, lacrimation, nasal discharge, swollen lymph nodes and haemoglobinuria were the symptoms observed. The postmortem lesions observed were punched out ulcers in abomasum, enlargement of spleen, massive pulmonary oedema, frothy exudates in trachea, epicardial and endocardial haemorrhage and haemorrhagic duodenitis. Peripheral blood smear examination revealed rod shaped Theileria sp. organisms. Polymerase chain reaction that amplify the T. orientalis specific P(32/33) gene, followed by cloning and sequencing, revealed maximum homology with Narathiwat (Thailand) and Jingole -1 (Indonesia) isolates which were positioned as isolate type 7 of T. orientalis.     

Estimating loss to follow-up in HIV-infected patients on antiretroviral therapy: the effect of the competing risk of death in Zambia and Switzerland

Background

Loss to follow-up (LTFU) is common in antiretroviral therapy (ART) programmes. Mortality is a competing risk (CR) for LTFU; however, it is often overlooked in cohort analyses. We examined how the CR of death affected LTFU estimates in Zambia and Switzerland.

Methods and Findings

HIV-infected patients aged ≥18 years who started ART 2004–2008 in observational cohorts in Zambia and Switzerland were included. We compared standard Kaplan-Meier curves with CR cumulative incidence. We calculated hazard ratios for LTFU across CD4 cell count strata using cause-specific Cox models, or Fine and Gray subdistribution models, adjusting for age, gender, body mass index and clinical stage. 89,339 patients from Zambia and 1,860 patients from Switzerland were included. 12,237 patients (13.7%) in Zambia and 129 patients (6.9%) in Switzerland were LTFU and 8,498 (9.5%) and 29 patients (1.6%), respectively, died. In Zambia, the probability of LTFU was overestimated in Kaplan-Meier curves: estimates at 3.5 years were 29.3% for patients starting ART with CD4 cells <100 cells/µl and 15.4% among patients starting with ≥350 cells/µL. The estimates from CR cumulative incidence were 22.9% and 13.6%, respectively. Little difference was found between naïve and CR analyses in Switzerland since only few patients died. The results from Cox and Fine and Gray models were similar: in Zambia the risk of loss to follow-up and death increased with decreasing CD4 counts at the start of ART, whereas in Switzerland there was a trend in the opposite direction, with patients with higher CD4 cell counts more likely to be lost to follow-up.

Conclusions

In ART programmes in low-income settings the competing risk of death can substantially bias standard analyses of LTFU. The CD4 cell count and other prognostic factors may be differentially associated with LTFU in low-income and high-income settings.     

Excellent adherence to antiretrovirals in HIV+ Zambian children is compromised by disrupted routine, HIV nondisclosure, and paradoxical income effects

Introduction

A better understanding of pediatric antiretroviral therapy (ART) adherence in sub-Saharan Africa is necessary to develop interventions to sustain high levels of adherence.

Methodology/Principal Findings

Adherence among 96 HIV-infected Zambian children (median age 6, interquartile range [IQR] 2,9) initiating fixed-dose combination ART was measured prospectively (median 23 months; IQR 20,26) with caregiver report, clinic and unannounced home-based pill counts, and medication event monitoring systems (MEMS). HIV-1 RNA was determined at 48 weeks. Child and caregiver characteristics, socio-demographic status, and treatment-related factors were assessed as predictors of adherence. Median adherence was 97.4% (IQR 96.1,98.4%) by visual analog scale, 94.8% (IQR 86,100%) by caregiver-reported last missed dose, 96.9% (IQR 94.5,98.2%) by clinic pill count, 93.4% (IQR 90.2,96.7%) by unannounced home-based pill count, and 94.8% (IQR 87.8,97.7%) by MEMS. At 48 weeks, 72.6% of children had HIV-1 RNA <50 copies/ml. Agreement among adherence measures was poor; only MEMS was significantly associated with viral suppression (p = 0.013). Predictors of poor adherence included changing residence, school attendance, lack of HIV disclosure to children aged nine to 15 years, and increasing household income.

Conclusions/Significance

Adherence among children taking fixed-dose combination ART in sub-Saharan Africa is high and sustained over two years. However, certain groups are at risk for treatment failure, including children with disrupted routines, no knowledge of their HIV diagnosis among older children, and relatively high household income, possibly reflecting greater social support in the setting of greater poverty.     

Evaluation of a Density-Based Rapid Diagnostic Test for Sickle Cell Disease in a Clinical Setting in Zambia

Although simple and low-cost interventions for sickle cell disease (SCD) exist in many developing countries, child mortality associated with SCD remains high, in part, because of the lack of access to diagnostic tests for SCD. A density-based test using aqueous multiphase systems (SCD-AMPS) is a candidate for a low-cost, point-of-care diagnostic for SCD. In this paper, the field evaluation of SCD-AMPS in a large (n = 505) case-control study in Zambia is described. Of the two variations of the SCD-AMPS used, the best system (SCD-AMPS-2) demonstrated a sensitivity of 86% (82–90%) and a specificity of 60% (53–67%). Subsequent analysis identified potential sources of false positives that include clotting, variation between batches of SCD-AMPS, and shipping conditions. Importantly, SCD-AMPS-2 was 84% (62–94%) sensitive in detecting SCD in children between 6 months and 1 year old. In addition to an evaluation of performance, an assessment of end-user operability was done with health workers in rural clinics in Zambia. These health workers rated the SCD-AMPS tests to be as simple to use as lateral flow tests for malaria and HIV.     

Unique TCR β-subunit variable gene haplotypes in Africans

This study investigated polymorphisms of genes in two regions of the T-cell antigen receptor beta-subunit (TCRB) locus, including BV9S2P, and BV6S7 in a 5' linkage group, and BV8S3, BV24S1, BV25S1, BV18S1, BV2S1, BV15S1 and BV3S1 in a 3' linkage group. These loci have been genotyped in individuals from five regions in Africa, including The Gambia, Nigeria, Cameroon, Tanzania, and Zambia, and in individuals from northern Britain, northern India, and Papua New Guinea (PNG). In the 3' linkage group, 11 unique haplotypes were identified in the combined African populations; two equally frequent haplotypes represent the majority of African chromosomes. One haplotype was found in all four regions studied. This is the most frequent haplotype in the northern British, northern Indian and PNG populations. Although present, it is infrequent in the African populations. A North-South gradient in the frequency of a common African haplotype was observed. The distribution did not represent that of a known disease. Evidence suggests that malaria is not responsible for selection of these haplotypes. Overall, this study highlights large differences in the genetic constitution of the TCRB locus between Africans and other populations.     

Predominance of one T-cell antigen receptor BV haplotype in African populations

 The human T-cell antigen receptor (TCR) is the counter-receptor for the HLA/peptide complex displayed on the surface of antigen-presenting cells. It confers antigen specificity on T lymphocytes and therefore plays a central role in pathogen recognition and host response. The most frequently used form of the TCR is a heterodimer composed of variable α and β chains. We investigated allele frequencies for four variable-region gene segments of the β chain (2S1, 3S1, 8S3, and 15S1) in 146 Caucasians and 165 Africans. The results reveal significant unexpected differences between the two populations for allele frequencies, phenotypes, genotypes, and haplotypes. Among Caucasians, there are 43 phenotypes, whereas there are 31 among the Africans studied. There are 17 haplotypes in the Caucasian sample but only 10 in Africans. This loss of diversity is largely due to the high frequency of one haplotype in the African sample which represents 65% of the informative chromosomes. At least one copy of this haplotype is present in 90% of informative individuals. As a result, 29% of Africans are homozygous for the common haplotype. Less genetic diversity at TCRBV is unexpected, since Africans usually show greater genetic diversity than other ethnic groups. For example, there are approximately twice as many HLA haplotypes in Africans compared to Caucasians. Homozygosity is also unexpected because it reduces the number of TCR variants available to recognize HLA pathogen-derived peptide complexes. Received: 23 September 1999 / Revised: 2 November 1999