Application of Balanced Scorecard in the Evaluation of a Complex Health System Intervention: 12 Months Post Intervention Findings from the BHOMA Intervention: A Cluster Randomised Trial in Zambia

Introduction

In many low income countries, the delivery of quality health services is hampered by health system-wide barriers which are often interlinked, however empirical evidence on how to assess the level and scope of these barriers is scarce. A balanced scorecard is a tool that allows for wider analysis of domains that are deemed important in achieving the overall vision of the health system. We present the quantitative results of the 12 months follow-up study applying the balanced scorecard approach in the BHOMA intervention with the aim of demonstrating the utility of the balanced scorecard in evaluating multiple building blocks in a trial setting.

Methods

The BHOMA is a cluster randomised trial that aims to strengthen the health system in three rural districts in Zambia. The intervention aims to improve clinical care quality by implementing practical tools that establish clear clinical care standards through intensive clinic implementations. This paper reports the findings of the follow-up health facility survey that was conducted after 12 months of intervention implementation. Comparisons were made between those facilities in the intervention and control sites. STATA version 12 was used for analysis.

Results

The study found significant mean differences between intervention(I) and control (C) sites in the following domains: Training domain (Mean I:C; 87.5.vs 61.1, mean difference 23.3, p = 0.031), adult clinical observation domain (mean I:C; 73.3 vs.58.0, mean difference 10.9, p = 0.02 ) and health information domain (mean I:C; 63.6 vs.56.1, mean difference 6.8, p = 0.01. There was no gender differences in adult service satisfaction. Governance and motivation scores did not differ between control and intervention sites.

Conclusion

This study demonstrates the utility of the balanced scorecard in assessing multiple elements of the health system. Using system wide approaches and triangulating data collection methods seems to be key to successful evaluation of such complex health intervention.

Trial number

ClinicalTrials.gov {"type":"clinical-trial","attrs":{"text":"NCT01942278","term_id":"NCT01942278"}}NCT01942278     

Measuring Health System Strengthening: Application of the Balanced Scorecard Approach to Rank the Baseline Performance of Three Rural Districts in Zambia

Introduction

There is growing interest in health system performance and recently WHO launched a report on health systems strengthening emphasising the need for close monitoring using system-wide approaches. One recent method is the balanced scorecard system. There is limited application of this method in middle- and low-income countries. This paper applies the concept of balanced scorecard to describe the baseline status of three intervention districts in Zambia.

Methodology

The Better Health Outcome through Mentoring and Assessment (BHOMA) project is a randomised step-wedged community intervention that aims to strengthen the health system in three districts in the Republic of Zambia. To assess the baseline status of the participating districts we used a modified balanced scorecard approach following the domains highlighted in the MOH 2011 Strategic Plan.

Results

Differences in performance were noted by district and residence. Finance and service delivery domains performed poorly in all study districts. The proportion of the health workers receiving training in the past 12 months was lowest in Kafue (58%) and highest in Luangwa district (77%). Under service capacity, basic equipment and laboratory capacity scores showed major variation, with Kafue and Luangwa having lower scores when compared to Chongwe. The finance domain showed that Kafue and Chongwe had lower scores (44% and 47% respectively). Regression model showed that children''s clinical observation scores were negatively correlated with drug availability (coeff −0.40, p = 0.02). Adult clinical observation scores were positively association with adult service satisfaction score (coeff 0.82, p = 0.04) and service readiness (coeff 0.54, p = 0.03).

Conclusion

The study applied the balanced scorecard to describe the baseline status of 42 health facilities in three districts of Zambia. Differences in performance were noted by district and residence in most domains with finance and service delivery performing poorly in all study districts. This tool could be valuable in monitoring and evaluation of health systems.     

Adherence support workers: a way to address human resource constraints in antiretroviral treatment programs in the public health setting in Zambia

Background

In order to address staff shortages and improve adherence counseling for people on antiretroviral therapy (ART), the Zambia Prevention, Care and Treatment Partnership (ZPCT) developed an innovative strategy of training community volunteers to provide adherence support at the health facility and community levels. The objective of this study was to assess the effectiveness of these ‘adherence support workers’ (ASWs) in adherence counseling, treatment retention and addressing inadequate human resources at health facilities.

Methodology/Principal Findings

The study used quantitative and qualitative research techniques at five selected ART sites in four provinces in Zambia. Five hundred patients on ART were interviewed using a structured questionnaire to compare the quality of adherence counseling before and after the ASW scheme was introduced at the selected sites and between ASWs and HCWs after the introduction of ASWs. In addition, 3,903 and 4,972 electronic records of all new patients accessing antiretroviral therapy for the time period of 12 months before and 12 months after the introduction of ASWs respectively, were analyzed to assess loss to follow-up rates. Two focus group discussions with ASWs and health care workers (HCWs) were conducted in each clinic. Key informant interviews in the ART clinics were also conducted. There was a marked shift of workload from HCWs to ASWs without any compromise in the quality of counseling. Quality of adherence counseling by ASWs was comparable to HCWs after their introduction. The findings suggest that the deployment of ASWs helped reduce waiting times for adherence counseling. Loss to follow-up rates of new clients declined from 15% to 0% after the deployment of ASWs.

Conclusion

Adherence counseling tasks can be shifted to lay cadres like ASWs without compromising the quality of counseling. Follow-up of clients by ASWs within the community is necessary to improve retention of clients on ART.     

Mapping the Geographical Distribution of Lymphatic Filariasis in Zambia

Background

Past case reports have indicated that lymphatic filariasis (LF) occurs in Zambia, but knowledge about its geographical distribution and prevalence pattern, and the underlying potential environmental drivers, has been limited. As a background for planning and implementation of control, a country-wide mapping survey was undertaken between 2003 and 2011. Here the mapping activities are outlined, the findings across the numerous survey sites are presented, and the ecological requirements of the LF distribution are explored.

Methodology/Principal findings

Approximately 10,000 adult volunteers from 108 geo-referenced survey sites across Zambia were examined for circulating filarial antigens (CFA) with rapid format ICT cards, and a map indicating the distribution of CFA prevalences in Zambia was prepared. 78% of survey sites had CFA positive cases, with prevalences ranging between 1% and 54%. Most positive survey sites had low prevalence, but six foci with more than 15% prevalence were identified. The observed geographical variation in prevalence pattern was examined in more detail using a species distribution modeling approach to explore environmental requirements for parasite presence, and to predict potential suitable habitats over unsurveyed areas. Of note, areas associated with human modification of the landscape appeared to play an important role for the general presence of LF, whereas temperature (measured as averaged seasonal land surface temperature) seemed to be an important determinant of medium-high prevalence levels.

Conclusions/significance

LF was found to be surprisingly widespread in Zambia, although in most places with low prevalence. The produced maps and the identified environmental correlates of LF infection will provide useful guidance for planning and start-up of geographically targeted and cost-effective LF control in Zambia.     

Influence of river geomorphologic features on hippopotamus density distribution along the Luangwa River,Zambia

River geomorphologic features along a 165‐km stretch of the Luangwa River in eastern Zambia were assessed in 2008 to determine their influence on hippopotamus pattern of density distribution. A fixed high wing Cessna 206 aircraft fitted with geographical positioning system (GPS) was flown over the stretch taking coordinates for each geomorphologic feature, followed by taking length in metres of each geomorphologic feature on the ground. A total hippopotamus count was later carried out along the same stretch. Sixty‐four geomorphologic features were recorded along the stretch of the river. Of the total geomorphologic features recorded, 34 (53%) were river bends, 13 (20%) lagoons and 17 (27%) were confluences. Hippo density was highest in river bends (75.48%), confluences (9.18%), lagoons 6.03% and all the other features put together had only 9.38%. It was concluded that geomorphologic features influenced hippopotamus pattern of density distribution in the Luangwa River.