Auditing audits: use and development of the Oxfordshire Medical Audit Advisory Group rating system.

OBJECTIVES--To assess the value of the Oxfordshire Medical Audit Advisory Group rating system in monitoring and stimulating audit activity, and to implement a development of the system. DESIGN--Use of the rating system for assessment of practice audits on three annual visits in Oxfordshire; development and use of an "audit grid" as a refinement of the system; questionnaire to all medical audit advisory groups in England and Wales. SETTING--All 85 general practices in Oxfordshire; all 95 medical audit advisory groups in England and Wales. MAIN OUTCOME MEASURES--Level of practices'' audit activity as measured by rating scale and grid. Use of scale nationally together with perceptions of strengths and weaknesses as perceived by chairs of medical audit advisory groups. RESULTS--After one year Oxfordshire practices more than attained the target standards set in 1991, with 72% doing audit involving setting target standards or implementing change; by 1993 this had risen to 78%. Most audits were confined to chronic disease management, preventive care, and appointments. 38 of 92 medical audit advisory groups used the Oxfordshire group''s rating scale. Its main weaknesses were insensitivity in assessing the quality of audits and failure to measure team involvement. CONCLUSIONS--The rating system is effective educationally in helping practices improve and summatively for providing feedback to family health service authorities. The grid showed up weakness in the breadth of audit topics studied. IMPLICATIONS AND ACTION--Oxfordshire practices achieved targets set for 1991-2 but need to broaden the scope of their audits and the topics studied. The advisory group''s targets for 1994-5 are for 50% of practices to achieve an audit in each of the areas of clinical care, access, communication, and professional values and for 80% of audits to include setting targets or implementing change.     

Cervical cytology in the Vale of Trent faculty of the Royal College of General Practitioners, 1985-8.

OBJECTIVE--To assess changes in the organisation and performance of cervical cytology programmes in the practices of members of the Vale of Trent faculty of the Royal College of General Practitioners. DESIGN--Retrospective audits completed in 1985 and 1988 by general practitioners on a sequential sample of 100 records of women aged 35-64 in their practice. SETTING--General practices in which one or more partners were members or associates of the Vale of Trent faculty of the college, of which 76 participated in the first audit and 55 (82% of 67 eligible practices) in the second. SUBJECTS--Sequential samples of 100 women born between 1 January 1920 and 31 December 1949 (first audit) and between 1 January 1923 and 31 December 1952 (second audit) whose surnames began with P or B respectively. MAIN OUTCOME MEASURES--Percentage of women in each practice with a record of cervical cytology performed in the previous five years and percentage with no recorded smear. RESULTS--Of the 76 practices completing the audit in 1985, 55 (82% of those eligible) repeated the exercise in 1988. Performance was not significantly different in practices that did and did not respond. The median percentage of women who had had a smear in the previous five years was 49% and 69% in 1985 and 1988 respectively (p less than 0.001) and that of women with no record of a cervical smear was 28% and 16% respectively (p less than 0.001). All but six practices showed improvement in both outcome measures. In both audits an active call system was associated with a significantly increased performance (p less than 0.05). In nine practices (16%) 80% or more of the samples of women had had a smear in the previous five years. CONCLUSION--Organisation and performance of practices audited improved between 1985 and 1988. Although this might result from participation in the first audit, it probably represents a more general trend within primary care. ACTION--Between the two audits more practices (87% v 67%) had developed a policy on screening, and this was more likely to include the aim of performing regular smears on all sexually active women (98% v 80%).     

Where have all the pennies gone? The work of Manchester Medical Audit Advisory Group.

Medical audit has its critics, who point to the large sums of NHS cash that seem to be disappearing down a medical plughole. These criticisms are recognised by medical audit advisory groups but there are many reasons why the work of these groups has not yet resulted in many publications in journals or bumped up health indicators. After discussing the criticism this article describes the work of the medical audit advisory group in Manchester. Real changes in cooperative working with general practice teams and between practices are taking place, and improved relationships between general practice and the hospitals are being helped by joint audit work. The Manchester group is also working to help in setting standards and to cooperate with purchasing. The work of the group is changing as it develops.     

Clinical audits in a postgraduate general practice training program: an evaluation of 8 years' experience

Background

Clinical audit can be of valuable assistance to any program which aims to improve the quality of health care and its delivery. Yet without a coherent strategy aimed at evaluating audits'' effectiveness, valuable opportunities will be overlooked. Clinical audit projects are required as a part of the formative assessment of trainees in the Family Medicine Residency Program (FMRP) in Kuwait. This study was undertaken to draw a picture of trainees'' understanding of the audit project with attention to the knowledge of audit theory and its educational significance and scrutinize the difficulties confronted during the experience.

Methodology/Principal Findings

The materials included the records of 133 audits carried out by trainees and 165 post course questionnaires carried out between 2004 and 2011. They were reviewed and analyzed. The majority of audit projects were performed on diabetic (44.4%) and hypertensive (38.3%) care. Regarding audits done on diabetic care, they were carried out to assess doctors'' awareness about screening for smoking status (8.6%), microalbuminuria (19.3%), hemoglobin A1c (15.5%), retinopathy (10.3%), dyslipidemia (15.8%), peripheral neuropathy (8.8%), and other problems (21.7%). As for audits concerning hypertensive care, they were carried out to assess doctors'' awareness about screening for smoking status (38.0%), obesity (26.0%), dyslipidemia (12.0%), microalbuminuria (10.0%) and other problems (14.0%). More than half the participants (68.48%) who attended the audit course stated that they ‘definitely agreed’ about understanding the meaning of clinical audit. Most of them (75.8%) ‘definitely agreed’ about realizing the importance of clinical audit in improving patients'' care. About half (49.7%) of them ‘agreed’ that they can distinguish between ‘criteria’ and ‘standards’.

Conclusion

The eight years of experience were beneficial. Trainees showed a good understanding of the idea behind auditing the services provided. They demonstrated their ability to improve the care given in health centers in which these projects were undertaken.     

Family doctors and change in practice strategy since 1986.

OBJECTIVES--To investigate the changes in practice strategy that have taken place since 1986. DESIGN--Comparison of practices in 1986 and 1992. SETTING--93% of group practices (26 practices) in a single family health services authority. MAIN OUTCOME MEASURES--Changes in staffing, premises, equipment, clinic services, and incomes between 1986 and 1992. RESULTS--In 1986, 28% of practices employed a nurse; in 1992, 92% did so. Between 1986 and 1992, 14 cost-rent schemes costing more than 10,000 pounds had been started. Certain practices, designated innovators, were more likely to possess specified items of equipment than other practices. Computer ownership was widespread: 77% of practices had a computer, compared with 36% in 1986. In 1992, 16 practices had a manager, compared with 10 in 1986. Clinic services provided by more than half of practices were well established services (antenatal, for example), new services for which a payment had been introduced (such as diabetes, asthma, minor surgery), or the more readily provided "new" clinic services (diet, smoking cessation). Gross income increased, but so did practice costs, especially for innovators. Practices in the more affluent area of the family health services authority were still more likely to invest in their premises and staff, and to provide more services than those in the declining area. In the more affluent area, practices had higher costs but also higher incomes. CONCLUSION--Between 1986 and 1992, practices in this area invested heavily in equipment and services, but differences remain, depending on the location of the practice. Investment has increased, particularly in the more deprived part of the area, so that the inconsistency in standards has been much reduced. Practice incomes have risen, but so also have workload and costs.     

Use of unsupervised online quizzes as formative assessment in a medical physiology course: effects of incentives on student participation and performance

Online quizzes were introduced into a large Medical Physiology class to provide students with formative assessment before midterm and final summative examinations. Use of unsupervised online quizzes was chosen to provide a flexible supplementary learning tool for students without overwhelming a small faculty. Several quiz models were applied, which varied in the availability of course credit points for participation and performance. The aims of the study were to investigate if participation in formative assessment was associated with improved course outcomes, if offering incentives for completing quizzes affected student participation, and if quiz performance was predictive of summative examination outcomes. Results showed that students who elected to use online quizzes performed better in summative examinations. Offering course credit of between 0.5% and 2% per quiz increased student participation. However, evidence was found for widespread inappropriate use of unsupervised online quizzes when incentives for participation were applied. Predictive validity of online quizzes could be demonstrated when comparing the first of several quiz attempts with subsequent summative examination scores.     

Collecting morbidity data in general practice: the Somerset morbidity project.

OBJECTIVE--To collect a valid, complete, continuous, and representative database of morbidity presenting to primary care and to use the data to help commission services on the basis of local need and effectiveness. SETTING--Computerised general practices in Somerset. METHODS--Participating general practices were selected to be representative of the district health authority population for general practice and population characteristics. All conditions presented at face to face consultations were assigned a Read code and episode type and the data were regularly validated. Data were sent by modem from the practices via a third party to the health authority each week. MAIN OUTCOME MEASURES--Proportion of consultations coded and accuracy of coding. RESULTS--11 practices agreed to participate. Validations for completeness during April 1994 to March 1995 revealed that 96.4% of the records were coded; 94% of the 1090 records validated had appropriate episode types and 87% appropriate Read codes. The results have been used to help formulate the health authority''s purchasing plans and have enabled a change in the local contracts for surgery for glue ear. CONCLUSIONS--The project has shown the feasibility of establishing a network of practices recording and reporting the morbidity seen in primary care. Early indications are that the data can be useful in evidence based purchasing.     

Counsellors in English and Welsh general practices: their nature and distribution.

OBJECTIVE--To establish the prevalence of counselling services in English and Welsh general practices and factors associated with their distribution; to describe qualifications, working arrangements, and case mix of "counsellors." DESIGN--Postal questionnaire and telephone interview survey of a sample of about one in 20 general practitioners in England and Wales. SETTING--English and Welsh general practices. SUBJECTS--1880 general practitioners of whom 1542 (82%) completed questionnaires. MAIN OUTCOME MEASURES--Prevalence and distribution of practice counselling services; counsellors'' qualifications and funding; types of patients referred. RESULTS--586 counsellors were distributed among 484 of the 1542 practices. Three types of counsellor predominated: community psychiatric nurses (187); "practice counsellors" (145); and clinical psychologists (95). Practice characteristics which independently predicted the presence of a counsellor were for community psychiatric nurses four or more partners (odds = 1.72, 95% confidence interval 1.18 to 2.26); for practice counsellors stress clinic (odds = 2.22; 1.83 to 2.61), training practice (odds = 1.70; 1.24 to 2.16), and health region (chi 2 = 55.94; df = 14; p < 0.001); and for clinical psychologists list size of > or = 10,500 (odds = 1.79; 1.09 to 2.49), training practice (odds = 1.78; 1.31 to 2.25), health region (chi 2 = 48.31; df = 14; p < 0.001). 197 counsellors had training in counselling. The qualifications of 85 were unknown to the general practitioner. The principal source of funding was the district health authority for community psychiatric nurses (150) and clinical psychologists (58) and the family health services authority for practice counsellors (76). All counsellors were referred a wide range of problems. CONCLUSIONS--Counselling services are wide-spread in general practice, but a high proportion of counsellors lack qualifications, and many may be referred problems outside their knowledge.     

Audit of the Medical Audit Committee.

We reviewed 39 medical audits started in one institution between 1981 and 1985 to determine whether they monitored the quality of health care in a scientific manner, whether recommendations were made that could improve the quality of health care and, if so, whether the recommendations were acted on. Thirty-three audits (85%) were completed; 21 (64%) failed to state their objectives, 30 (91%) failed to compare their results with those in the literature, 9 (27%) made no recommendations that could improve the quality of health care, and 9 were poorly researched and written. The number of patients in each audit was usually adequate. Feedback was rarely received by the Medical Audit Committee concerning the Medical Advisory Committee''s response to the audit. Information concerning the implementation of recommendations was available for 17 of the 24 audits that made recommendations; 7 (41%) had failed to implement half or more of the recommendations. Despite these problems, 8 (24%) of the audits were considered to be of a quality that could improve health care. We present recommendations to improve the audit procedure and foster the growing confidence among doctors in the medical audit.     

General practitioners' views about the statutory annual practice report.

OBJECTIVE--To ascertain the views of primary care professionals about the current purpose, uses, potential, and workload implications of the statutory general practice annual report. DESIGN--Postal questionnaire survey. SETTING--General practices in the Northern region. SUBJECTS--All practices in the region that were singlehanded, fundholding, non-fundholding and with more than five partners, and a one in three random sample of all non-fundholding practices (n = 318). RESULTS--263 practices responded (83%). The report took a median of 12 hours to produce (95% confidence interval 11 to 15 hours; interquartile range 7-35). The main perceived purpose of the report was to monitor practice activity (165 respondents; 63% (95% confidence interval 57% to 69%)), but 44 respondents (17%; 13% to 22%) produced it only because it was contractually required. Practices included statutory and non-statutory data in these reports and would have liked comparative practice activity information (155 respondents; 59%) and "good ideas" (165 respondents; 63%) fed back to them. Respondents would have liked the annual report used to improve practice development planning (122 respondents; 46% (40% to 52%)), to facilitate audit (115 respondents; 44% (38% to 50%)), and to influence resource allocation (104 respondents; 40% (34% to 46%)). One hundred and eighteen practices (45%; 39% to 51%) would produce an annual report even if not contractually required. Data collected were perceived to be already available elsewhere. CONCLUSIONS--Primary care professionals have concerns about the current annual report. They would prefer to collect relevant, standardised data which could lead to better audit, planning, and resource allocation.     

Health checks on patients 75 years and over in Nottinghamshire after the new GP contract.

OBJECTIVE--To investigate annual health checks for patients of 75 years and over required by the 1990 contract for general practitioners. DESIGN--Visits to practices to collect information on how assessments were organised and carried out; completion of questionnaires for every patient who had been assessed in a sample month, using information provided by the practice records. SETTING--20 general practices in one family health services authority. SUBJECTS--Patients of 75 years and over in 20 general practices. RESULTS--Three practices (15%) had not performed checks. Thirteen practices sent a letter to invite patients to undergo a check. Of these practices, seven followed up non-responders. Two practices visited patients'' homes unannounced, and two did checks on an opportunistic basis only. Sixteen practices used a checklist. Sixteen practices involved their practice nurses; at eight of these, doctors also performed checks; in six practices the nurses undertaking the checks had no training in assessing old people. Ten practices assessed more than 75% of their old people in the first year of the new contract. Practices that did not follow up patients who had not responded to the invitation for assessment completed significantly fewer checks. During the sample month, 331 patients were assessed in the 17 practices. 204 new problems were discovered in 143 patients. Significantly more problems per patient were found in inner city areas. CONCLUSIONS--The way health checks were performed varied greatly, both in their organisation and the practices'' attitudes. Many old people did not respond to letters asking if they wanted an assessment but very few refused one if followed up. Forty three per cent of those assessed had some unmet need. The number of new problems found per patient may reduce over the next few years if the assessments are successful. The need for annual assessment should be kept under review and adequate resources made available for the needs uncovered. Improved training for practice nurses in assessment is needed. Effectiveness of the checks must be monitored. If most unmet need falls in particular high risk groups it would seem sensible to modify the annual check to target these groups.     

Survey of general practice audit in Leeds.

OBJECTIVE--To determine general practitioners'' attitudes to medical audit and to establish what initiatives are already being undertaken; to define future ideas for audit and perceived difficulties in implementing audit in primary care. DESIGN--Analysis of responses to a self administered postal questionnaire. SETTING--Urban conurbation with a population of about 750,000. PARTICIPANTS--386 general practitioners on the general medical list of Leeds Family Practitioner Committee. MAIN OUTCOME MEASURES--Extent of recording of practice activity data and outcome measures and clinical data, use of data, and audit performed; ideas for audit and perceived difficulties. RESULTS--317 doctors responded to the questionnaire (individual response rate 82%) from 121 practices (practice response rate 88%). In all, 206 doctors thought that audit could improve the quality of care; 292 collected practice activity data, though 143 of them did not use it. A total of 111 doctors recorded some outcome measures, though half of them did not use them. Varying proportions of doctors had registers, for various diseases (136 had at least one register), disease management policies (60 doctors), and prescribing policies. In all, 184 doctors met monthly with other members of the primary health care team. CONCLUSIONS--Much poorly focused data collection is taking place. Some doctors have experience in setting up basic information systems and practice policies, and some audit is being performed. The family health services authorities need to take seriously the perceived difficulties of time, organisation, and resources concerned with audit.     

Using patient and general practice characteristics to explain variations in cervical smear uptake rates.

OBJECTIVES--To produce practice and patient variables for general practices from census and family health services authority data, and to determine the importance of these variables in explaining variation in cervical smear uptake rates between practices. DESIGN--Population based study examining variations in cervical smear uptake rates among 126 general practices using routine data. SETTING--Merton, Sutton, and Wandsworth Family Health Services Authority, which covers parts of inner and outer London. MAIN OUTCOME MEASURE--Percentage of women aged 25-64 years registered with a general practitioner who had undergone a cervical smear test during the five and a half years preceding 31 March 1992. RESULTS--Cervical smear uptake rates varied from 16.5% to 94.1%. The estimated percentage of practice population from ethnic minority groups correlated negatively with uptake rates (r = -0.42), as did variables associated with social deprivation such as overcrowding (r = -0.42), not owning a car (r = -0.41), and unemployment (r = -0.40). Percentage of practice population under 5 years of age correlated positively with uptake rate (r = 0.42). Rates were higher in practices with a female partner than in those without (66.6% v 49.1%; difference 17.5% (95% confidence interval 10.5% to 24.5%)), and in computerised than in non-computerised practices (64.5% v 50.5%; 14.0% (6.4% to 21.6%)). Rates were higher in larger practices. In a stepwise multiple regression model that explained 52% of variation, five factors were significant predictors of uptake rates: presence of a female partner; children under 5; overcrowding; number of women aged 35-44 as percentage of all women aged 25-64; change of address in past year. CONCLUSIONS--Over half of variation in cervical smear uptake rates can be explained by patient and practice variables derived from census and family health services authority data; these variables may have a role in explaining variations in performance of general practices and in producing adjusted measures of practice performance. Practices with a female partner had substantially higher uptake rates.     

New roles for general practitioners.

General practice is likely to change greatly over the next few years. Increases in care in the community and day surgery will lead to more work, and the demand for better data on practice activity will mean the development of audit and epidemiological work. To make time general practitioners will have to learn to delegate work that does not require a doctor. Fundholding has already stimulated some practices to bring services to patients rather than send patients to hospital, and this trend seems set to continue. It is important to pool resources, not only within practices but among other practices in the area--joint action will increase the ability to improve the services for patients. If general practitioners take the opportunity to gain control of the changes the morale of the profession should improve.     

Informal complaints procedure in general practice: first year's experience.

OBJECTIVE--To evaluate the first year''s experience of an informal patient complaints system that encourages extensive patient participation. DESIGN--Audit of an informal complaints procedure. SETTING--The Marylebone Health Centre, London. SUBJECTS--39 complaints received over the audit period. MAIN OUTCOME MEASURES--Types of complaints (administrative, about doctors or medical care or both, staff about patients, mixed, other) and resolution of complaints (how complaints were dealt with and their resolution). RESULTS--37 of the 39 complaints were resolved within two weeks. Two complaints sent direct to the family health services authority were resolved (with patients'' agreement) by the informal complaints procedure. CONCLUSIONS--The informal complaints procedure was more cost effective than the family health services authority system and was comparatively straightforward to implement within the practice without major organisational restructuring. The two way process of the procedure ensured patients received a quick response to complaints and helped morale of health centre staff.     

An external dosimetry audit programme to credential static and rotational IMRT delivery for clinical trials quality assurance

PurposeExternal dosimetry audits give confidence in the safe and accurate delivery of radiotherapy. The RTTQA group have performed an on-site audit programme for trial recruiting centres, who have recently implemented static or rotational IMRT, and those with major changes to planning or delivery systems.MethodsMeasurements of reference beam output were performed by the host centre, and by the auditor using independent equipment. Verification of clinical plans was performed using the ArcCheck helical diode array.ResultsA total of 54 measurement sessions were performed between May 2014 and June 2016 at 28 UK institutions, reflecting the different combinations of planning and delivery systems used at each institution. Average ratio of measured output between auditor and host was 1.002 ± 0.006. Average point dose agreement for clinical plans was −0.3 ± 1.8%. Average (and 95% lower confidence intervals) of gamma pass rates at 2%/2 mm, 3%/2 mm and 3%/3 mm respectively were: 92% (80%), 96% (90%) and 98% (94%). Moderately significant differences were seen between fixed gantry angle and rotational IMRT, and between combination of planning systems and linac manufacturer, but not between anatomical treatment site or beam energy.ConclusionAn external audit programme has been implemented for universal and efficient credentialing of IMRT treatments in clinical trials. Good agreement was found between measured and expected doses, with few outliers, leading to a simple table of optimal and mandatory tolerances for approval of dosimetry audit results. Feedback was given to some centres leading to improved clinical practice.     

Family doctors and innovation in general practice

Family doctors have been presented with changes in government policies and incentives in a recent white paper on primary care. Little work has been done, however, to find out how general practitioners respond to such measures. The response of general practitioners to professional and economic incentives was examined in relation to the location of the practice and the characteristics of the practitioners in seven different areas of England. The areas represented urban, rural, affluent, and deprived communities. The overall response rate was 74%, but the response varied among the areas, being poorest (64%) in an inner city area. Practices were subdivided as innovative, traditional, or intermediate, according to whether they employed a nurse and participated in the cost rent scheme and the vocational training scheme. Innovative practices were defined as fulfilling two of these criteria and traditional practices as fulfilling none; the remainder were classed as intermediate. The results showed that these three types of practice had distinct strategies that were related to financial constraints and the local population. Innovative practices had more partners and were often located in rural or affluent suburban areas; traditional practices had fewer partners and were more common in urban and working class areas. Innovative practices seemed to be in the best position to increase their services, and hence their incomes, in response to the recent proposals in the white paper. Practices in areas of developmental difficulty (predominantly urban but not necessarily inner city areas) had been less able to respond to existing incentives and had a smaller margin available for developing their services.In view of the effect of local constraints of economics and population on the strategy of practices, concentrating resources for primary care in local budgets for working class and urban areas may be preferable to extending the system of charging fees for services provided by family doctors.     

Use of vitamin B12 injections among elderly patients by primary care practitioners in Ontario

BACKGROUND: Excess use of parenteral vitamin B12 has been reported from audits of clinical practices. The authors assessed the use of vitamin B12 injections in patients aged 65 years and over in Ontario. METHODS: A cross-sectional analysis was conducted that included all elderly people covered by the Ontario Health Insurance Plan who received insured services from general practitioners or family physicians (GP/FPs). For each practice the proportion of elderly patients who received regular vitamin B12 injections between July 1996 and June 1997 was calculated. The frequency of injections was determined for each patient receiving regular B12 replacement. RESULTS: Of the 1,196,748 elderly patients (mean age 74.8 [standard deviation 6.8], 58.0% female) treated by 14,177 GP/FPs, 23,651 (2.0%) received regular B12 injections. The rate of B12 injections per patient, standardized for age and sex, varied between practices (range 0%-48.6%). Although no authoritative sources support the practice, 3303 (19.8%) of the 16,707 patients receiving long-term parenteral therapy had, on average, overly frequent injections (more than 1 injection every 4 weeks). For 76 (12.3%) of the 617 practices with 10 or more patients receiving regular vitamin B12 injections, the mean injection frequency was greater than once every 4 weeks. The proportion of patients in these 617 practices who received overly frequent injections varied extensively (0%-100%). INTERPRETATION: Our findings indicate that some primary care physicians in Ontario administer unnecessary vitamin B12 injections to elderly patients.