Monitoring standards for molecular genetic testing in the United Kingdom, the Netherlands, and Ireland

Molecular genetic techniques have entered many areas of clinical practice. Public expectations from this technology are understandably high. To maintain confidence in this technology, laboratories must implement the highest standards of quality assurance (QA). External quality assessment (EQA) is recognized as an essential component of QA. The United Kingdom National External Quality Assessment Service (UKNEQAS) for Molecular Genetics, first set up in 1991, is currently the longest provider of EQA to molecular genetic testing laboratories in the UK, The Netherlands, and Ireland. Errors in the scheme are sporadic events. However, evidence from this and other EQA schemes suggests that a residual error rate persists, which should be taken into account in clinical practice. This EQA scheme has evolved from the respective scientific bodies of the constituent countries and retains a strong emphasis on collective peer review. It is essential that the steps taken to ensure quality in this rapidly expanding field are clear and transparent to participants and public alike. We describe the procedures developed and the governance imposed to monitor and improve analytical and reporting standards in participant laboratories and we compare our experiences with those of equivalent EQA services in the United States.     

Cervical cytology quality assurance in Washington State.

The objectives of this study were to establish a profile of cervical cytology laboratories in Washington State, identify quality assurance problems amenable to correction through education or legislation, and describe differences between large and small cytology laboratories. All 43 Washington laboratories that perform cervical cytology were surveyed by mail during 1989. Completed surveys were returned by 37 (86%) of the laboratories. Nearly half (43%) of the respondents reported processing less than 10,000 Papanicolaou smears annually. Only one-third (35%) of the respondents reported participating in relevant proficiency programs. A proportion of smaller cytology laboratories were compensating their cytotechnologists on the basis of the number of slides read and allowing Papanicolaou smears to be read outside the confines of the laboratory. The results of this study suggest that cytotechnologists in some larger Washington laboratories have been exceeding work load limits recommended by professional associations. Recent legislation includes regulations that address cervical cytology quality assurance. However, continued efforts will need to be made to encourage voluntary adoption of quality control measures not addressed by this legislation.     

Development of a technical external quality assurance scheme in non‐gynaecological cytology in UK

Technical external quality assurance (EQA) schemes are well established for histopathology and cervical cytology but, to date, sadly lacking for diagnostic cytology (DC). This timely review redresses the balance by describing the development and evaluation of a technical EQA scheme for DC available to the UK, Europe and beyond.     

Personal commentaryExternal quality assessment (EQA) in gynaecological cytology: radical rethinking required?

Accepted for publication 19 February 1999
THE PAST
The value of EQA in pathology, including proficiency testing in gynaecological cytology, has been the subject of much recent lively debate^1–6. In summary, the proven role of EQA in the monitoring of personal performance and ability is still uncertain. Furthermore, its potential to prevent critical incidents remains purely speculative. Also, whether EQA would be better replaced, for some or all professional groups, by other methods of quality assurance is unknown. Similarly, whether EQA can justify the considerable professional time, effort and expense involved has not been resolved.  

THE PRESENT

Not surprisingly, events at Kent & Canterbury Hospitals abruptly halted much of the debate about the foregoing issues⁷. Indeed, as a direct consequence of this and other incidents, the NHS Executive stipulated unilaterally that EQA is now mandatory within the NHS Cervical Screening Programme (CSP)⁸ and reinforced the necessity for all qualified laboratory staff to participate^8–10. A surprise, however, was the Executive's enlightened comments on EQA. First, and of greatest significance, that the principal function of EQA in pathology is to improve standards and advance quality through personal education. Second, that EQA should be seen to complement other QA systems for the early identification of potential problems which might affect patient care. Third, that the identification of individual poor performance through EQA will be exceptional and in essence is a by-product of the basic educational exercise. The latter conclusion was drawn, as far as one can judge, from previous experience with the NHS Breast Screening Programme (NHSBSP) EQA.  

THE FUTURE—PERSONAL SUGGESTIONS FOR OPEN DEBATE

     

External Quality Assessment for Tuberculosis Diagnosis and Drug Resistance in the European Union: A Five Year Multicentre Implementation Study

BackgroundExternal quality assurance (EQA) systems are essential to ensure accurate diagnosis of TB and drug-resistant TB. The implementation of EQA through organising regular EQA rounds and identification of training needs is one of the key activities of the European TB reference laboratory network (ERLTB-Net). The aim of this study was to analyse the results of the EQA rounds in a systematic manner and to identify potential benefits as well as common problems encountered by the participants.MethodsThe ERLTB-Net developed seven EQA modules to test laboratories’ proficiency for TB detection and drug susceptibility testing using both conventional and rapid molecular tools. All National TB Reference laboratories in the European Union and European Economic Area (EU/EEA) Member States were invited to participate in the EQA scheme.ResultsA total of 32 National TB Reference laboratories participated in six EQA rounds conducted in 2010–2014. The participation rate ranged from 52.9% - 94.1% over different modules and rounds. Overall, laboratories demonstrated very good proficiency proving their ability to diagnose TB and drug-resistant TB with high accuracy in a timely manner. A small number of laboratories encountered problems with identification of specific Non-tuberculous Mycobacteria (NTMs) (N = 5) and drug susceptibility testing to Pyrazinamide, Amikacin, Capreomycin, and Ethambutol (N = 4).ConclusionsThe European TB Reference laboratories showed a steady and high level of performance in the six EQA rounds. A network such as ERLTB-Net can be instrumental in developing and implementing EQA and in establishing collaboration between laboratories to improve the diagnosis of TB in the EU/EEA.     

Decentralization of Acid Fast Bacilli(AFB) External Quality Assurance Using Blind Rechecking for Sputum Smear Microscopy in Ethiopia

Introduction

Ethiopia achieved a rapid expansion of TB microscopic centers for acid fast bacilli (AFB). However, external quality assurance (EQA) services were, until recently, limited to few regional and sub-regional laboratories. In this paper, we describe the decentralization experience and the result of EQA using random blinded rechecking.

Materials and Methods

The routine EQA quarterly report was compiled and analyzed. A positive result by the microscopic center while the EQA center reported negative result is categorized as false positive (FP). A negative result by the microscopic center while the EQA center reported positive is considered false negative (FN). The reading of EQA centers was considered a gold standard to compute the sensitivity, specificity, positive predictive (PPV) and negative predictive values (NPV) of the readings of microscopic centers.

Results

We decentralized sputum smear AFB EQA from 4 Regional Laboratories (RRLs) to 82 EQA centers and enrolled 956 health facilities in EQA schemes. Enrollment of HFs in EQA was gradual because it required training and mentoring laboratory professionals, institutionalizing internal QA measures, equipping all HFs to perform diagnosis, and establishing more EQA centers. From 2012 to 2014 (Phase I), the FP rate declined from 0.6% to 0.2% and FN fell from as high as 7.6% to 1.6% in supported health facilities (HFs). In HFs that joined in Phase II, FN rates ranged from 5.6 to 7.3%. The proportion of HFs without errors has increased from 77.9% to 90.5% in Phase I HFs and from 82.9% to 86.9% in Phase II HFs. Overall sensitivity and specificity were 95.0% and 99.7%, respectively. PPV and NPV were 93.3% and 99.7%, respectively.

Conclusion

Decentralizing blinded rechecking of sputum smear microscopy is feasible in low-income settings. While a comprehensive laboratory improvement strategy enhanced the quality of microscopy, laboratory professionals’ capacity in slide reading and smear quality requires continued support.     

Statistical diagnostics emerging from external quality control of real-time PCR

Besides the application of conventional qualitative PCR as a valuable tool to enrich or identify specific sequences of nucleic acids, a new revolutionary technique for quantitative PCR determination has been introduced recently. It is based on real-time detection of PCR products revealed as a homogeneous accumulating signal generated by specific dyes. However, as far as we know, the influence of the variability of this technique on the reliability of the quantitative assay has not been thoroughly investigated. A national program of external quality assurance (EQA) for real-time PCR determination involving 42 Italian laboratories has been developed to assess the analytical performance of real-time PCR procedures. Participants were asked to perform a conventional experiment based on the use of an external reference curve (standard curve) for real-time detection of three cDNA samples with different concentrations of a specific target. In this paper the main analytical features of the standard curve have been investigated in an attempt to produce statistical diagnostics emerging from external quality control. Specific control charts were drawn to help biochemists take technical decisions aimed at improving the performance of their laboratories. Overall, our results indicated a subset of seven laboratories whose performance appeared to be markedly outside the limits for at least one of the standard curve features investigated. Our findings suggest the usefulness of the approach presented here for monitoring the heterogeneity of results produced by different laboratories and for selecting those laboratories that need technical advice on their performance.     

临床检验前阶段室间质量评价方案的设计

在检验医学领域中,有很多研究已描述检验全过程不同阶段出现的最频繁的差错,且这些差错的很大部分都出现在检验前阶段。检验中阶段的误差登记然后反馈给参加者的方案已经由室间质评（External Quality Assessment,EQA）组织在大多数国家进行了数十年。但是迄今为止,只有很少的组织专注于检验前阶段,而且大多数的EQA组织并不提供检验前EQA方案（External Quality Assessment Schemes,EQAS）。执行和标准化检验前EQAS是比较困难的,认可机构也并没有对实验室参加这类方案提出要求。然而,一些正在进行的检验前阶段EQA计划是存在的,使用的方法可以分为三种类型：收集实验室检验前程序的信息、发放真实样品来收集可能影响测量程序的干扰因素、或者登记真实的实验室差错并将这些内容关联到质量指标。这三个类型有不同的侧重点和不同的实施挑战,这三者的结合可能是检出和监视出现在检验前阶段的广泛差错所必须的。     

Tissue-specific differences in metabolites and transcripts contribute to the heterogeneity of ricinoleic acid accumulation in Ricinus communis L. (castor) seeds

Metabolomics - Up to now, quality assurance (QA) and quality control (QC) in metabolomics are procedures that most labs did using their own in-house developed procedures and rules since there was...     

O-12The use of control samples in the PAPANICOLAOU technical external quality assessment scheme

Technical external quality assessment (TEQA) in Wales is based on NHSCSP publication 19, which sets out policy and procedures for the scheme. The purpose of EQA is to sustain and improve the quality of patient care by promoting a high standard of performance. Following the introduction of liquid base cytology (LBC) technical limitations, particularly in assessing counterstaining, have been noted. LBC provides the means to address these limitations - As part of a development plan for TEQA in Wales, a control sample procedure was introduced to the scheme. A pooled control sample was composed, containing residual material from six 'matched' negative samples, re-suspended in collection fluid. Aliquots of this sample were distributed for processing and staining, to the 13 laboratories registered with the scheme. The slides produced were assessed at a scheduled TEQA assessment in accordance with the standard criteria. Initial overall scoring for these control samples produced acceptable levels of staining for 12 of the laboratories - one laboratory produced a marginal score. Repeat distributions have shown maintained or improved results. This method provides a prospective quality assessment tool, which counters the emphasis on slide selection and eliminates potential selection bias, whilst introducing consistency and improving comparability across participant laboratories. The method may also prove helpful in identifying technical inconsistencies such as equipment or handling errors that may occur during sample processing prior to or during staining. The control process, which is now used routinely in the Welsh TEQA scheme; is considered complimentary to, and not a replacement for, the selection process established in NHSCSP #19. However, we feel that this development could be considered as a new initiative in the National TEQA scheme. The control process is applicable to all LBC systems in current use.     

Toward implementation of a regional quality assurance program in cytopathology: the Hong Kong experience

OBJECTIVE: To develop a local quality assurance program in cytopathology based on circulation of patient specimens on glass slides, with limited resources. STUDY DESIGN: A working group was set up for design and running of the program. Participation is on a laboratory basis. The scope and frequency of testing are defined. Well-documented cases (including gynecologic, nongynecologic and fine needle aspiration cytology) with commonly encountered diagnoses are collected. Consensus concerning the diagnosis, interpretive menu and scoring system is sought before the actual slide circulations using express mail. After returning their answers to the program organizer, the participating laboratories receive immediate feedback on their scores, with reference answers, explanatory notes, "whole-mount" images of glass slides and cumulative responses of peer laboratories for on-site checking. At the end of each year, an electronic file containing representative photomicrographs of all cases examined is provided to individual laboratories for their permanent records and training purposes. RESULTS: The program was launched in mid-2003. There were 24 and 27 participating laboratories from Hong Kong (and Macau) in 2003 and 2004, respectively. To date, >150 well-documented cytology cases are available in the slide pool and ready for circulation. As the revenue is mainly to cover the expenses of express mail, the program can be carried out at a relatively low cost. CONCLUSION: In order to have any cytology quality assurance program accepted by local laboratories, it has to be fair and practical. Strict confidentiality needs to be observed throughout the process. This program emphasizes both performance assessment and educational value. Adequate representation from experienced local cytology workers, detailed documentation support from authorities and assistance from dedicated staff are essential to the success of any external proficiency testing scheme. Regular review and evaluation are also necessary for continuous improvement. The Hong Kong experience can serve as an example of running a glass slide-based cytology quality assurance program in a small region with limited resources.     

Cost-effectiveness of the modifications in the quality assurance system in radiotherapy in the example of in-vivo dosimetry

PurposeTo present the methodology for the evaluation of cost-effectiveness of the quality assurance protocol modifications associated with increasing demands on accuracy and reliability in radiotherapy and to present results on cost-effectiveness of in-vivo dosimetry as the chosen example of a technical procedure.Material and methodsIn-vivo dosimetry was used as an example of a quality assurance procedure, whose modifications have an impact on several procedures in the QA system and thus on the cost of radiotherapy. An analysis of 6864 patients, treated between 2001 and 2005 for tumours in the head and neck, breast, pelvis, or lung, was performed. The quality of radiotherapy was expressed as the accuracy of dose delivery and the cost was estimated from labour, equipment and materials.ResultsModifications implemented in the quality assurance protocol have gradually improved the quality of irradiation. Mean deviations between measured and calculated doses, recorded for several groups of treatment sites, were reduced from ?1.5% to 0.5%, 3.4% to 1.4%, 3.9% to 0.1% and ?2.1% to 1.8% for head and neck, breast, pelvis and lung respectively. The standard deviations of the measured values decreased also consistently. Total monthly cost in radiotherapy (related to in-vivo dosimetry) increased from € 4376 to € 10,696 while the unitary cost of radiotherapy procedures remained at the same level. The predominant cost component of in-vivo dosimetry was labour, limited at first to physics staff and later extended to quality assurance personnel and technicians.ConclusionThe application of the presented methodology revealed cost-effectiveness relationships in tested technical procedures.     

Second external quality assurance study for the serological diagnosis of hantaviruses in Europe

Hantaviruses are endemic throughout the world and hosted by rodents and insectivores. Two human zoonoses, hemorrhagic fever with renal syndrome (HFRS) and hantavirus pulmonary syndrome (HPS), are caused by hantaviruses and case fatality rates have reached 12% for HFRS and 50% for HPS in some outbreaks. Symptomatic hantavirus infections in Europe are summarised as HFRS mainly due to Puumala, Dobrava-Belgrade and Saaremaa virus. While HFRS has an overall low incidence in Europe, the number of cases varies from 100 per year in all Eastern and Southern Europe up to 1,000 per year only in Finland. To assess the quality of hantavirus diagnostics, the European Network for the Diagnostics of "Imported" Viral Diseases (ENIVD) organised a first external quality assurance (EQA) in 2002. The purpose of this second EQA study is to collect updated information on the efficiency and accurateness of hantavirus serological methods applied by expert laboratories. A serum panel of 14 samples was sent to 28 participants in Europe of which 27 sent results. Performance in hantavirus diagnosis varied not only on the method used but also on the laboratories and the subclass of antibodies tested. Commercial and in-house assays performed almost equally. Enzyme immunoassays were mainly used but did not show the best performances while immunoblot assays were the less employed and showed overall better performances. IgM antibodies were not detected in 61% of the positive IgM samples and IgM detection was not performed by 7% of the laboratories indicating a risk of overlooking acute infections in patients. Uneven performances using the same method is indicating that there is still a need for improving testing conditions and standardizing protocols.     

External quality assurance for cervical cytology in developing countries. Experience in Peru and Nicaragua

Given interest from the professionals concerned, an external quality assurance scheme for cervical cytology can successfully be introduced in developing countries. This is a very important precondition if screening programs are to be expanded and decreases in mortality from cervical cancer are to occur in developing countries. Nicaragua and Peru have been experimenting with an external quality assurance system adapted from the Scottish and Northern Ireland scheme. It has been received with enthusiasm and acceptance and has helped cytology laboratories in these countries focusing on quality issues. Nevertheless, a successful quality control scheme that is to result in improvements in the quality of professionals' diagnostic skills needs to be accompanied by a remedial program for subperformers.     

Proficiency testing in cytology laboratories in Ontario, Canada: a decade of experience. I. Introduction and description of the testing model

There are few formally documented proficiency testing programs for cytology laboratories, and those that have been documented are not entirely comparable in format. The first of three papers documenting a mandatory universal proficiency testing program for cytology laboratories in the Province of Ontario, Canada, presents data on the structure and function of the participating laboratories (including a comparison of the data for 1974 and 1980) and on the organization of the testing model (including selection of terminology, construction and use of the survey and assessment of responses). In 1980, of the 463 medical laboratories in Ontario, 91 of 222 hospital laboratories and 65 of 216 nonhospital laboratories were licensed in cytology. In that year, the 156 cytology laboratories processed 1.48 million cytology specimens, 92% of which were gynecologic. Hospital laboratories processed 87.5% of the nongynecologic cytology specimens and 30% of the gynecologic cytology specimens. These proportions have been virtually constant for several years. Between 1974 and 1980, there was a trend in Ontario to fewer laboratories processing less than 5,000 cytology specimens per annum. Subsequent papers in this series describe the results of the initial surveys in this program and a precision study to evaluate the consistency of reporting by individual laboratories.     

External quality assurance of the carcinoembryonic antigen (CEA) assay: main findings in six years' experience.

In 1984 we initiated a national external quality assessment (EQA) program (supported by the Italian National Research Council, CNR) for the CEA assay; at present, about 200 Italian laboratories are participating in the program. The laboratories assayed the quality control (QC) samples according to their routine procedures and returned the results together with the name of the method/kit they used. The collected results were computer-processed and reports were sent back to the participants. A significant reduction of the CVt (mean between-laboratory agreement) of the CEA assay was observed throughout the EQA survey (from 35% in 1985 to 20-25% in the last cycles). In order to better clarify the differences in variability observed in the first QC cycles against the last ones, we used the ANOVA technique to evaluate the components of variability. The improvement in between-laboratory agreement was mainly due to the reduction of the between-kit component (from 30.5% to 15.2%), rather than to the smaller decrease observed for the within-kit variability (from 18.4% to 14.0%). The results reported for QC samples from different materials showed differences in the between-lab variability and substantial changes of the kit biases, thus suggesting a different specificity of the antibodies used in the various method/kits against different families of CEA molecules. Considerable uncertainty was also encountered in the clinical classification of low pathological samples, which seems mainly due to the variability in cut-off values used by the laboratories for the clinical assessment of the same analytical results. Our data indicate a progressive increase in the reliability of CEA determination during our study and confirm that EQA has improved the reliability of analysis carried out by the participating laboratories, thus stimulating the kit manufacturers to provide more reliable products.     

Patient load and medical staffing in adult dialysis units in the United Kingdom.

A survey of medical staffing in 50 adult dialysis units in the United Kingdom in 1986 showed a wide range of patient to staff ratios or staffing score ratios. The total patient load (patients receiving haemodialysis in hospital and at home and those receiving continuous ambulatory peritoneal dialysis) varied from 12 to 270 per unit. Patients receiving acute haemodialysis or who had received a transplant were not included. The unit staffing score, on a weighted scale based on experience, varied from 6.0 to 40.5. Previous surveys have all been regionally or nationally based so criteria for assessing the adequacy of staffing in single units do not exist. This survey attempts to provide a guideline by describing the range of medical staffing compared with patient load in single dialysis units. No unit considered itself to be overstaffed, and several considered themselves to be greatly understaffed. Individual dialysis units should plead their own case in the light of their own circumstances and up to date information provided in nationwide staffing surveys such as this one.     

Quality assurance activities of the College of American Pathologists

Since its inception, the College of American Pathologists (CAP) has played a fundamental and pivotal role in the development and execution of quality assurance programs for laboratories. Within the realm of anatomic pathology, operational programs include those in surgical pathology, immunohistochemistry and cytopathology. The emphasis of prior cytopathology programs on cervical cytology has now been expanded to include body fluids and fine needle aspiration material. CAP's role in the expansion of quality assurance programs in cytology may be enhanced in the future by intersociety cooperation with established cytology organizations and will also be influenced by and closely linked to the expansion of its quality assurance programs in surgical pathology. As for the Papanicolaou smear, it can no longer be regarded as the "Cinderella of cytology"; it is in fact the present-day cynosure in the laboratory. In recognition of this, CAP has undertaken efforts to help heighten public awareness about the value of Papanicolaou smear testing and is encouraging women to become more informed about the process involved in the examination of their smears.     

Results of the National External Quality Assessment for Toxoplasmosis Serological Testing in China

Background

Toxoplasmosis is typically diagnosed by serologic testing. External quality assessment (EQA) of clinical laboratories could ensure the accuracy and reliability of serological tests. We assessed the quality of toxoplasma serological assays in Chinese clinical laboratories by an EQA performed between 2004 and 2013 by the National Center for Clinical Laboratories.

Methodology and Findings

EQA panels were prepared and shipped at room temperature to participating laboratories that employed toxoplasma IgG and IgM serological detection. By 2013, 5,384 EQA test reports for toxoplasma-specific IgM and 2,666 reports for toxoplasma-specific IgG were collected. Enzyme-linked immunosorbent (ELISA) and chemical immunofluorescent assays were the most commonly used detection methods. The overall coincidence rates of negative samples were better than those of positive samples. The overall EQA score for toxoplasma-specific IgM detection ranged between 84.3% and 99.6%. The ratio of laboratories that achieved correct IgG detection ranged from 61.1% to 99.3%. However, the inter- and intra-assay variabilities were found to be considerable. The most common problem was failure to detect low titers of antibody.

Conclusion

The EQA scheme showed an improvement in toxoplasma serological testing in China. However, further optimization of assay sensitivity to detect challenging samples remains a future challenge.