Concepts in the validation of neurochemical methods: the proper generation and use of statistics

Nearly every advance made in the study of neurotransmission has resulted from the development of new analytical methods or from the application of such methods to neurochemical problems in new ways. Each investigator places extremely high dependence on the laboratory method from which the data are gathered. It is therefore vital that these methods be proven valid when first selected and when used throughout the experiments being conducted. The process of method validation has flourished and been refined in the field of laboratory medicine. Methods are primarily validated by their accuracy and reproducibility in determining the analyte of interest in the tissue(s) to be studied. It is important that standards of performance be established that will allow objective decisions to be made when methods are tested for these characteristics. Performance standards for several neurotransmitters are suggested. Studies performed to collect method performance data are presented. From these data, statistics can be generated that help to estimate analytical errors. Guidelines for the proper generation and use of these statistics are discussed. Use of these validation approaches should be expanded in the whole of neurochemistry, which should enrich the data gathered within a laboratory and improve the harmonization between laboratories.     

Molecular genetic testing in the United States: comparison with international practice

OBJECTIVE: To compare data on the practices of molecular genetic testing (MGT) in laboratories in the United States with those in 18 other countries. METHODS: A Web-based survey of MGT laboratory directors (n = 827; response rate 63%) in 18 countries on three continents was carried out, and the response from U.S. laboratories compared to all others. Quality assurance and reporting indices were developed and calculated for each responding laboratory. RESULTS: A comparison of U.S. results with all other countries identified differences in laboratory setting, personnel qualifications, and the specific tests being offered, but similar rates of adherence to MGT quality standards and reporting practices were found. The survey also documented substantial transborder flow of specimens, most commonly due to the lack of availability of the test in the United States or because the test was available only through a research protocol, highlighting the need for common reporting and practice guidelines for the international MGT community. CONCLUSION: The findings presented here provide further support for the need to consider the application of the Organisation for Economic Cooperation and Development (OECD) Guidelines and the establishment of compatible accreditation programs or equivalent mechanisms across national borders to ensure the quality of laboratory services and the clinical usefulness of molecular genetic test reports for referred specimens.     

Accreditation and the use of validated/recognised methods to analyse human semen

Accreditation of laboratories who perform diagnostic semen analysis in Australia and New Zealand is a requirement of the healthcare system. Within the accreditation process laboratories are required to set ISO standards within their policies and procedures. In order to achieve their aims, laboratories need to be able to measure a number of defined semen parameters both accurately and repetitively, especially around the lower limit of the reference intervals. The methods documented in the WHO-manual are used almost universal as the laboratory standard. Some laboratories incorporate minor method variations into their procedures. As part of the ISO requirements all variations require validation using internally approved processes that are documented and that incorporate appropriate statistical analysis and comparison of results. Validation is an ongoing process and regular review is essential. Evidence of the validation must be available for review by external auditors during accreditation. Where any validated variant method returns results that are significantly different to any method within the WHO-manual, the laboratory needs to develop its own, in-house reference interval for that method.     

Review: domestic animal forensic genetics – biological evidence,genetic markers,analytical approaches and challenges

This review highlights the importance of domestic animal genetic evidence sources, genetic testing, markers and analytical approaches as well as the challenges this field is facing in view of the de facto ‘gold standard’ human DNA identification. Because of the genetic similarity between humans and domestic animals, genetic analysis of domestic animal hair, saliva, urine, blood and other biological material has generated vital investigative leads that have been admitted into a variety of court proceedings, including criminal and civil litigation. Information on validated short tandem repeat, single nucleotide polymorphism and mitochondrial DNA markers and public access to genetic databases for forensic DNA analysis is becoming readily available. Although the fundamental aspects of animal forensic genetic testing may be reliable and acceptable, animal forensic testing still lacks the standardized testing protocols that human genetic profiling requires, probably because of the absence of monetary support from government agencies and the difficulty in promoting cooperation among competing laboratories. Moreover, there is a lack in consensus about how to best present the results and expert opinion to comply with court standards and bear judicial scrutiny. This has been the single most persistent challenge ever since the earliest use of domestic animal forensic genetic testing in a criminal case in the mid‐1990s. Crime laboratory accreditation ensures that genetic test results have the courts’ confidence. Because accreditation requires significant commitments of effort, time and resources, the vast majority of animal forensic genetic laboratories are not accredited nor are their analysts certified forensic examiners. The relevance of domestic animal forensic genetics in the criminal justice system is undeniable. However, further improvements are needed in a wide range of supporting resources, including standardized quality assurance and control protocols for sample handling, evidence testing, statistical analysis and reporting that meet the rules of scientific acceptance, reliability and human forensic identification standards.     

通过ISO 15189认可实验室质量指标制定及监测情况调查

目的 了解目前我国通过ISO 15189认可的实验室质量指标制定及监测情况。 方法 设计质量指标制定及监测情况调查表并下发给185家认可实验室,要求在10月31日前以邮件形式回报结果,使用Microsoft Excel 2010软件进行统计分析。 结果 共85家（占45.95%）实验室回报结果,76家临床实验室,9家独立实验室。不同认可实验室制定的质量指标数量及分布差异较大。76家临床实验室共设置质量指标1105项,每家实验室平均14.5项。9家独立实验室设置质量指标195项,每家实验室平均21.7项。临床实验室制定了更多的检验前、检验中质量指标,独立实验室在设置质量指标时还包括了较多的检验后、支持性过程和其他质量指标。 结论 目前我国认可实验室质量指标制定及监测情况尚不理想。在推进质量指标一致化的同时,实验室应加强宣贯与教育,建议一套完整、科学和实用的质量指标体系。

     

Method verification study for nucleic acid test system

To identify a suitable method for nucleic acid test(NAT) system verification, several methods were used to verify the system''s key parameters, such as the lowest limit of detection, specificity, accuracy and anti-interference ability. The lowest limit of detection for Grifols'' Procleix Tigris System were HBV DNA 3.1 U/mL, HCV RNA 5.0 U/mL, HIV RNA 21.2 U/mL; accuracy 100%; anti-interference lipemia (triglyceride)< 33.23 mmol/L, hemolysis (hemoglobin concentration)<5 g/L. There were no significant differences between the claimed specification of both the Grifols'' Procleix Tigris Systems and reagents, which both met with published test requirements. New equipment installation or regular verification are necessary to ensure the reliable operation of equipment, which ensure the quality of analysis and test. A systematic method was practiced in our laboratory, which was able to confirm that commercial NAT reagents meet the rigorous standards of blood screening. This study provides a very useful model for other blood screening laboratories and NAT kits.     

Regional Variation in Analytical Techniques used in the Diagnosis and Monitoring of Porphyria: a Case for Harmonisation?

Background:The Royal College of Pathologists of Australasia (RCPA) Porphyrin Quality Assurance Program assesses the measurement of urine, faecal, plasma and whole blood porphyrins and their components plus urinary porphobilinogen and delta aminolaevulinic acid and has laboratories enrolled from around the world. It was observed that there was a wide scatter in results submitted to some subsections of the program.Methods:A detailed questionnaire covering the analytical techniques used in the diagnosis of porphyria was sent to all laboratories enrolled in the RCPA Porphyrin Quality Assurance Program. Additionally, self-enrolment data over a five year period was examined for trends/changes in standardisation, reagent sources and analytical technique.Results:Twenty of the 45 laboratories enrolled in the Porphyrin Quality Assurance Program completed the survey, providing a snapshot of the analytical techniques used world-wide. Post survey self enrolment data indicated only little or no noticeable changes to analytical standardisation of techniques despite the continual lack of agreement of results in subsections of the External Quality Assurance program.Conclusions:While some aspects of porphyria testing are relatively consistent between laboratories, other diagnostic techniques vary widely. A wide variety of individualised reference intervals and reporting techniques is currently in use world-wide. While most of the participants in the survey are regional reference centres specialising in the diagnosis of porphyria and, as such, their diagnostic capability is not in question, international guidelines or global harmonisation of analytical techniques should allow better inter-laboratory comparisons to be made, ultimately improving diagnostic accuracy.     

Analytical Performance Characteristics of the Cepheid GeneXpert Ebola Assay for the Detection of Ebola Virus

BackgroundThe recently developed Xpert® Ebola Assay is a novel nucleic acid amplification test for simplified detection of Ebola virus (EBOV) in whole blood and buccal swab samples. The assay targets sequences in two EBOV genes, lowering the risk for new variants to escape detection in the test. The objective of this report is to present analytical characteristics of the Xpert® Ebola Assay on whole blood samples.ConclusionIn summary, we found the Xpert® Ebola Assay to have high analytical sensitivity and specificity for the detection of EBOV in whole blood. It offers ease of use, fast turnaround time, and remote monitoring. The test has an efficient viral inactivation protocol, fulfills inclusivity and exclusivity criteria, and has specimen stability characteristics consistent with the need for decentralized testing. The simplicity of the assay should enable testing in a wide variety of laboratory settings, including remote laboratories that are not capable of performing highly complex nucleic acid amplification tests, and during outbreaks where time to detection is critical.     

Molecular Genetics External Quality Assessment Pilot Scheme for Irinotecan-Related UGT1A1 Genotyping in China

Irinotecan is widely used in the treatment of solid tumors, especially in colorectal cancer and lung cancer. Molecular testing for UGT1A1 genotyping is increasingly required in China for optimum irinotecan administration. In order to determine the performance of laboratories with regard to the whole testing process for UGT1A1 to ensure the consistency and accuracy of the test results, the National Center for Clinical Laboratories conducted an external quality assessment program for UGT1A1*28 genotyping in 2015. The panel, which comprised of four known mutational samples and six wild-type samples, was distributed to 45 laboratories that test for the presence of UGT1A1*28 polymorphisms. Participating laboratories were allowed to perform polymorphism analysis by using their routine methods. The accuracy of the genotyping and reporting of results was analyzed. Other information from the individual laboratories, including the number of samples tested each month, accreditation/certification status, and test methodology, was reviewed. Forty-four of the 45 participants reported the correct results for all samples. There was only one genotyping error, with a corresponding analytical sensitivity of 99.44% (179/180 challenges; 95% confidence interval: 96.94−99.99%) and an analytical specificity of 100% (270/270 challenges; 95% confidence interval: 98.64−100%). Both commercial kits and laboratory development tests were commonly used by the laboratories, and pyrosequencing was the main methodology used (n = 26, 57.8%). The style of the written reports showed large variation, and many reports showed a shortage of information. In summary, the first UGT1A1 genotyping external quality assessment result demonstrated that UGT1A1 genotype analysis of good quality was performed in the majority of pharmacogenetic testing centers that were investigated. However, greater education on the reporting of UGT1A1 genetic testing results is needed.     

Piloting Laboratory Quality System Management in Six Health Facilities in Nigeria

Background

Achieving accreditation in laboratories is a challenge in Nigeria like in most African countries. Nigeria adopted the World Health Organization Regional Office for Africa Stepwise Laboratory (Quality) Improvement Process Towards Accreditation (WHO/AFRO– SLIPTA) in 2010. We report on FHI360 effort and progress in piloting WHO-AFRO recognition and accreditation preparedness in six health facility laboratories in five different states of Nigeria.

Method

Laboratory assessments were conducted at baseline, follow up and exit using the WHO/AFRO– SLIPTA checklist. From the total percentage score obtained, the quality status of laboratories were classified using a zero to five star rating, based on the WHO/AFRO quality improvement stepwise approach. Major interventions include advocacy, capacity building, mentorship and quality improvement projects.

Results

At baseline audit, two of the laboratories attained 1- star while the remaining four were at 0- star. At follow up audit one lab was at 1- star, two at 3-star and three at 4-star. At exit audit, four labs were at 4- star, one at 3-star and one at 2-star rating. One laboratory dropped a ‘star’ at exit audit, while others consistently improved. The two weakest elements at baseline; internal audit (4%) and occurrence/incidence management (15%) improved significantly, with an exit score of 76% and 81% respectively. The elements facility and safety was the major strength across board throughout the audit exercise.

Conclusion

This effort resulted in measurable and positive impact on the laboratories. We recommend further improvement towards a formal international accreditation status and scale up of WHO/AFRO– SLIPTA implementation in Nigeria.     

Quality standards in proteomics research facilities: Common standards and quality procedures are essential for proteomics facilities and their users

Proteomics research infrastructures and core facilities within the Core for Life alliance advocate for community policies for quality control to ensure high standards in proteomics services.

Core facilities and research infrastructures have become an essential part of the scientific ecosystem. In the field of proteomics, national and international networks and research platforms have been established during the past decade that are supposed to set standards for high‐quality services, promote an exchange of professional information, and enable access to cutting‐edge, specialized proteomics technologies. Either centralized or distributed, these national and international proteomics infrastructures and technology platforms are generating massive amounts of data for the research community, and support a broad range of translational, computational and multi‐omics initiatives and basic research projects.By delegating part of their work to these services, researchers expect that the core facility adjusts their analytical protocols appropriately for their project to acquire data conforming best research practice of the scientific community. The implementation of quality assessment measures and commonly accepted quality controls in data generation is therefore crucially important for proteomics research infrastructures and the scientists who rely on them.However, current quality control and quality assessment procedures in proteomics core facilities and research infrastructures are a motley collection of protocols, standards, reference compounds and software tools. Proteomics relies on a customized multi‐step workflow typically consisting of sample preparation, data acquisition and data processing, and the implementation of each step differs among facilities. For example, sample preparation involves enzymatic digestion of the proteins, which can be performed in‐solution, in‐gel, or on‐beads, with often different proteolytic enzymes, chemicals, and conditions among laboratories. Data acquisition protocols are often customized to the particular instrument set up, and the acquired spectra and chromatograms are processed by different software tools provided by equipment vendors, third parties or developed in‐house.

…current quality control and quality assessment procedures in proteomics core facilities and research infrastructures are a motley collection of protocols, standards, reference compounds and software tools.

Moreover, core facilities implement their own guidelines to monitor the performance and quality of the entire workflow, typically utilizing different commercially available standards such as pre‐digested cell lysates, recombinant proteins, protein mixtures, or isotopically labeled peptides. Currently, there is no clear consensus on if, when and how to perform quality control checks. There is even less quality control in walk‐in facilities, where the staff is only responsible for correct usage of the instruments and users select and execute the analytical workflow themselves. It is not surprising therefore that instrument stability and robustness of the applied analytical approach are often unclear, which compromises analytical rigor.     

Voluntary accreditation of cellular therapies: Foundation for the Accreditation of Cellular Therapy (FACT)

Voluntary accreditation of cells, tissues, and cellular and tissue-based products intended for human transplantation is an important mechanism for improving quality in cellular therapy. The Foundation for the Accreditation of Cellular Therapy (FACT) has developed and implemented programs of voluntary inspection and accreditation for hematopoietic cellular therapy, and for cord blood banking. These programs are based on the standards of the clinical and laboratory professionals of the American Society of Blood and Marrow Transplantation (ASBMT), the International Society for Cellular Therapy (ISCT), and NETCORD. FACT has collaborated with European colleagues in the development of the Joint Accreditation Committee in Europe (jACIE). FACT has published standards documents, a guidance manual, accreditation checklists, and inspection documents; and has trained as inspectors over 300 professionals active in the field. All inspectors have a minimum of 5 years' experience in the area they inspect. Since the incorporation of FACT in 1996, 215 hematopoietic progenitor cell facilities have applied for FACT accreditation. Of these facilities, 113 are fully accredited; the others are in the process of document submission or inspection. Significant opportunities and challenges exist for FACT in the future, including keeping standards and guidance materials current and relevant, recruiting and retaining expert inspectors, and establishing collaborations to develop standards and accreditation systems for new cellular products. The continuing dialogue with the Food and Drug Administration (FDA) is also important to ensure that they are aware of the accomplishments of voluntary accreditation, and keep FACT membership alerted to FDA intentions for the future. Other potential avenues of communication and cooperation with FDA and other regulatory agencies are being investigated and evaluated.     

Multi-laboratory assay for harmonization of enumeration of viable CD34+ and CD45+ cells in frozen cord blood units

Background aimsIn 2016, specifications for both pre-cryopreserved and post-thawed cord blood were defined in the sixth edition of NetCord Foundation for the Accreditation of Cellular Therapy (FACT) Standards for Cord Blood Banks. However, for several experts, harmonization regarding flow cytometry analysis performed on post-thawed samples is still a concern. A multicenter study led by Héma-Québec aimed to provide scientific data to support the cord blood accreditation bodies such as NetCord FACT in the revision of standards.MethodsTwelve cord blood units were processed for plasma and red cell reduction following standard operating procedures. Cord blood unit aliquots were shipped to eight participating centers under cryogenic conditions for analysis before and after standardization of protocol. Repeatability of stem cell count, measured pre- and post-intervention with the centers, was estimated using multilevel linear regression models with a heterogeneous compound symmetry correlation structure among repeated measures.ResultsExcellent inter-center repeatability was reported by each participant regarding the viable CD34+ cells concentration, and a successful improvement effect of protocol standardization was also observed. However, we observed that better control over the critical parameters of the protocol did not have a significant effect on improving homogeneity in the enumeration of CD45+ cells.ConclusionsThe current practice in cord blood selection should now also consider relying on post-thaw CD34+ concentration, providing that all cord blood banks or outsourcing laboratories in charge of the analysis of post-thaw CB samples take into account the consensual recommendations provided in this work and adhere to a good-quality management system.     

Curriculum for education and training of Medical Physicists in Nuclear Medicine: Recommendations from the EANM Physics Committee,the EANM Dosimetry Committee and EFOMP

PurposeTo provide a guideline curriculum covering theoretical and practical aspects of education and training for Medical Physicists in Nuclear Medicine within Europe.Material and methodsNational training programmes of Medical Physics, Radiation Physics and Nuclear Medicine physics from a range of European countries and from North America were reviewed and elements of best practice identified. An independent panel of experts was used to achieve consensus regarding the content of the curriculum.ResultsGuidelines have been developed for the specialist theoretical knowledge and practical experience required to practice as a Medical Physicist in Nuclear Medicine in Europe. It is assumed that the precondition for the beginning of the training is a good initial degree in Medical Physics at master level (or equivalent). The Learning Outcomes are categorised using the Knowledge, Skill and Competence approach along the lines recommended by the European Qualifications Framework. The minimum level expected in each topic in the theoretical knowledge and practical experience sections is intended to bring trainees up to the requirements expected of a Medical Physicist entering the field of Nuclear Medicine.ConclusionsThis new joint EANM/EFOMP European guideline curriculum is a further step to harmonise specialist training of Medical Physicists in Nuclear Medicine within Europe. It provides a common framework for national Medical Physics societies to develop or benchmark their own curricula. The responsibility for the implementation and accreditation of these standards and guidelines resides within national training and regulatory bodies.     

The 2012/2013 ABRF Proteomic Research Group Study: Assessing Longitudinal Intralaboratory Variability in Routine Peptide Liquid Chromatography Tandem Mass Spectrometry Analyses*

Questions concerning longitudinal data quality and reproducibility of proteomic laboratories spurred the Protein Research Group of the Association of Biomolecular Resource Facilities (ABRF-PRG) to design a study to systematically assess the reproducibility of proteomic laboratories over an extended period of time. Developed as an open study, initially 64 participants were recruited from the broader mass spectrometry community to analyze provided aliquots of a six bovine protein tryptic digest mixture every month for a period of nine months. Data were uploaded to a central repository, and the operators answered an accompanying survey. Ultimately, 45 laboratories submitted a minimum of eight LC-MSMS raw data files collected in data-dependent acquisition (DDA) mode. No standard operating procedures were enforced; rather the participants were encouraged to analyze the samples according to usual practices in the laboratory. Unlike previous studies, this investigation was not designed to compare laboratories or instrument configuration, but rather to assess the temporal intralaboratory reproducibility. The outcome of the study was reassuring with 80% of the participating laboratories performing analyses at a medium to high level of reproducibility and quality over the 9-month period. For the groups that had one or more outlying experiments, the major contributing factor that correlated to the survey data was the performance of preventative maintenance prior to the LC-MSMS analyses. Thus, the Protein Research Group of the Association of Biomolecular Resource Facilities recommends that laboratories closely scrutinize the quality control data following such events. Additionally, improved quality control recording is imperative. This longitudinal study provides evidence that mass spectrometry-based proteomics is reproducible. When quality control measures are strictly adhered to, such reproducibility is comparable among many disparate groups. Data from the study are available via ProteomeXchange under the accession code PXD002114.The broad-reaching use and application of mass spectrometry-based proteomics in the international research community continues to exponentially grow and expand. As the technology has developed and practitioners have become skilled in performing complex workflows, the community has not only gained interest in assessing data across laboratories but also in maintaining consistent quality control within a laboratory. Koecher et al. raised the issue of quality control measures and how this aspect of mass spectrometry-based proteomics is generally neglected in scientific publications (1). Fortunately, studies characterizing the stability of liquid chromatography-tandem MS (LC-MSMS)¹ quality control performance among numerous laboratories are emerging. The relationship between sample preparation schemes, data acquisition and reduction strategies, and bioinformatic analyses have been comprehensively reviewed by Tabb (2).Several studies exist where intra- and interlaboratory reproducibility between multiple sites has been assessed under different settings. Perhaps the most systematic and detailed of these investigations are from the Human Proteome Organization (HuPO) test sample working group (3); the National Cancer Institute Clinical Proteomic Tumor Analysis Consortium (NCI CPTAC) (4); and the ProteoRed Consortium (5, 6). The HuPO group utilized an equimolar mixture of 20 highly purified recombinant human proteins (5 pmol per protein) distributed to 27 different laboratories and analyzed without constraint according to optimized LCMS and database search protocols from each of the laboratories (3). The study was not an assessment of instrument performance for highly sensitive detection of proteins, as all participating laboratories had acquired raw data of sufficient quality to identify all 20 proteins (and a specific subset of tryptic peptides). The study revealed, however, that discrepancies in peptide identification and protein assignment were the result of differences in data analysis strategies rather than data collection.The NCI CPTAC group used a standardized Saccharomyces cerevisiae proteome digest that was analyzed on ion-trap-based LCMS platforms in five independent laboratories according to both an established standard operating procedure (SOP) and with no SOP constraint (4). All data analysis was centralized, and thus, any observed variations were entirely because of the LCMS platform. By applying the performance metrics developed by Rudnick et al. (7), several key points emerged: (1) as expected, intralaboratory variation was less than interlaboratory variation; and (2) overall, the interlaboratory variation in peptide identifications and some of the other performance metrics were comparable between instruments, although there were large differences in the average values for some metrics (e.g. MS1 signal intensity, dynamic sampling).The ProteoRed Consortium initiated the ProteoRed Multicenter Experiment for Quality Control (PMEQC) (5, 6). This longitudinal QC multicenter study involved 12 institutes, and was designed to assess: (1) intralaboratory repeatability of LC-MSMS proteomic data; (2) interlaboratory reproducibility; and (3) reproducibility across multiple instrument platforms. Participants received samples of undigested or tryptically digested yeast proteins and were requested to follow strict analytical guidelines. Data analysis was centralized and performed under standard procedures using a common workflow. The study revealed that the overall performance with respect to metrics such as reproducibility, sensitivity, dynamic range etc. was directly related to the degree of operator expertise, and less dependent on instrumentation.Several studies not specifically focused on quality control have also yielded insight into proteomic reproducibility. The HuPO plasma proteome project (HuPO PPP) distributed 20 human samples (five serum plus 3 × 5 plasma samples treated with three different anticoagulants) to 35 laboratories spanning 13 countries (8). The purpose of this large-scale study was not to assess reproducibility per se, but rather to generate the largest and most comprehensive data set on the protein composition of human plasma/serum. On a smaller scale, the ISB standard 18 protein mixture (purified proteins from cow, horse, rabbit, chicken, E. coli, and B. licheniformus) was also assessed between laboratories on eight different LCMS platforms (9). These data reside in a comprehensive, multiplatform database as a resource for the proteomic community. Additional interlaboratory assessments have consisted of multiple reaction monitoring-based measurements of peptides/proteins in plasma (10, 11) and protein–protein interactions at both the biochemical and proteomic level (12).For team leaders/directors of proteomic laboratories and any researcher collaborating with such groups, major questions that may arise concerning data consistency are: how well are quality controls being implemented in the daily operations? Do the quality control measures effectively support data reproducibility? To address this, the Protein Research Group of the Association of Biomolecular Resource Facilities (ABRF-PRG) designed a study whereby LC-MSMS data obtained from the analysis of a commercially available bovine protein mixture predigested with trypsin were collected at routine intervals over a period of 9 months. Raw MS data files from a total of 64 participating laboratories were accumulated, and HPLC and MS performance were evaluated through QC metrics (13). The main impetus of the study was to recognize key sources of variability in HPLC and MS analyses under extended and routine operating conditions for each laboratory and to catalog the state of quality control in a diverse set of proteomic laboratories.No standard operating protocol was imposed on the participants; instead, contributors were encouraged to employ the methods that were typically applied in individual laboratories. Optimization of instrument methods on the provided sample was discouraged. A survey was conducted with each sample submission to catalog individual laboratory practices, instrument configurations, acquisition settings, including routine and nonroutine maintenance procedures. Unlike previous investigations where emphasis was placed on the preparation, distribution, and evaluation of protein standards to appraise and/or standardize LCMS platforms between laboratories, the key interest in this study was purely to determine the intralaboratory performance, reproducibility, and consistency of participating laboratories over an extended period of time.The rapidly expanding number of proteomic laboratories have incorporated divergent HPLC systems, mass spectrometers, solvent systems, columns etc. As a result, analyzing data from a large number of laboratories necessitates tools that can accommodate data from a broad range of platforms. For example, to expect a small laboratory with a decade-old three-dimensional ion trap mass spectrometer to achieve the same sensitivity as a laboratory with a high-resolution hybrid instrument would be unfair. Correspondingly, the data analysis needs to include axes beyond simple peptide-level sensitivity. Nevertheless, the laboratory with the older instrumentation may be consistently better at maximizing performance from the chosen instrument platform compared with a laboratory with the latest high-end equipment.The focus of this study was to estimate the degree of variability in intralaboratory performance over a 9-month period. This goal was achieved using quality metrics that are applicable to most LC-MSMS workflows. The inclusion of data from many laboratories will enable the proteomic community to determine the current state of quality control within a typical laboratory. The survey data enabled the mapping of some alterations in instrument performance to documented laboratory events, e.g. mass spectrometer calibration. The study was designed neither to compare one laboratory with another, nor to discriminate between classes of instrumentation.Questions of data quality and performance in the proteomic community are appropriately aligned with the heightened awareness of a perceived lack of reproducibility of scientific findings in general (1). This community has endeavored to provide tools to assess proteomic data quality, and this study provides additional insight into the application of such tools and the quality of data within respective laboratories.     

Quality assessment of measurement

Assessing the quality of measurements is of interest to organizers of external quality assessment schemes (EQAS, or proficiency testing schemes), laboratory analysts and managers, users of laboratory results and other agencies. Scheme organizers run test programmes, define standards of acceptable and non-acceptable performance, and interact with participants and oversight authorities. Laboratory personnel are responsible for the quality management system and to choose whether to accept the standards set by scheme organizers or to adopt their own. Users receive and act upon the laboratory results.Schemes within the same analytical sector are often organized very differently causing contradictory assessment of performance. The Network of EQAS in occupational and environmental laboratory medicine established collaborative projects designed to enhance assessment of measurement quality and to improve the reliability of laboratory results.To address the issue of variations in assessing the quality of measurements, and in response to comments from some participants, standards derived from biological variation, rather than on the analytical performance of participants have been developed. Evaluation of test materials with respect to homogeneity and stability, and work on methods to give the assigned value to test materials, has also been undertaken. Following from these projects, further collaboration is planned which will provide better quality assessment of measurement to scheme participants and their users.     

Interlaboratory Study Characterizing a Yeast Performance Standard for Benchmarking LC-MS Platform Performance

Optimal performance of LC-MS/MS platforms is critical to generating high quality proteomics data. Although individual laboratories have developed quality control samples, there is no widely available performance standard of biological complexity (and associated reference data sets) for benchmarking of platform performance for analysis of complex biological proteomes across different laboratories in the community. Individual preparations of the yeast Saccharomyces cerevisiae proteome have been used extensively by laboratories in the proteomics community to characterize LC-MS platform performance. The yeast proteome is uniquely attractive as a performance standard because it is the most extensively characterized complex biological proteome and the only one associated with several large scale studies estimating the abundance of all detectable proteins. In this study, we describe a standard operating protocol for large scale production of the yeast performance standard and offer aliquots to the community through the National Institute of Standards and Technology where the yeast proteome is under development as a certified reference material to meet the long term needs of the community. Using a series of metrics that characterize LC-MS performance, we provide a reference data set demonstrating typical performance of commonly used ion trap instrument platforms in expert laboratories; the results provide a basis for laboratories to benchmark their own performance, to improve upon current methods, and to evaluate new technologies. Additionally, we demonstrate how the yeast reference, spiked with human proteins, can be used to benchmark the power of proteomics platforms for detection of differentially expressed proteins at different levels of concentration in a complex matrix, thereby providing a metric to evaluate and minimize preanalytical and analytical variation in comparative proteomics experiments.Access to proteomics performance standards is essential for several reasons. First, to generate the highest quality data possible, proteomics laboratories routinely benchmark and perform quality control (QC)1 monitoring of the performance of their instrumentation using standards. Second, appropriate standards greatly facilitate the development of improvements in technologies by providing a timeless standard with which to evaluate new protocols or instruments that claim to improve performance. For example, it is common practice for an individual laboratory considering purchase of a new instrument to require the vendor to run “demo” samples so that data from the new instrument can be compared head to head with existing instruments in the laboratory. Third, large scale proteomics studies designed to aggregate data across laboratories can be facilitated by the use of a performance standard to measure reproducibility across sites or to compare the performance of different LC-MS configurations or sample processing protocols used between laboratories to facilitate development of optimized standard operating procedures (SOPs).Most individual laboratories have adopted their own QC standards, which range from mixtures of known synthetic peptides to digests of bovine serum albumin or more complex mixtures of several recombinant proteins (1). However, because each laboratory performs QC monitoring in isolation, it is difficult to compare the performance of LC-MS platforms throughout the community.Several standards for proteomics are available for request or purchase (2, 3). RM8327 is a mixture of three peptides developed as a reference material in collaboration between the National Institute of Standards and Technology (NIST) and the Association of Biomolecular Resource Facilities. Mixtures of 15–48 purified human proteins are also available, such as the HUPO (Human Proteome Organisation) Gold MS Protein Standard (Invitrogen), the Universal Proteomics Standard (UPS1; Sigma), and CRM470 from the European Union Institute for Reference Materials and Measurements. Although defined mixtures of peptides or proteins can address some benchmarking and QC needs, there is an additional need for more complex reference materials to fully represent the challenges of LC-MS data acquisition in complex matrices encountered in biological samples (2, 3).Although it has not been widely distributed as a reference material, the yeast Saccharomyces cerevisiae proteome has been extensively used by the proteomics community to characterize the capabilities of a variety of LC-MS-based approaches (4–15). Yeast provides a uniquely attractive complex performance standard for several reasons. Yeast encodes a complex proteome consisting of ∼4,500 proteins expressed during normal growth conditions (7, 16–18). The concentration range of yeast proteins is sufficient to challenge the dynamic range of conventional mass spectrometers; the abundance of proteins ranges from fewer than 50 to more than 10⁶ molecules per cell (4, 15, 16). Additionally, it is the most extensively characterized complex biological proteome and the only one associated with several large scale studies estimating the abundance of all detectable proteins (5, 9, 16, 17, 19, 20) as well as LC-MS/MS data sets showing good correlation between LC-MS/MS detection efficiency and the protein abundance estimates (4, 11, 12, 15). Finally, it is inexpensive and easy to produce large quantities of yeast protein extract for distribution.In this study, we describe large scale production of a yeast S. cerevisiae performance standard, which we offer to the community through NIST. Through a series of interlaboratory studies, we created a reference data set characterizing the yeast performance standard and defining reasonable performance of ion trap-based LC-MS platforms in expert laboratories using a series of performance metrics. This publicly available data set provides a basis for additional laboratories using the yeast standard to benchmark their own performance as well as to improve upon the current status by evolving protocols, improving instrumentation, or developing new technologies. Finally, we demonstrate how the yeast performance standard, spiked with human proteins, can be used to benchmark the power of proteomics platforms for detection of differentially expressed proteins at different levels of concentration in a complex matrix.     

Evaluation of automated hematology analyzer DYMIND DH76 compared to SYSMEX XN 1000 system

BackgroundDYMIND DH76 (DYMIND BIOTECH, China) is a new automated hematology system designed to provide CBC count, including a 5-part WBC differential count, and its analytical performance should be assessed before adoption for clinical use.MethodsThe analyzer was evaluated according to the International Council for Standardization in Haematology guideline. The purposes of this study were to assess its analytical performance in comparison to SYSMEX XN 1000 hematology analyzer currently used in our laboratory, as well as to compare the automated and manual WBC differential.ResultsWithin-run precision in all concentration ranges was very good with coefficients of variation (CVs) between 0.02% and 2.5% except for platelets over 500×109/L (CV 9.5%). Within-batch imprecision showed CVs lower the declared deviation ranges. Accuracy (defined as trueness) was excellent for all CBC and white cell differential parameters, compared with the state of the art%. Linearity was confirmed with excellent regression coefficients (0.999-1.000), even in the lowest values, and carryover was ≤ 1%. Comparison between DYMIND DH76 and SYSMEX XN 1000 was also very good with correlation coefficients (R2) for WBC (1.000), RBC (0.999), hemoglobin (0.999) and PLT over 50×10⁹/L (0.994) and R2 was lower but still acceptable (0.910) for PLT<50×10⁹/L. R2 for neutrophils, lymphocytes, eosinophils, basophils, and monocytes were 0.974, 0.982, 0.957, 0.625, and 0.836, respectively, in the comparison between the manual and DYMIND DH76 automated differential WBC counts.ConclusionsWith excellent analytical performance and acceptable comparative analysis, DYMIND DH76 hematology analyser covered the predefined international standards and requirements and is fully appropriate for clinical application.     

External Quality Assessment for Tuberculosis Diagnosis and Drug Resistance in the European Union: A Five Year Multicentre Implementation Study

BackgroundExternal quality assurance (EQA) systems are essential to ensure accurate diagnosis of TB and drug-resistant TB. The implementation of EQA through organising regular EQA rounds and identification of training needs is one of the key activities of the European TB reference laboratory network (ERLTB-Net). The aim of this study was to analyse the results of the EQA rounds in a systematic manner and to identify potential benefits as well as common problems encountered by the participants.MethodsThe ERLTB-Net developed seven EQA modules to test laboratories’ proficiency for TB detection and drug susceptibility testing using both conventional and rapid molecular tools. All National TB Reference laboratories in the European Union and European Economic Area (EU/EEA) Member States were invited to participate in the EQA scheme.ResultsA total of 32 National TB Reference laboratories participated in six EQA rounds conducted in 2010–2014. The participation rate ranged from 52.9% - 94.1% over different modules and rounds. Overall, laboratories demonstrated very good proficiency proving their ability to diagnose TB and drug-resistant TB with high accuracy in a timely manner. A small number of laboratories encountered problems with identification of specific Non-tuberculous Mycobacteria (NTMs) (N = 5) and drug susceptibility testing to Pyrazinamide, Amikacin, Capreomycin, and Ethambutol (N = 4).ConclusionsThe European TB Reference laboratories showed a steady and high level of performance in the six EQA rounds. A network such as ERLTB-Net can be instrumental in developing and implementing EQA and in establishing collaboration between laboratories to improve the diagnosis of TB in the EU/EEA.