Relevance of the College of American Pathologists guideline for validating whole slide imaging for diagnostic purposes to cytopathology

Whole slide imaging (WSI) allows pathologists to view virtual versions of slides on computer monitors. With increasing adoption of digital pathology, laboratories have begun to validate their WSI systems for diagnostic purposes according to reference guidelines. Among these the College of American Pathologists (CAP) guideline includes three strong recommendations (SRs) and nine good practice statements (GPSs). To date, the application of WSI to cytopathology has been beyond the scope of the CAP guideline due to limited evidence. Herein we systematically reviewed the published literature on WSI validation studies in cytology. A systematic search was carried out in PubMed-MEDLINE and Embase databases up to November 2021 to identify all publications regarding validation of WSI in cytology. Each article was reviewed to determine if SRs and/or GPSs recommended by the CAP guideline were adequately satisfied. Of 3963 retrieved articles, 25 were included. Only 4/25 studies (16%) satisfied all three SRs, with only one publication (1/25, 4%) fulfilling all three SRs and nine GPSs. Lack of a suitable validation dataset was the main missing SR (16/25, 64%) and less than a third of the studies reported intra-observer variability data (7/25, 28%). Whilst the CAP guideline for WSI validation in clinical practice helped the widespread adoption of digital pathology, more evidence is required to routinely employ WSI for diagnostic purposes in cytopathology practice. More dedicated validation studies satisfying all SRs and/or GPSs recommended by the CAP are needed to help expedite the use of WSI for primary diagnosis in cytopathology.     

An update on the validation of whole slide imaging systems following FDA approval of a system for a routine pathology diagnostic service in the United States

Pathologists have used light microscopes and glass slides to interpret the histologic appearance of normal and diseased tissues for more than 150 years. The quality of both microtomes used to cut tissue sections and microscopes has improved significantly during the past few decades, but the process of rendering diagnoses has changed little. By contrast, major advances in digital technology have occurred since the introduction of hand held electronic devices, including the development of whole slide imaging (WSI) systems with software packages that can convert microscope images into virtual (digital) slides that can be viewed on computer monitors and via the internet. To date, however, these technological developments have had minimal impact on the way pathologists perform their daily work, with the exception of using computers to access electronic medical records and scholarly web sites for pertinent information to assist interpretation of cases. Traditional practice is likely to change significantly during the next decade, especially since the Federal Drug Administration in the USA has approved the first WSI system for routine diagnostic practice. I review here the development and slow acceptance of WSI by pathology departments. I focus on recent advances in validation of WSI systems that is required for routine diagnostic reporting of pathology cases using this technology.     

Accuracy of a remote quantitative image analysis in the whole slide images

The rationale for choosing a remote quantitative method supporting a diagnostic decision requires some empirical studies and knowledge on scenarios including valid telepathology standards. The tumours of the central nervous system [CNS] are graded on the base of the morphological features and the Ki-67 labelling Index [Ki-67 LI]. Various methods have been applied for Ki-67 LI estimation. Recently we have introduced the Computerized Analysis of Medical Images [CAMI] software for an automated Ki-67 LI counting in the digital images. Aims of our study was to explore the accuracy and reliability of a remote assessment of Ki-67 LI with CAMI software applied to the whole slide images [WSI]. The WSI representing CNS tumours: 18 meningiomas and 10 oligodendrogliomas were stored on the server of the Warsaw University of Technology. The digital copies of entire glass slides were created automatically by the Aperio ScanScope CS with objective 20x or 40x. Aperio's Image Scope software provided functionality for a remote viewing of WSI. The Ki-67 LI assessment was carried on within 2 out of 20 selected fields of view (objective 40x) representing the highest labelling areas in each WSI. The Ki-67 LI counting was performed by 3 various methods: 1) the manual reading in the light microscope - LM, 2) the automated counting with CAMI software on the digital images - DI , and 3) the remote quantitation on the WSIs - as WSI method. The quality of WSIs and technical efficiency of the on-line system were analysed. The comparative statistical analysis was performed for the results obtained by 3 methods of Ki-67 LI counting. The preliminary analysis showed that in 18% of WSI the results of Ki-67 LI differed from those obtained in other 2 methods of counting when the quality of the glass slides was below the standard range. The results of our investigations indicate that the remote automated Ki-67 LI analysis performed with the CAMI algorithm on the whole slide images of meningiomas and oligodendrogliomas could be successfully used as an alternative method to the manual reading as well as to the digital images quantitation with CAMI software. According to our observation a need of a remote supervision/consultation and training for the effective use of remote quantitative analysis of WSI is necessary.     

Color standardization and optimization in Whole Slide Imaging

Introduction

Standardization and validation of the color displayed by digital slides is an important aspect of digital pathology implementation. While the most common reason for color variation is the variance in the protocols and practices in the histology lab, the color displayed can also be affected by variation in capture parameters (for example, illumination and filters), image processing and display factors in the digital systems themselves.

Method

We have been developing techniques for color validation and optimization along two paths. The first was based on two standard slides that are scanned and displayed by the imaging system in question. In this approach, one slide is embedded with nine filters with colors selected especially for H&;E stained slides (looking like tiny Macbeth color chart); the specific color of the nine filters were determined in our previous study and modified for whole slide imaging (WSI). The other slide is an H&;E stained mouse embryo. Both of these slides were scanned and the displayed images were compared to a standard. The second approach was based on our previous multispectral imaging research.

Discussion

As a first step, the two slide method (above) was used to identify inaccurate display of color and its cause, and to understand the importance of accurate color in digital pathology. We have also improved the multispectral-based algorithm for more consistent results in stain standardization. In near future, the results of the two slide and multispectral techniques can be combined and will be widely available.We have been conducting a series of researches and developing projects to improve image quality to establish Image Quality Standardization. This paper discusses one of most important aspects of image quality – color.

     

COST Action “EuroTelepath”: digital pathology integration in electronic health record,including primary care centres

INTRODUCTION: Digital pathology includes the information technology that allows for the management of information, including data and images, generated in an anatomic pathology department. COST ACTION IC0604: The integration of digital slides in the electronic health record is one of the main objectives of COST Action IC0604 "Telepathology Network in Europe" (EURO-TELEPATH). Fostering use of medical informatics standards and adapting them to current needs is needed to manage efficiently extremely large medical images, like digital slide files. DIGITAL SLIDES IN PATHOLOGY: Digital slides can play a role in disease prevention, primary diagnosis, and second opinion. In all these tasks, automated image analysis can also be a most valuable tool. INTEROPERABILITY IN PATHOLOGY INFORMATION SYSTEMS: In order to achieve an efficient interoperability between pathology information systems with other clinical information systems, obtaining a seamless integration of pathology images (gross pictures and digital slides) with LIS-Pathology Information system in a web environment is an important task. Primary care information systems should also be included in the integration, since primary care centres play an essential role in the generation of clinical information and specimen collection. A common terminology, based in SNOMED CT is also needed. CONCLUSIONS: Main barrier in the integration of digital slides in pathology workflow and eHealth record is the cost of current digital slide scanners. Pathology information system vendors should participate in standardization bodies.     

Whole slide images and digital media in pathology education, testing, and practice: the Oklahoma experience

Examination of glass slides is of paramount importance in pathology training. Until the introduction of digitized whole slide images that could be accessed through computer networks, the sharing of pathology slides was a major logistic issue in pathology education and practice. With the help of whole slide images, our department has developed several online pathology education websites. Based on a modular architecture, this program provides online access to whole slide images, still images, case studies, quizzes and didactic text at different levels. Together with traditional lectures and hands-on experiences, it forms the back bone of our histology and pathology education system for residents and medical students. The use of digitized whole slide images has a.lso greatly improved the communication between clinicians and pathologist in our institute.     

Virtual telepathology in Egypt,applications of WSI in Cairo University

Telepathology, the practice of pathology at a long distance, has advanced continuously since 1986. Today, fourth-generation telepathology systems, so-called virtual slide telepathology systems, are being used for education applications. Both conventional and innovative surgical pathology diagnostic services are being designed and implemented as well. We have a successful experience in Egypt in applying the static & dynamic techniques in a pilot project between the Italian Hospital in Cairo (NPO) and the Civico Hospital in Palermo This project began in 2003 and continued till now. In 2004, centers in Venice, London and Pittsburgh participated actively in our project. During the past seven years we consulted on many problematic pathological cases with these different specialized pathological centers in Italy, UK & USA. In addition to the highly specialized scientific value of consulting on the cases and exchanging knowledge, we saved a lot of time and money and succeeded in providing our patients with a better medical service. In view of this success we have already established a new Digital Telepathology unit (DTU) in the pathology department, Cairo University, using the latest technique of telepathology which is Whole Slide Imaging (WSI) since one year. This unit is considered the first Digital pathology unit in all the universities of the whole Middle East. During the passed year we created a digital pathology library for the under graduate students using the WSI technique and changed the teaching method of the histopathology slides to be completely digital. We are building another digital pathology library (for post graduate candidates) which will be available to all pathology candidates in Egyptian universities & universities in the surrounding Arabic countries. We are also creating a digital pathology network between pathology centers in the Middle East for exchanging knowledge & telepathology.     

Eye Movements as an Index of Pathologist Visual Expertise: A Pilot Study

A pilot study examined the extent to which eye movements occurring during interpretation of digitized breast biopsy whole slide images (WSI) can distinguish novice interpreters from experts, informing assessments of competency progression during training and across the physician-learning continuum. A pathologist with fellowship training in breast pathology interpreted digital WSI of breast tissue and marked the region of highest diagnostic relevance (dROI). These same images were then evaluated using computer vision techniques to identify visually salient regions of interest (vROI) without diagnostic relevance. A non-invasive eye tracking system recorded pathologists’ (N = 7) visual behavior during image interpretation, and we measured differential viewing of vROIs versus dROIs according to their level of expertise. Pathologists with relatively low expertise in interpreting breast pathology were more likely to fixate on, and subsequently return to, diagnostically irrelevant vROIs relative to experts. Repeatedly fixating on the distracting vROI showed limited value in predicting diagnostic failure. These preliminary results suggest that eye movements occurring during digital slide interpretation can characterize expertise development by demonstrating differential attraction to diagnostically relevant versus visually distracting image regions. These results carry both theoretical implications and potential for monitoring and evaluating student progress and providing automated feedback and scanning guidance in educational settings.     

Building of a composite virtual slide from contiguous tissue samples

Background

Currently available microscope slide scanners produce whole slide images at various resolutions from histological sections. Nevertheless, acquisition area and so visualization of large tissue samples are limited by the standardized size of glass slides, used daily in pathology departments. The proposed solution has been developed to build composite virtual slides from images of large tumor fragments.

Materials and methods

Images of HES or immunostained histological sections of carefully labeled fragments from a representative slice of breast carcinoma were acquired with a digital slide scanner at a magnification of 20×. The tiling program involves three steps: the straightening of tissue fragment images using polynomial interpolation method, and the building and assembling of strips of contiguous tissue sample whole slide images in × and y directions. The final image is saved in a pyramidal BigTiff file format. The program has been tested on several tumor slices. A correlation quality control has been done on five images artificially cut.

Results

Sixty tumor slices from twenty surgical specimens, cut into two to twenty six pieces, were reconstructed. A median of 98.71% is obtained by computing the correlation coefficients between native and reconstructed images for quality control.

Conclusions

The proposed method is efficient and able to adapt itself to daily work conditions of classical pathology laboratories.

     

The importance of optical optimization in whole slide imaging (WSI) and digital pathology imaging

In the last 10 years, whole slide imaging (WSI) has seen impressive progress not only in image quality and scanning speed but also in the variety of systems available to pathologists. However, we have noticed that most systems have relatively simple optics axes and rely on software to optimize image quality and colour balance. While much can be done in software, this study examines the importance of optics, in particular optical filters, in WSI.Optical resolution is a function of the wavelength of light used and the numerical aperture of the lens system (Resolution = (f) wavelength/2 NA). When illumining light is not conditioned correctly with filters, there is a tendency for the wavelength to shift to longer values (more red) because of the characteristics of the lamps in common use. Most microscopes (but remarkably few WSI devices) correct for this with ND filter for brightness and Blue filter (depends on the light source) for colour correction.Using H&E slides research microscopes (Axiophot, Carl Zeiss MicroImaging, Inc. NY. Eclipse 50i., Nikon Inc. NY) at 20x, an attached digital camera (SPOT RT741 Slider Color, Diagnosis Instruments., MI USA), and a filter set, we examined the effect of filters and software enhancement on digital image quality. The focus value (as evaluated by focus evaluation software developed in house and SPOT imaging Software v4.6) was used as a proxy for image quality. Resolution of tissue features was best with the use of both the Blue and ND filters (in addition to software enhancement). Images without filters but with software enhancement while superficially good, lacked some details of specimen morphology and were unclear compared with the images with filters.The results indicate that the appropriate use of optical filters could measurably improve the appearance and resolution of WSI images.     

iPathology cockpit diagnostic station: validation according to College of American Pathologists Pathology and Laboratory Quality Center recommendation at the Hospital Trust and University of Verona

Background

Validation of digital whole slide images is crucial to ensure that diagnostic performance is at least equivalent to that of glass slides and light microscopy. The College of American Pathologists Pathology and Laboratory Quality Center recently developed recommendations for internal digital pathology system validation. Following these guidelines we sought to validate the performance of a digital approach for routine diagnosis by using an iPad and digital control widescreen-assisted workstation through a pilot study.

Methods

From January 2014, 61 histopathological slides were scanned by ScanScope Digital Slides Scanner (Aperio, Vista, CA). Two independent pathologists performed diagnosis on virtual slides in front of a widescreen by using two computer devices (ImageScope viewing software) located to different Health Institutions (AOUI Verona) connected by local network and a remote image server using an iPad tablet (Aperio, Vista, CA), after uploading the Citrix receiver for iPad. Quality indicators related to image characters and work-flow of the e-health cockpit enterprise system were scored based on subjective (high vs poor) perception. The images were re-evaluated two weeks apart.

Results

The whole glass slides encountered 10 liver: hepatocarcinoma, 10 renal carcinoma, 10 gastric carcinoma and 10 prostate biopsies: adenocarcinoma, 5 excisional skin biopsies: melanoma, 5 lymph-nodes: lymphoma. 6 immuno- and 5 special stains were available for intra- and internet remote viewing. Scan times averaged two minutes and 54 seconds per slide (standard deviation 2 minutes 34 seconds). Megabytes ranged from 256 to 680 (mean 390) per slide storage. Reliance on glass slide, image quality (resolution and color fidelity), slide navigation time, simultaneous viewers in geographically remote locations were considered of high performance score. Side by side comparisons between diagnosis performed on tissue glass slides versus widescreen were excellent showing an almost perfect concordance (0.81, kappa index).

Conclusions

We validated our institutional digital pathology system for routine diagnostic facing with whole slide images in a cockpit enterprise digital system or iPad tablet. Computer widescreens are better for diagnosing scanned glass slide that iPad. For urgent requests, iPad may be used. Legal aspects have to be soon faced with to permit the clinical use of this technology in a manner that does not compromise patient care.

     

Teaching cytopathology to second-year medical students. An interactive,case-based approach

OBJECTIVE: To develop and implement an electronic, interactive, case-based cytopathology educational system for second-year medical students. STUDY DESIGN: Ten different learning modules, corresponding to various organ systems in pathology and encompassing the essence of clinical cytopathology, were developed. The modules are software-based, menu-driven, digital programs that are displayed on a computer monitor and can be projected onto a large screen via LCD projectors. Each module takes 15-20 minutes to complete and contains three to four case-based interactive clinical scenarios. Each case contains sequenced, multiple-choice questions with immediate feedback. Each module also includes an atlas mode for viewing all the case images plus additional images to enrich the learning experience. RESULTS: Preliminary feedback from students taking the pathology course has shown encouraging responses. The students enjoyed the practical approach of the modules, finding them easy to use. Their interest level remained high as they discovered how their general medical knowledge could apply to solving real-life clinical cytopathology cases. CONCLUSION: By utilizing these unique instructional modules in the second-year pathology course, medical students learn the application of clinical cytopathology in everyday medical practice. This not only provides them with the theoretical background and morphologic knowledge of the diseases taught but also gives them the ability to apply it in simulated real-life clinical scenarios.     

Towards better digital pathology workflows: programming libraries for high-speed sharpness assessment of Whole Slide Images

Background

Since microscopic slides can now be automatically digitized and integrated in the clinical workflow, quality assessment of Whole Slide Images (WSI) has become a crucial issue. We present a no-reference quality assessment method that has been thoroughly tested since 2010 and is under implementation in multiple sites, both public university-hospitals and private entities. It is part of the FlexMIm R&D project which aims to improve the global workflow of digital pathology. For these uses, we have developed two programming libraries, in Java and Python, which can be integrated in various types of WSI acquisition systems, viewers and image analysis tools.

Methods

Development and testing have been carried out on a MacBook Pro i7 and on a bi-Xeon 2.7GHz server. Libraries implementing the blur assessment method have been developed in Java, Python, PHP5 and MySQL5. For web applications, JavaScript, Ajax, JSON and Sockets were also used, as well as the Google Maps API. Aperio SVS files were converted into the Google Maps format using VIPS and Openslide libraries.

Results

We designed the Java library as a Service Provider Interface (SPI), extendable by third parties. Analysis is computed in real-time (3 billion pixels per minute). Tests were made on 5000 single images, 200 NDPI WSI, 100 Aperio SVS WSI converted to the Google Maps format.

Conclusions

Applications based on our method and libraries can be used upstream, as calibration and quality control tool for the WSI acquisition systems, or as tools to reacquire tiles while the WSI is being scanned. They can also be used downstream to reacquire the complete slides that are below the quality threshold for surgical pathology analysis. WSI may also be displayed in a smarter way by sending and displaying the regions of highest quality before other regions. Such quality assessment scores could be integrated as WSI's metadata shared in clinical, research or teaching contexts, for a more efficient medical informatics workflow.

     

Issues in using whole slide imaging for diagnostic pathology: “routine” stains,immunohistochemistry and predictive markers

The traditional microscope, together with the “routine” hematoxylin and eosin (H & E) stain, remains the “gold standard” for diagnosis of cancer and other diseases; remarkably, it and the majority of associated biological stains are more than 150 years old. Immunohistochemistry has added to the repertoire of “stains” available. Because of the need for specific identification and even measurement of “biomarkers,” immunohistochemistry has increased the demand for consistency of performance and interpretation of staining results. Rapid advances in the capabilities of digital imaging hardware and software now offer a realistic route to improved reproducibility, accuracy and quantification by utilizing whole slide digital images for diagnosis, education and research. There also are potential efficiencies in work flow and the promise of powerful new analytical methods; however, there also are challenges with respect to validation of the quality and fidelity of digital images, including the standard H & E stain, so that diagnostic performance by pathologists is not compromised when they rely on whole slide images instead of traditional stained tissues on glass slides.     

O-8Detection of human telomerase gene (TERC) amplification in cervical neoplasia: a retrospective study of 50 patients with normal smears or mild or moderate dyskaryosis

Circulation E of the North West non gynaecological cytology EQA scheme was split into two with one half receiving conventional glass slides and the other half receiving digital images supplied by the SlidePath company over the web. More of the participants eligible participated in the slide half (43/65) than the digital half (17/41).
Similar results were obtained for seven out of ten of the scoring cases and for both the educational cases. However for one case eight of the digital participants could not obtain the images over the web. In the remaining three scoring cases the digital participants were less good at obtaining the correct answer compared to the slide based participants. The use of this virtual microscopy has shortened the circulation length considerably. The digital participants complained about the speed the digital slides appeared on their computers and whilst most agreed that this is the way forward they are still uncomfortable with their performance being marked on their digital submissions. Delegates will have the opportunity of viewing still images taken for each case from this circulation.     

Automated region of interest retrieval and classification using spectral analysis

Efficient use of whole slide imaging in pathology needs automated region of interest (ROI) retrieval and classification, through the use of image analysis and data sorting tools. One possible method for data sorting uses Spectral Analysis for Dimensionality Reduction. We present some interesting results in the field of histopathology and cytohematology. In histopathology, we developed a Computer-Aided Diagnosis system applied to low-resolution images representing the totality of histological breast tumour sections. The images can be digitized directly at low resolution or be obtained from sub-sampled high-resolution virtual slides. Spectral Analysis is used (1) for image segmentation (stroma, tumour epithelium), by determining a "distance" between all the images of the database, (2) for choosing representative images and characteristic patterns of each histological type in order to index them, and (3) for visualizing images or features similar to a sample provided by the pathologist. In cytohematology, we studied a blood smear virtual slide acquired through high resolution oil scanning and Spectral Analysis is used to sort selected nucleated blood cell classes so that the pathologist may easily focus on specific classes whose morphology could then be studied more carefully or which can be analyzed through complementary instruments, like Multispectral Imaging or Raman MicroSpectroscopy.     

Histology,imaging and new diagnostic work-flows in pathology

Introduction

Since their introduction in 1999, fully automated, high speed, high-resolution whole slide imaging devices have become increasing more reliable, fast and capable. While by no means perfect, these devices have evolved to a point where one can consider placing them in a pre-diagnostic role in a clinical histology lab.

Methods

At the Massachusetts General Hospital, we are running a pilot study placing high end WSI devices in our main clinical histology lab (after the cover slipper and before slides are sent to the pathologist) to examine the requirement for both the machine and the laboratory.

Results

Placing WSI systems in the clinical lab stresses the system in terms of reliability and throughput. Significantly however, success requires significant modification to the lab workflow. It is likely laboratories need to move from manual, large batch processes to increasingly automated, continuous flow (or mini-batch) processes orchestrated by the LIS using bar coding to track and direct slides, and incorporating the decision to image into the specimen type and the histology orders. Furthermore, image quality, capture speed and reliability are functions of the quality of the histology presented to the WSI devices.

Conclusion

Imaging in pathology does not begin in a WSI robot but in the grossing room and in the histology lab. As more and more imaging devices are placed in histology lab, the inter-relationships histology and pathology imaging will become increasing understood.

     

Semantic Focusing Allows Fully Automated Single-Layer Slide Scanning of Cervical Cytology Slides

Liquid-based cytology (LBC) in conjunction with Whole-Slide Imaging (WSI) enables the objective and sensitive and quantitative evaluation of biomarkers in cytology. However, the complex three-dimensional distribution of cells on LBC slides requires manual focusing, long scanning-times, and multi-layer scanning. Here, we present a solution that overcomes these limitations in two steps: first, we make sure that focus points are only set on cells. Secondly, we check the total slide focus quality. From a first analysis we detected that superficial dust can be separated from the cell layer (thin layer of cells on the glass slide) itself. Then we analyzed 2,295 individual focus points from 51 LBC slides stained for p16 and Ki67. Using the number of edges in a focus point image, specific color values and size-inclusion filters, focus points detecting cells could be distinguished from focus points on artifacts (accuracy 98.6%). Sharpness as total focus quality of a virtual LBC slide is computed from 5 sharpness features. We trained a multi-parameter SVM classifier on 1,600 images. On an independent validation set of 3,232 cell images we achieved an accuracy of 94.8% for classifying images as focused. Our results show that single-layer scanning of LBC slides is possible and how it can be achieved. We assembled focus point analysis and sharpness classification into a fully automatic, iterative workflow, free of user intervention, which performs repetitive slide scanning as necessary. On 400 LBC slides we achieved a scanning-time of 13.9±10.1 min with 29.1±15.5 focus points. In summary, the integration of semantic focus information into whole-slide imaging allows automatic high-quality imaging of LBC slides and subsequent biomarker analysis.     

High throughput slanted scanning whole slide imaging system for digital pathology

In whole slide imaging (WSI), normally only a one layer imaging of the slide is performed. Autofocus at multiple positions is usually required. But defocus blur still exists due to tissue folding or specimen thickness. Repeated Z-stack scan be applied here, which, however, is too time consuming. Here, a high throughput slanted scanning WSI system is reported. In this system, the slide surface was slanted 1° relative to the focal plane. Thus, the focal plane spanned multiple layers of the sample. By moving the slide, multi-layer image data of the sample can be acquired simultaneously at a time frame comparable to conventional 1-layer imaging. With image fusion, defocus blur can be avoided. High quality and fast imaging of both cytological and histological slide specimens was demonstrated without applying aberration correction. The system can be a highly efficient way for the application of WSI in digital pathology.     

An overview of the College of American Pathologists' programs in surgical pathology and cytopathology. Data summary of diagnostic performance in cervical cytopathology

The survey programs in surgical pathology and cytopathology of the College of American Pathologists are described, along with a summary of the data produced on diagnostic performance in cytopathology. The levels of agreement between individual diagnoses and consensus diagnoses were highest for slides in the "negative" and "positive" categories and somewhat less for slides in the "suspicious" categories. Submission of slides with 70% or less agreement and slides on which a consensus had not been reached to a new group of observers yielded similar levels of agreement, indicating that there is a fairly small percentage of cervical smears that are diagnostic problems. Agreement levels, especially in relation to false negatives and false positives, represent a commendable achievement, especially considering that cervical cytology is a screening procedure and that "suspicious" smears result in further investigations. It is also important to recognize that additional clinical data and studies are often taken into consideration before definitive treatment is begun or additional follow-up smears are taken.