Interobserver and intraobserver variability in the cytologic diagnosis of normal and abnormal metaplastic squamous cells in pap smears

OBJECTIVE: Interoberver variability has important implications for patient care, diagnostic error and medical litigation. In the management of any cervical epithelial abnormality, its biologic significance as well as diagnostic reproducibility is very important. Interobserver variability has not been measured adequately for metaplastic squamous lesions. We analyzed interobserver and intraobserver variability and diagnostic accuracy in the diagnosis of dysplastic metaplastic cells. STUDY DESIGN: Sixty Pap smears from patients with abnormalities of metaplastic squamous cells of varying severity were selected from the files of Lankenau Hospital, Wynnewood, Pennsylvania, U.S.A., diagnosed between 1990 and 1996. These were reviewed by four observers with different levels of cytology experience. Each of the observers blindly and independently reviewed all Pap smears. Tabulated results were analyzed to determine interobserver and intraobserver variability and diagnostic accuracy. RESULTS: Statistically significant interobserver reproducibility was found between both inexperienced observers as well as between observers 1 (experienced) and 3 (inexperienced) and between observers 2 (experienced) and 4 (inexperienced). The observed degree of agreement between both experienced observers (1 and 2) reflected random rating rather than reproducibility. There was no difference in interobserver reproducibility in low vs. high grade lesions. Intraobserver reproducibility had no significant correlation with experience of the observer. The sensitivity ranged from 0.69 to 0.97 (mean, 0.79), while the specificity ranged from 0.09 to 0.46 (mean, 0.30). Mean diagnostic accuracy was better in benign and low grade squamous intraepithelial lesions in comparison to high grade squamous intraepithelial lesions. CONCLUSION: There was good interobserver agreement in classifying squamous metaplastic lesions. The agreement did not correlate with grade of dysplasia or experience of the cytopathologists. These findings should be considered in making treatment, quality assurance and legal decisions. A larger study is indicated to study interobserver and intraobserver variability and define cytologic criteria for lesions of metaplastic squamous cells.