Cumulative sum procedure in evaluation of EUS-guided FNA cytology: the learning curve and diagnostic performance beyond sensitivity and specificity

Background: Using cumulative sum (CUSUM) chart, we address two questions: (i) Over time, how will an EUS‐FNA (endoscopic ultrasound guided fine needle aspiration) service maintain an acceptable non‐diagnostic rate defined as technical failures, unsatisfactory specimens and atypical and suspicious diagnoses? (ii) Over time, how will EUS‐FNA maintain acceptable diagnostic errors (false‐positives plus false‐negative diagnosis)? Methods: The study included all consecutive patients who underwent EUS‐FNA at our institution from July 2000 to October 2003 and were followed up until December 2004. Using a simple spread sheet, we designed CUSUM charts and used them to track trends and assess performance at a preset acceptable rate of 10% and a preset unacceptable rate of 15% for non‐diagnostic rate and diagnostic errors. We assessed all cases collectively and then in groups defined by site, size and cytopathologist. Results: Of 876 patients undergoing EUS‐FNA, 83 (9.5%) had non‐diagnostic results: 43 (51%) of these diagnoses were ‘atypical’, 27(33%) were ‘suspicious for malignancy’, eight (10%) were ‘insufficient material for diagnosis’ and five (6%) were ‘technical failure’. In 585 cases with adequate follow up, there were 26 (6.3%) diagnostic errors: three (0.5%) were false positive and 23 (3.1) were false negative. The overall CUSUM charts for both non‐diagnostic rate and for diagnostic error rate start with a small period of learning then cross to a significantly acceptable level at case numbers 121 and 97 respectively. Our diagnostic performance was better in lymph nodes than in the pancreas and other organs and was not significantly different for lesions ≤25 mm compared with lesion >25 mm in diameter. Performance was better for pathologists with prior experience than for pathologists without experience. Conclusion: In the current climate of proficiency testing, error tracking and competence evaluation, there is a great potential for the use of CUSUM charts to assess procedure failure and error tracking in quality control programs, particularly when a new procedure such as EUS‐FNA is introduced in the laboratory. Additionally, the method can be used to assess trainee competency and to track the proficiency of practicing cytologists.