—  SYMPOSIUM #13  —

Patient Safety in Anatomic Pathology
Moderator: Peter Furness

Section 4 - U.S. National Safety Databases in Anatomic Pathology

Stephen S. Raab
University of Pittsburgh School of Medicine
Department of Pathology
Pittsburgh , PA USA


In 1999, the Institute of Medicine (IOM) reported on the extent of medical error. The IOM defined medical error to be a failure of a planned action to be completed as intended or the use of the wrong plan to achieve an aim. Medical errors are found at all levels of patient care, and the IOM reported on means of reducing error and improving patient safety. Researchers have directed more of an effort studying some error types, such as action-related errors (e.g., medication errors in which a wrong dose or a wrong drug is administered) compared to studying other error types, such as diagnostic errors.

Pathology errors are detected through several methods, and the pathology community has not reached consensus on the optimal methods for error detection. Anatomic pathology laboratories use a number of quality assurance methods that may be used as methods of error detection. For surgical pathology and cytology, these methods include audit systems, benchmarking systems, and immediate error reduction systems (e.g., Lean Production System, Toyota Production System, or Six Sigma). In anatomic pathology, most research has focused on errors in diagnostic interpretation, although the literature would indicate that this is a small percentage of the source of all error. Detection of interpretation error usually takes the form of secondary review by one or more pathologists. Methods of secondary review include review of all specimens or a subgroup of specimens, review of a fixed percentage of cases, review of all discrepancies, review of cases presented at conferences, and review of cases through consultative services. Most of the diagnostic interpretation error data reported in the pathology literature are from studies of cases reviewed post-sign out, although most anatomic pathology laboratories perform some form of pre-sign out secondary review. Little is known about the benefits of pre-sign out review. In general, pathology error analysis to date simply has documented errors without using methods to control and limit errors and improve patient outcomes.

Because anatomic pathology errors occur relatively infrequently, a more accurate analysis depends on the collection of data across multiple institutions. Currently in the United States, there are two major efforts examining anatomic pathology errors and evaluating practice patterns through database research. The first is an effort by the College of American Pathologists (CAP) and the second is an effort funded by the Agency for Healthcare Research and Quality (AHRQ).

College of American Pathologists Error Reduction Initiatives
The CAP evaluates medical error using two methods: 1) the Q-PROBES program that looks at quality indicators at multiple hospitals at a fixed point in time, and 2) the Q-TRACKS program that looks at quality indicators at multiple institutions over a period of time.

The Q-PROBES program has measured and defined a number of key quality indicators, including patient safety indicators, in anatomic and clinical pathology. In anatomic pathology, Raab et al reported Q-PROBES program data on the frequency of anatomic pathology discrepancies and the causes of these discrepancies. [44] In a Q-PROBES monitor, 74 American laboratories self-reported the number of anatomic pathology discrepancies in their laboratory by prospectively performing secondary review (post sign out) of 100 surgical pathology or cytology specimens. Reasons for the secondary review included external review by consultants, departmental and hospital conferences, internal quality assurance policies (e.g., cytologic-histologic correlation), and physician request. The main outcome measures were the frequency of anatomic pathology discrepancy, type of discrepancy (i.e., change in margin status, change in diagnosis, change in patient information, or typographical error), effect of discrepancy on patient outcome (i.e., no harm, near miss, or harm), and clarity of report.

The laboratories reviewed 6,186 specimens and reported 415 discrepancies. The mean and median laboratory discrepancy frequency was 6.7% and 5.1%, respectively. Forty-eight percent of all discrepancies were due to a change within the same category of interpretation (e.g., one tumor type was changed to another tumor type). Twenty-one percent of all discrepancies were due to a change across categories of interpretation (e.g., a malignant diagnosis was changed to a benign diagnosis). Of the remaining discrepancies, 48% resulted in a change in the same category of diagnosis (e.g., squamous cell carcinoma to adenocarcinoma), 18% were typographical errors, 4% resulted in a change in margin status, and 9% resulted in a change in patient or specimen information. Participants estimated that the majority of discrepancies had no effect on patient care, although 5.3% had a moderate or marked effect on patient care. It is important to note that outcome assessment was performed at the discretion of the laboratory, and most laboratories did not perform chart review.

Zarbo et al reported the findings from two CAP anatomic pathology Q-TRACKS Ò monitors that evaluated errors over time. The first monitored gynecologic cytologic-histologic correlation data and the second monitored frozen section-permanent section correlation data. The gynecologic correlation Q-TRACKS Ò monitor is currently active, and the frozen section discrepancy monitor has been discontinued. In the Q-TRACKS Ò program, laboratories send data every quarter and these data are stored in a database and analyzed to determine changes in particular metrics over time. Laboratories are benchmarked by specific metrics, and at the end of the year, the CAP reports "best practices" characteristics of laboratories who have the best (or most improved) quality metrics.

One hundred seventy-four laboratories self-reported data in the frozen section Q-TRACKS Ò monitor and the mean frozen-permanent section discordant and deferred diagnostic frequencies and changes in these frequencies were recorded over time. Raab et al reported that the mean and median frozen-permanent section discordant frequencies were 1.36% and 0.70%, respectively. [56] Longer participation in the Q-TRACKSÒ program was significantly associated (P = .0401) with lower discordant frequencies; 4 or 5 year participation showed a decrease in discordant frequency of 0.99% whereas 1 year participation showed a decrease in discordant frequency of 0.84%. Longer participation in the Q-TRACKSÒ monitor was associated with lower microscopic sampling frequencies for discordant diagnoses (P = .0351). The mean and median deferred diagnostic frequencies were 2.35% and 1.20%, respectively. Increased length of participation in the Q-TRACKSÒ program was significantly associated (P = .0437) with lower deferred diagnostic frequencies. Raab et al concluded that the long-term monitoring of frozen-permanent section correlation is associated with sustained improvement in performance.

Agency for Healthcare Research and Quality Initiative
The NIH has funded a number of patient safety initiatives involving subspecialty areas such as pharmacy, primary care, anesthesiology, internal medicine, and surgery. In 2002, the Agency for Healthcare Research and Quality (AHRQ) (a branch of the NIH) funded four pathology departments to study anatomic pathology-detected errors. [43] The specific aims are to: 1) determine baseline anatomic pathology error frequencies using a Web-based database, 2) determine the clinical impact of anatomic pathology diagnostic errors using patient outcome information, 3) perform root cause analysis to determine the cause of these anatomic pathology errors, 4) devise error reduction strategies based on the root cause analysis, and 5) assess the success of these error reduction strategies using both quantitative and qualitative measures. This project is focused on determining how anatomic pathology quality assurance practices may be used to change laboratory and/or clinical practice to reduce pathology-detected errors.

A different error detection method has been added in each year of the project. In 2002, the pathology departments began collecting errors detected by the cytologic-histologic correlation process. The pathology departments used the Year 2002 data to establish cytologic-histologic correlation error frequencies, causes, and outcomes. In 2003, the pathology departments began collecting error data based on review of amended reports, and in 2005, the pathology departments began collecting error data based on frozen section-permanent section correlation.

Standardization of cytologic-histologic correlation review process
In the beginning of the project, the labs standardized the cytologic-histologic error data collection process. On a monthly basis, a cytotechnologist used an existing laboratory information system program to identify all patients who had both cytology and surgical specimens from the same anatomic site that had been obtained within 6 months of each other prior to the date of review. A site-specific pathologist selected cases in which the cytologic and surgical specimens were discrepant. The cytotechnologist reviewed the slides and reports and generated a hard-copy review sheet, and the review pathologist examined the material and determined the cause of error.

Definition of cytologic-histologic error and cause
The consortium of laboratories defined a discrepancy as a difference between the cytologic and histologic diagnoses. Because cytology and surgical diagnostic schema are somewhat different, these laboratories considered the diagnoses in a scaled categorical context in order to determine if a discrepancy occurred. The categorical context was different if the specimens were gynecologic (e.g., Pap test and cervical biopsy) or non-gynecologic (e.g., thyroid gland fine needle aspiration and thyroid gland excision). These laboratories defined a cytologic-histologic correlation error as at least a 2-step discrepancy. These laboratories evaluated only 2-step or greater cytologic-histologic correlation discrepancies, because of the lack of reproducibility and clinical import of 1-step discrepancies. For example, a diagnostic error occurred if a patient's thyroid gland fine needle aspiration specimen was diagnosed as papillary carcinoma and the patient's thyroid gland excisional specimen was diagnosed as lymphocytic thyroiditis.

The site-specific pathologist microscopically examined all slides and determined if the cytology, surgical, both or neither diagnosis was in error. The site-specific pathologist then assigned a "cause" of the error, using the categories of interpretation, sampling, or both. An interpretation error was an error in disease classification, and this error was further classified as a false positive (or an overcall) or a false negative (or an undercall). A sampling error was an error in which the diagnostic material was not present on the slide. Using the above example, if the site-specific pathologist concurred with the original thyroid gland fine needle aspiration diagnosis and disagreed with the thyroid gland excisional diagnosis, an interpretive error occurred on the surgical pathology specimen.

In one study, the institutions analyzed one year of retrospective errors detected through a the standardized cytologic-histologic correlation process and medical record reviews were performed to determine patient outcomes. [52] The researchers evaluated institutional frequency, cause (i.e., pathologist interpretation or sampling), and clinical impact of diagnostic cancer errors. The frequency of errors in cancer diagnosis was institution dependent (P < .001) and ranged from 1.79% to 9.42% and from 4.87% to 11.8% of all correlated gynecologic and non-gynecologic cases, respectively. A statistically significant association existed between institution and error cause (P < .001); the cause of errors due to pathologic misinterpretation ranged from 5.0% to 50.7% (the remainder due to clinical sampling). A statistically significant association existed between institution and assignment of the clinical impact of error (P < .001); the aggregated data showed that for gynecologic and non-gynecologic errors, 45% and 39%, respectively, were associated with harm. The pairwise kappa statistic for interobserver agreement on cause of error ranged from 0.118 to 0.737.

The data have been used in a number of initiatives to design and implement interventions to reduce error frequency.

For example, the researchers attempted to lower errors in cervical cancer screening by using the Toyota Production System (TPS) process to redesign practice in a clinical office and the cytology lab. [54] The researchers performed an eight month non-concurrent cohort study that included 464 case and 639 control women who had a Pap test. They redesigned office workflow using TPS methods by introducing a one-by-one continuous flow process. They measured the frequency of Pap tests without a transformation zone component, follow-up and Bethesda System diagnostic frequency of atypical squamous cells – undetermined significance (ASC-US), and diagnostic error frequency. After the intervention, the percentage of Pap tests lacking a transformation zone component decreased from 9.9% to 4.7% (P = .001). The percentage of Pap tests with a diagnosis of ASC-US decreased from 7.8% to 3.9% (P = .007). The frequency of error per correlating cytologic-histologic specimen pair decreased from 9.52% to 7.84%. The researchers concluded that the introduction of the TPS process resulted in improved Pap test quality.

In another study, three project sites performed pre-sign out double viewing of pulmonary cytology cases (n=431). Two-step or more differences in diagnosis were arbitrated as interpretive errors and the effect of double viewing was measured by comparing the frequency of cytologic-histologic correlation-detected errors in the previous 2 years to the double viewing period. The number of interpretive errors detected by double viewing for the three institutions was 2.7%, 0% and 1.9%, respectively. Double viewing did not lower the frequency of cytologic-histologic correlation false negative errors. The authors concluded that double viewing detected errors in up to 1 of every 37 cases and that biases in the double viewing process limited error detection.

Additional interventions will be discussed in the presentation.

References
  1. Clary JM, Silverman JF, Liu Y, et al. Cytohistologic discrepancies: a means to improve pathology practice and patient outcomes. Am J Clin Pathol. 2002;117:567-573

  2. Raab SS, Nakhleh RE, Ruby SG. Patient safety in anatomic pathology: measuring discrepancy frequencies and causes. Arch Pathol Lab Med. (in press).

  3. Bruner JM, Inouye L, Fuller GN, Langford LA. Diagnostic discrepancies and their clinical impact in a neuropathology referral practice. Cancer. 1997;79:796-803.

  4. Furness PN, Lauder I. A questionnaire-based survey of errors in diagnostic histopathology throughout the United Kingdom. J. Clin Pathol. 1997;50:457-460.

  5. Raab SS. Improving patient safety by examining pathology errors. Clin Lab Med. 2004;24:849-863.

  6. Whitehead ME, Fitzwater JF, Lindley SK , Kern SB, Ulirsch, RC, Winecoff WF 3rd. Quality assurance of histopathologic diagnoses: a prospective audit of three thousand cases. Am J Clin Pathol. 1984;81:487-491.

  7. Zardawi IM, Bennett G, Jain S, Brown M. Internal quality assurance activities of a surgical pathology department in an Australian teaching hospital. J Clin Pathol. 1998;51:695-699.

  8. Zuk JA, Kenyon WE, Myskow MW. Audit in histopathology: description of an internal quality assessment scheme with analysis of preliminary results. J Clin Pathol. 1991;44:10-15.

  9. Wakely SL, Baxendine-Jones, Gallagher PJ, Mullee M. Pickering R. Aberrant diagnoses by individual surgical pathologists. Am J Surg Pathol. 1998;22:77-82.

  10. Safrin RE, Bark CJ. Surgical pathology signout. Routine review of every case by a second pathologist. Am J Surg Pathol. 1993;17:1190-1192.

  11. Renshaw AA. Measuring and reporting errors in surgical pathology. Lessons from gynecologic cytology. Am J Clin Pathol. 2001;115:338-341.

  12. Ramsay AD, Gallagher PJ. Local audit of surgical pathology. 18 month's experience of peer-review-based quality assurance in an English teaching hospital. Am J Surg Pathol. 1992;16:476-482.

  13. Jones BA, Novis DA. Cervical biopsy-cytology correlation: a College of American Pathologists Q-Probes study of 22,439 correlations in 348 laboratories. Arch Pathol Lab Med. 1996;120:523-531.

  14. Zarbo RJ, Hoffman GG, Howanitz PJ. Interinstitutional comparison of frozen section consultation: A College of American Pathologists Q-Probes study of 79,647 consultations in 297 North American institutions. Arch Pathol Lab Med. 1991;115:1187-1194.

  15. Department of Health and Human Services, Health Care Financing Administration. Clinical laboratory improvement amendments of 1988: final rule, 57 Federal Register 7146 (1992) (codified at 42 CFR §493).

  16. Jones BA, Novis DA. Follow-up of abnormal gynecologic cytology: a College of American Pathologists Q-Probes study of 16,132 cases from 306 laboratories. Arch Pathol Lab Med. 2000;124:665-671.

  17. Joste NE, Crum CP, Cibas ES. Cytologic/histologic correlation for quality control in cervicovaginal cytology. Experience with 1,582 paired cases. Am J Clin Pathol. 1995;103:32-34.

  18. Kohn LT, Corrigan JM, Donaldson MS. Eds. To err is human: building a safer health system. Washington , DC : National Academy Press: 1999.

  19. Battles JB, Kaplan HS, Van der Schaaf TW, Shea CE. The attributes of medical event reporting systems: experience with a prototype medical event reporting system for transfusion medicine. Arch Pathol Lab Med 1998;122:231-238.

  20. Dovey SM, Meyers, DS, Phillips RL Jr, et al. A preliminary taxonomy of medical errors in family practice. Qual Saf Health Care. 2002;11:233-238.

  21. Rubin G, George A, Chinn DJ, Richardson C. Errors in general practice: development of an error classification and pilot study of a method for detecting errors. Qual Saf Health Care. 2003;12:443-447.

  22. Fernald DH, Pace WD, Harris DM, West DR, Main DS, Westfall JM. Event reporting to a primary care patient safety reporting system: a report from the ASIPS collaborative. Ann Fam Med. 2004;2:327-332.

  23. O'Sullivan JP. Observer variation in gynaecological cytopathology. Cytopathology. 1998;9:6-14.

  24. Llewellyn H. Observer variation, dysplasia grading, and HPV typing: a review. Am J Clin Pathol. 2000;114:S21-35.

  25. Schlemper RJ, Kato Y, Stolte M. Review of histological classifications of gastrointestinal epithelia neoplasia: differences in diagnosis of early carcinomas between Japanese and Western pathologists. Gastroenterol. 2001;36:445-456.

  26. Carlson GD, Calvanese CB, Kahane H, Epstein JI. Accuracy of biopsy Gleason scores from a large uropathology laboratory: use of a diagnostic protocol to minimize observer variability. Urology. 1998;51:525-529.

  27. Dalton LW, Page DL, Dupont WD. Histologic grading of breast carcinoma. A reproducibility study. Cancer. 1994;73:2765-2770.

  28. Pap Test. Primary Care Consultants. Availalble at http://www.pccdocs.com. Accessed February 3, 2005 .

  29. Mason DJ. Who says it's an error? Research highlights a disagreement among health care workers. Am J Nurs. 2004;104:7.

  30. Birkmeyer JD, Sharp SM, Finlayson SR, Fisher ES, Wennberg JE. Variation profiles of common surgical procedures. Surgery. 1998;124:917-923.

  31. Garg PP, Landrum MB, Normand SL, et al. Understanding individual and small area variation in the underuse of coronary angiography following acute myocardial infarction. Med Care. 2002;40:614-626.

  32. Carlisle DM, Valdez RB, Shapiro MF, Brook RH. Geographic variation in rates of selected surgical procedures within Los Angeles County . Health Serv Res. 1995;30:27-42.

  33. Wrobel JS, Mayfield JA, Reiber GE. Geographic variation of lower-extremity major amputation in individuals with and without diabetes in the Medicare population. Diabetes Care. 2001;24:860-864.

  34. McPherson K, Wennberg JE, Hovind OB , Clifford P. Small-area variations in the use of common surgical procedures: an international comparison of New England , England , and Norway. N Engl J Med. 1982;307:1310-1314.

  35. Fisher ES, Wennberg JE. Health care quality, geographic variations, and the challenge of supply-sensitive care. Perspect Biol Med. 2003;46:69-79.

  36. Banja JD. Medical errors and medical narcissism. Boston , MA : Jones & Bartlett Publishers:2005

  37. Fahey MT , Irwig L, Macaskill P. Meta-analysis of Pap test accuracy. Am J Epidemiol. 1996;143:406-407.

  38. Nanda K, McCrory DC, Myers ER, et al. Accuracy of the Papanicolaou test in screening for and follow-up of cervical cytologic abnormalities: a systematic review. Ann Intern Med. 2000;132:810-819.

  39. Page DL, Dupont WD, Jensen RA, Simpson JF. When and to what end do pathologists agree? J Natl Cancer Inst. 1998;90:88-89.

  40. Schnitt SJ, Connolly JL, Tavassoli FA, et al. Interobserver reproducibility in the diagnosis of ductal proliferative breast lesions using standardized criteria. Am J Surg Pathol. 1992 16:1133-1143.

  41. Becich MJ, Gilbertson JR, Gupta D, Patel A, Grzybicki DM, Raab SS. Pathology and patient safety: the critical role of pathology informatics in error reduction and quality initiatives. Clin Lab Med 2004;24:913-943.

  42. Condel JR, Sharbaugh DT, Raab SS. Error-free pathology: applying lean production methods in anatomic pathology. Clin Lab Med 2004;24:865-899.

  43. Raab SS. Improving patient safety by examining pathology errors. Clin Lab Med 2004;24:849-863.

  44. Raab SS, Nakhleh RE, Ruby SG. Patient safety in anatomic pathology: measuring discrepancy frequencies and causes. Arch Pathol Lab Med 2005;129:459-466.

  45. Raab SS, Grzybicki DM, Zarbo RJ, Meier F, Geyer SJ, Jensen C. Anatomic pathology databases and patient safety. Arch Pathol Lab Med 2005;129:1246-1251.

  46. Zarbo RJ, Meier FA, Raab SS. Error detection in anatomic pathology. Arch Pathol Lab Med 2005;129:1237-1245.

  47. Dhir R, Condel JL, Raab SS. Identification and correction of errors in the anatomic pathology gross room. Pathol Case Rev 2005;10:79-82.

  48. Condel JL, Jukic DM, Sharbaugh DT, Raab SS. Histology errors: use of real-time root cause analysis to improve practice. Pathol Case Rev 2005;10:82-87.

  49. Parwani AV, Raab SS. Amended report resulting from an interpretation error: a case report and review of the literature. Pathol Case Rev 2005;10:74-78.

  50. Vrbin CM, Grzybicki DM, Zaleski MS, Raab SS. Variability in cytologic-histologic correlation practices and implications on patient safety. Arch Pathol Lab Med 2005;129:893-898.

  51. Grzybicki DM, Turcsanyi B, Becich MJ, Gupta D, Gilbertson JR, Raab SS. Database construction for improving patient safety by examining pathology errors. Am J Clin Pathol 2005;124:1-10.

  52. Raab SS, Grzybicki DM, Janosky JE, Zarbo RJ, Meier FA, Jensen C, Geyer SJ. Clinical impact and frequency of anatomic pathology errors in cancer diagnosis. Cancer 2005;104:2205-2213.

  53. Nodit L, Balassanian R, Sudilovsky D, Raab SS. Improving the quality of cytology diagnosis: root cause analysis for errors in bronchial washing and brushing specimens. Am J Clin Pathol 2005;124:883-893.

  54. Raab SS, Andrew-Jaja C, Condel JL, Dabbs DJ. Improving Pap test quality and reducing medical errors by using Toyota Production System methods. Am J Obstet Gyn 2006;194:57-64.

  55. Raab SS, Stone CH, Jensen CS, Zarbo RJ, Meier F, Grzybicki DM, Vrbin CM, Ohori NP, Dahmoush L. Double slide viewing as a cytology quality improvement initiative. Am J Clin Pathol 2006;125:526-533.

  56. Raab SS, Tworek JA, Souers R, Zarbo RJ. The value of monitoring frozen-permanent section correlation data over time. Arch Pathol Lab Med 2006;130:337-342.

  57. Raab SS, Vrbin CM, Grzybicki DM, Sudilovsky D, Balassanian R, Zarbo RJ, Meier FA. Errors in thyroid gland fine needle aspiration. Am J Clin Pathol 2006:125:873-882.

  58. Raab SS, Meier FA, Zarbo RJ, Jensen C, Geisinger KR, Booth CN, Krishnamurti U, Stone CH, Janosky JE, Grzybicki DM. The "Big Dog" effect: variability in assessing the causes of error in patients with lung cancer. J Clin Oncol 2006;24:2808-2814.