A framework for evaluating diagnostic discordance in pathology discovered during research studies

Little is known about the frequency of discordant diagnoses identified during research. To describe diagnostic discordance identified during research and apply a newly designed research framework for investigating discordance. Breast biopsy cases (N = 407) from registries in Vermont and New Hampshire were independently reviewed by a breast pathology expert. The following research framework was developed to assess those cases: (1) compare the expert review and study database diagnoses, (2) determine the clinical significance of diagnostic discordance, (3) identify and correct data errors and verify the existence of true diagnostic discrepancies, (4) consider the impact of borderline cases, and (5) determine the notification approach for verified disagreements. Initial overall discordance between the original diagnosis recorded in our research database and a breast pathology expert was 32.2% (131 of 407). This was reduced to less than 10% after following the 5-step research framework. Detailed review identified 12 cases (2.9%) with data errors (2 in the underlying pathology registry, 3 with incomplete slides sent for expert review, and 7 with data abstraction errors). After excluding the cases with data errors, 38 cases (9.6%) among the remaining 395 had clinically meaningful discordant diagnoses (κ = 0.82; SE, 0.04; 95% confidence interval, 0.76-0.87). Among these 38 cases, 20 (53%) were considered borderline between 2 diagnoses by either the original pathologist or the expert. We elected to notify the pathology registries and facilities regarding discordant diagnoses. Understanding the types and sources of diagnostic discordance uncovered in research studies may lead to improved scientific data and better patient care.