Interobserver agreement in dysplasia grading: toward an enhanced gold standard for clinical pathology trials

Abstract

Objective Inter-observer agreement in the context of oral epithelial dysplasia (OED) grading has been notoriously unreliable and can impose barriers for developing new molecular markers and diagnostic technologies. This paper aimed to report the details of a 3-stage histopathology review and adjudication process with the goal of achieving a consensus histopathologic diagnosis of each biopsy. Study Design Two adjacent serial histological sections of oral lesions from 846 patients were independently scored by two different pathologists from a pool of four. In instances where the original two pathologists disagreed, a third, independent adjudicating pathologist conducted a review of both sections. If a majority agreement was not achieved, the third stage involved a face-to-face consensus review. Results Individual pathologist pair kappa values ranged from 0.251 – 0.706 (fair – good) before the 3-stage review process During the initial review phase, the two pathologists agreed on a diagnosis for 69.9% of the cases. After the adjudication review by a third pathologist, an additional 22.8% of cases were given a consensus diagnosis (agreement of 2 out of 3 pathologists). Following the face-to-face review, the remaining 7.3% of cases had a consensus diagnosis. Conclusion The use of the defined protocol resulted in a substantial increase (30%) in diagnostic agreement and has the potential to improve the level of agreement for establishing gold standards for studies based on histopathologic diagnosis.

Algorithm to reduce subjectivity in grading oral epithelial dysplasia - A preliminary study

“…[4] Malignant transformation depends on the type, site, and the microscopic picture. [5] situ. Subjectivity in grading OED has been well documented.…”

“…In the present study, the WHO grading system alone was considered, and only interobserver variability was checked for. In a study conducted by Speight et al, [5] a 3 stage review process was performed. Fair to good agreement was procured by an individual oral pathologist.…”

‘Cytology-on-a-chip’ based sensors for monitoring of potentially malignant oral lesions

“…Longitudinal studies would be necessary to validate the malignant potential of these identified lesions. Histopathological grading has been shown to behave similarly [21], where higher accuracy and inter-observer agreement have been reported for high grade dysplastic or malignant lesions than for low grade dysplastic lesions, where considerable overlap from inflammatory and reactive changes can confound grading systems [35]. However, longitudinal studies are necessary to validate the malignant potential of these identified lesions.…”

“…Informed consent was obtained for all participants in the trial after the possible consequences of the procedures were explained. The study design and clinical protocol for this study have been reported previously in detail [21]. Briefly, lesion samples from a total of 714 patients were measured, of which 85 were previously diagnosed malignant cases.…”

“…These comprised 348 benign, 49 mild dysplasia, 18 moderate dysplasia, 12 severe dysplasia, 2 carcinoma in situ (CIS), and 135 malignant lesions in addition to 150 healthy controls. To obtain greater confidence in the gold-standard pathological diagnoses, which have been notoriously unreliable [22,23], a 3-stage adjudication and consensus review process was performed which achieved 100% consensus agreement from an initial 69.9% agreement rate between any two pathologists across all patient biopsy specimens [21]. …”