Intra‐ and inter‐observer reproducibility of <scp>CIN</scp>tec®<scp>PLUS</scp> in ThinPrep® cytology preparations

McMenamin, Mary; McKenna, Michael; McDowell, Andrew; Dawson, C.; McKenna, Raymond

doi:10.1111/cyt.12426

Cited by 13 publications

(32 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Nevertheless, our findings are in agreement with other studies in which additional training with dual stain interpretation resulted in increased concordance, 10,12,16,28 with a higher concordance among evaluators experienced in dual stain interpretation as compared to new evaluators ( κ = 0.74 vs κ = 0.50) 26 . Additionally, whereas some studies defined experts as evaluators with great experience in dual stain interpretation, 11 other studies defined experts as evaluators with great experience in cytology interpretation 10 or p16 single immunostaining 27 . These differences in how an “expert” was defined may have affected level of concordance across studies, particularly because the “expert” is typically used as a reference.…”

Section: Discussionsupporting

confidence: 91%

“…Our results are in line with some studies reporting a good concordance in dual stain interpretation between new and experienced evaluators, ranging from a kappa value of 0.61‐0.70 10,26 . Another study reported a moderate concordance ( κ = 0.49), 16 while two studies reported a very good concordance ( κ = 0.82‐0.89) 11,27 . The underlying reason for these differences across studies is unclear, but may be explained by differences in characteristics of the study cohort, including age of the women, and whether p16/Ki67 was performed as triage of HPV‐positive women, among women with abnormal cytology, or in a population of women referred to colposcopy.…”

Section: Discussionsupporting

confidence: 89%

“…In the present study, discordance in dual stain results was mostly due to slides being categorized as positive by novice evaluators, but deemed negative by the expert, a finding also replicated by others 27 . In some cases, we found that the disagreement was explained by strong background staining making it difficult to decide if the cell should be scored dual stain positive, an issue raised previously 12 …”

Section: Discussionsupporting

confidence: 82%

“…Our study suggests that a significant amount of training may be needed to achieve an agreement of ≥90%, as recommended by the manufacturer. Agreement exceeded the recommended threshold after attending a second training course and after reviewing more than 500 slides, a number that is well above that reviewed by evaluators in most previous studies assessing interobserver variation 11,12,26,27 . This may be due to the fact that we, as opposed to most previous studies, only included slides from older women that may be more difficult to interpret because of cellular atrophy.…”

Section: Discussionmentioning

confidence: 64%

See 3 more Smart Citations

Implementation of p16/Ki67 dual stain cytology in a Danish routine screening laboratory: Importance of adequate training and experience

et al. 2020

View full text Add to dashboard Cite

Background Immunocytochemical staining with p16/Ki67 has been suggested as a promising triage biomarker in cervical cancer screening. As dual staining is a subjective method, proper training may be required to ensure safe implementation in routine laboratories and reduce risk of misclassification. We determined concordance between novice evaluators and an expert, stratified by number of slides reviewed at three reading points. Methods The study was conducted at the Department of Pathology, Randers, Denmark. Women were eligible if they were aged ≥45, had been enrolled in one of two ongoing clinical studies, and had a dual stain slide available. Dual staining was performed using the CINtec plus assay. Slides were randomly selected from three reading points at which novice evaluators had reviewed <30, ~300, and ≥500 dual stain slides respectively. Level of concordance was estimated using Cohen's Kappa, κ. Results Of 600 eligible slides, 50 slides were selected for review as recommended by the manufacturer. Median age was 68 years (range: 58‐74). Overall concordance was good (κ = 0.68, 95% confidence interval [CI]: 0.60‐0.76), with an overall agreement of 84% (95% CI: 70.9%‐92.8%). Concordance improved with increasing number of slides reviewed at a given reading point, from a moderate concordance (κ = 0.47, 95% CI: 0.05‐0.90) after reviewing <30 slides to a good concordance (κ = 0.66, 95% CI: 0.20‐0.88) and a very good concordance (κ = 0.88, 95% CI: 0.66‐1.00) after reviewing ~300 and ≥500 slides, respectively. Conclusions When interpreting dual stain slides from older women, concordance increased slightly as novice evaluators received more training and experience. Although further evaluation is warranted, these findings indicate that a significant amount of training and experience of novice evaluators may be needed to ensure accurate dual stain interpretation in this age group. Future studies should accurately describe training and experience of evaluators to enable a better comparison of concordance and diagnostic accuracy across studies. Trial registration NCT04114968 and NCT04298957.

show abstract

Section: Discussionsupporting

confidence: 91%

Section: Discussionsupporting

confidence: 89%

Section: Discussionsupporting

confidence: 82%

Section: Discussionmentioning

confidence: 64%

See 2 more Smart Citations

Implementation of p16/Ki67 dual stain cytology in a Danish routine screening laboratory: Importance of adequate training and experience

et al. 2020

View full text Add to dashboard Cite

show abstract

“…Authors have used different training approaches in studies investigating inter-observer reproducibility and accuracy of DS. [1][2][3][4][5][6] Most training protocols described in these studies consisted of initial and additional training. The initial training was provided by the manufacturer, however, it was not exactly the same in all cases except that it was completed by a proficiency test.…”

Section: Introductionmentioning

confidence: 99%

Evaluation of the training program for p16/ Ki-67 dual immunocytochemical staining interpretation for laboratory staff without experience in cervical cytology and immunocytochemistry

Prevodnik

Marinsek

Zalar

et al. 2020

Radiology and Oncology

View full text Add to dashboard Cite

Backgroundp16/Ki-67 dual immunocytochemical staining (DS) is considered easy to interpret if evaluators are properly trained, however, there is no consensus on what constitutes proper training. In the present study we evaluated a protocol for teaching DS evaluation on students inexperienced in cervical cytology.MethodsInitial training on 40 DS conventional smears was provided by a senior cytotechnologist experienced in such evaluation. Afterwards, two students evaluated 118 cases. Additional training consisted mainly of discussing discrepant cases from the first evaluation and was followed by evaluation of new 383 cases. Agreement and accuracy of students’ results were compared among the participants and to the results of the reference after both evaluations. We also noted time needed for evaluation of one slide as well as intra-observer variability of the teacher’s results.ResultsAt the end of the study, agreement between students and reference was higher compared to those after initial training (overall percent agreement [OPA] 81.4% for each student, kappa 0.512 and 0.527 vs. OPA 78.3% and 87.2%, kappa 0.556 and 0.713, respectively). However, accuracy results differed between the two students. After initial training sensitivity was 4.3% points and 2.9% points higher, respectively compared to the reference, while specificity was 30.6% points and 24.4% points lower, respectively, compared to the reference. At the end of the study, the sensitivity reached by one student was the same as that of the reference, while it was 2.6% points lower for the other student. There was a statistically significant difference in specificity between one student and the reference and also between students (16.7 and 15.1% points). Towards the end of the study, one student needed 5.2 min for evaluating one slide while the other needed 8.2 min. The intra-observer variability of the senior cytotechnologist was in the range of “very good” in both arms of the study.ConclusionsIn teaching DS evaluation, the students’ progress has to be monitored using several criteria like agreement, accuracy and time needed for evaluating one slide. The monitoring process has to continue for a while after students reach satisfactory results in order to assure a continuous good performance. Monitoring of teacher’s performance is also advisable.

show abstract

Interlaboratory concordance of p16/Ki‐67 dual‐staining interpretation in HPV‐positive women in a screening population

Benevolo

Mancuso

Allìa

et al. 2020

Cancer Cytopathology

View full text Add to dashboard Cite

BACKGROUND: p16/Ki-67 dual staining is a candidate biomarker for triaging human papillomavirus (HPV)-positive women.Reproducibility is needed for adopting a test for screening. This study assessed interlaboratory reproducibility in HPVpositive women. METHODS: All women positive for HPV from the Italian New Technologies for Cervical Cancer 2 study, were included in this study. ThinPrep slides were immunostained for p16/Ki-67 in 4 laboratories and were interpreted in 7 laboratories. Each slide had 3 reports from different laboratories. Slides were classified as valuable or inadequate, and valuable slides were classified as positive (at least 1 double-stained cell) or negative. Interlaboratory reproducibility was evaluated with κ values. RESULTS: Overall, we obtained 9300 reports for 3100 cases; 905 reports (9.7%) were inadequate. The overall adequacy concordance was poor (κ = 0.224; 95% confidence interval [CI], 0.183-0.263). The overall positivity concordance was moderate (κ = 0.583; 95% CI, 0.556-0.610). Of the 176 cervical intraepithelial neoplasia 2+ (CIN-2+) lesions found in HPV DNApositive women, 158 had a valid result: 107 were positive in all 3 reports (sensitivity for CIN-2+, 67.7%; 95% CI, 59.8%-74.9%), 23 were positive in 2 reports (sensitivity of the majority report, 82.3%; 95% CI, 75.4%-87.9%), and 15 were positive in 1 report (sensitivity of at least 1 positive result, 91.8%; 95% CI, 86.3%-95.5%). Thirteen CIN-2+ cases were negative in all 3 reports.The overall positivity concordance in CIN-2+ samples was κ = 0.487 (95% CI, 0.429-0.534), whereas in the non-CIN-2+ samples, it was κ = 0.558 (95% CI, 0.528-0.588). CONCLUSIONS: The p16/Ki-67 assay showed poor reproducibility for adequacy and good reproducibility for positivity comparable to that of cervical cytology. Nevertheless, the low reproducibility does not affect the sensitivity for CIN-2+.

show abstract

Intra‐ and inter‐observer reproducibility of CINtec^®PLUS in ThinPrep^® cytology preparations

Cited by 13 publications

References 23 publications

Implementation of p16/Ki67 dual stain cytology in a Danish routine screening laboratory: Importance of adequate training and experience

Implementation of p16/Ki67 dual stain cytology in a Danish routine screening laboratory: Importance of adequate training and experience

Evaluation of the training program for p16/ Ki-67 dual immunocytochemical staining interpretation for laboratory staff without experience in cervical cytology and immunocytochemistry

Interlaboratory concordance of p16/Ki‐67 dual‐staining interpretation in HPV‐positive women in a screening population

Contact Info

Product

Resources

About

Intra‐ and inter‐observer reproducibility of CINtec®PLUS in ThinPrep® cytology preparations

Cited by 13 publications

References 23 publications

Implementation of p16/Ki67 dual stain cytology in a Danish routine screening laboratory: Importance of adequate training and experience

Implementation of p16/Ki67 dual stain cytology in a Danish routine screening laboratory: Importance of adequate training and experience

Evaluation of the training program for p16/ Ki-67 dual immunocytochemical staining interpretation for laboratory staff without experience in cervical cytology and immunocytochemistry

Interlaboratory concordance of p16/Ki‐67 dual‐staining interpretation in HPV‐positive women in a screening population

Contact Info

Product

Resources

About

Intra‐ and inter‐observer reproducibility of CINtec^®PLUS in ThinPrep^® cytology preparations