Observer performance study to examine the feasibility of the AI-powered PD-L1 analyzer to assist pathologists’ assessment of PD-L1 expression using tumor proportion score in non–small cell lung cancer.

Abstract

8529 Background: Programmed death ligand 1 (PD-L1) expression is the standard biomarker for PD-L1 inhibitors in advanced non-small cell lung cancer (NSCLC). However, evaluation of PD-L1 tumor proportion score (TPS) by pathologists causes inter-observer variation and demands time to interpret. This study aimed to evaluate the benefit of the artificial intelligence (AI) algorithm in assisting pathologists to determine TPS on PD-L1 immunohistochemistry (IHC) whole-slide images (WSIs) in NSCLC. Methods: Lunit SCOPE PD-L1, an AI-powered PD-L1 TPS analyzer, was developed from 393,565 tumor cells annotated by board-certified pathologists for PD-L1 expression in 802 WSIs stained by 22C3 pharmDx IHC. The AI model was developed based on a region-based convolutional neural network, and the model can detect and count PD-L1 positive or negative tumor cells from WSIs to calculate TPS. Seven independent board-certified pathologists scored ground truth (GT) of PD-L1 TPS from 199 WSI of NSCLC stained by 22C3 pharmDx IHC. TPS from each GT reader was grouped as negative (< 1%), low (1% to 49%), or high (≥ 50%). The GT of each slide was determined by the consensus of GT readers. Another twelve independent board-certified pathologists scored PD-L1 TPS from the same WSIs as observer performance testers (OPT). They scored TPS twice with a washout interval of 4 weeks, with or without AI assistance. TPS accuracy change and reading time of OPT reader according to the presence or absence of AI assistance were analyzed. Results: The standalone accuracy of the AI model was 0.809 (95% CI: 0.690–0.941). With AI assistance, the overall accuracy of TPS had been changed from 0.799 (95% confidence interval [CI]: 0.764–0.836) to 0.832 (95% CI: 0.796–0.869) (P = 0.004). AI assistance increased the accuracy rate in 11 out of 12 OPT readers. The result of the generalized linear mixed model revealed that AI assistance and specimen type affected the probability of correct answer, while the order of reading did not (Table). The mean time to read with AI was 195.4±506.5 (mean±standard deviation) seconds, which was significantly shorter than the mean time to read without AI (285.1±1578.4, P <0.001). Conclusions: This study demonstrates that an AI-powered PD-L1 TPS analyzer can assist board-certified pathologists in evaluating TPS of NSCLC by improving the accuracy of TPS group evaluation and reducing the time to read slides.[Table: see text]