PURPOSE Biomarkers on the basis of tumor-infiltrating lymphocytes (TIL) are potentially valuable in predicting the effectiveness of immune checkpoint inhibitors (ICI). However, clinical application remains challenging because of methodologic limitations and laborious process involved in spatial analysis of TIL distribution in whole-slide images (WSI). METHODS We have developed an artificial intelligence (AI)–powered WSI analyzer of TIL in the tumor microenvironment that can define three immune phenotypes (IPs): inflamed, immune-excluded, and immune-desert. These IPs were correlated with tumor response to ICI and survival in two independent cohorts of patients with advanced non–small-cell lung cancer (NSCLC). RESULTS Inflamed IP correlated with enrichment in local immune cytolytic activity, higher response rate, and prolonged progression-free survival compared with patients with immune-excluded or immune-desert phenotypes. At the WSI level, there was significant positive correlation between tumor proportion score (TPS) as determined by the AI model and control TPS analyzed by pathologists ( P < .001). Overall, 44.0% of tumors were inflamed, 37.1% were immune-excluded, and 18.9% were immune-desert. Incidence of inflamed IP in patients with programmed death ligand-1 TPS at < 1%, 1%-49%, and ≥ 50% was 31.7%, 42.5%, and 56.8%, respectively. Median progression-free survival and overall survival were, respectively, 4.1 months and 24.8 months with inflamed IP, 2.2 months and 14.0 months with immune-excluded IP, and 2.4 months and 10.6 months with immune-desert IP. CONCLUSION The AI-powered spatial analysis of TIL correlated with tumor response and progression-free survival of ICI in advanced NSCLC. This is potentially a supplementary biomarker to TPS as determined by a pathologist.
This paper proposes the progressive attention memory network (PAMN) for movie story question answering (QA). Movie story QA is challenging compared to VQA in two aspects: (1) pinpointing the temporal parts relevant to answer the question is difficult as the movies are typically longer than an hour, (2) it has both video and subtitle where different questions require different modality to infer the answer. To overcome these challenges, PAMN involves three main features: (1) progressive attention mechanism that utilizes cues from both question and answer to progressively prune out irrelevant temporal parts in memory, (2) dynamic modality fusion that adaptively determines the contribution of each modality for answering the current question, and (3) belief correction answering scheme that successively corrects the prediction score on each candidate answer. Experiments on publicly available benchmark datasets, MovieQA and TVQA, demonstrate that each feature contributes to our movie story QA architecture, PAMN, and improves performance to achieve the state-of-the-art result. Qualitative analysis by visualizing the inference mechanism of PAMN is also provided.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.