This paper aims to investigate direct imitation learning from human drivers for the task of lane keeping assistance in highway and country roads using grayscale images from a single front view camera. The employed method utilizes convolutional neural networks (CNN) to act as a policy that is driving a vehicle. The policy is successfully learned via imitation learning using real-world data collected from human drivers and is evaluated in closed-loop simulated environments, demonstrating good driving behaviour and a robustness for domain changes. Evaluation is based on two proposed performance metrics measuring how well the vehicle is positioned in a lane and the smoothness of the driven trajectory.
Understanding the complex biology of the tumor microenvironment (TME) is necessary to understand the mechanisms of action of immuno-oncology therapies and to match the right therapies to the right patients. Multiplex immunofluorescence (mIF) is a useful technology that has tremendous potential to further our understanding of cancer patho-biology; however, tools that fully leverage the high dimensionality of this data are still in their infancy. We describe here a novel deep learning pipeline aimed to allow Graph-based Inspection of Tissues via Embeddings, GraphITE. GraphITE transforms mIF data into a graph representation, where unsupervised learning algorithms can be utilised to generate embeddings representing cellular `neighbourhoods'. The embeddings can be downprojected and explored for clustering analysis, and patterns can be mapped back to the image as well as interrogated for phenotypical, morphological, or structural distinctiveness. GraphITE supports the extraction of information not only on the phenotypes of individual cells or the relationships between specific cell types, but is able to characterize cell neighborhoods to look for more complex interactions, thereby allowing pathologists and data scientists to explore mIF data sets, uncovering patterns that are otherwise obscured by the high-dimensionality of the data. In this work, we showcase the current setup of the system, going from raw input data all the way to a user friendly exploration tool. Using this tool, we show how the data can be navigated in a way previously not possible.
No abstract
Characterization of the location and phenotype of cells in the tumor microenvironment (TME) is important to inform the development and monitoring of anti-cancer therapeutic interventions, especially immunotherapies designed to stimulate the immune system to have an anti-cancer effect. Multiplex immunofluorescence (mIF) imaging is being increasingly employed to simultaneously label multiple cell types and subtypes in the tumor microenvironment, but interpretation of these images to gain a robust understanding of tumor and immune cell interactions remains a complicated and challenging process. The rich phenotypic information contained in mIF images has to be taken into account with the spatial topology of the cells in order to be able to distil potential predictive indicators of patient response to therapies as well as prognostic indicators of outcome. While contemporary computational methods allow pathologists to view aggregated phenotypical information and cell interactions on a limited, generally one-to-one basis, these methods have been largely descriptive and geared toward addressing hypotheses as opposed to holistically leveraging the spatial and phenotypic data into a single predictive model. Additional methods are needed to provide a fuller picture of the spatial structure of the TME as captured in mIF images. In this work, we propose a novel pipeline that uses graphs generated from image analysis results and user-defined distance criteria to represent the tumor cellular microstructure. This graph-based approach complements existing mIF analysis techniques by providing information on the spatial, phenotypic, and morphological features of cells in the context of their neighborhood. These graphs subsequently enable characterization of protein expression in detail, description of interactions between individual cells or cell types and their neighbors, interactive tissue querying, and exploration of the cell-level biodiversity. The graph approach not only allows pathologists to efficiently interrogate data contained in mIF images in a hypothesis-driven manner, but importantly also supports more holistic data-driven approaches which, by leveraging state of the art graph convolutional neural networks to obtain numerical embeddings representing each graph and its nodes, enable additional downstream activities such as cell similarity search, and the development of predictive models for patient outcomes and response to therapies. Citation Format: Jason Hipp, Christopher Innocenti, Zhenning Zhang, Jake Cohen-Setton, Balaji Selvaraj, Michalis Frangos, Carlos Pedrinaci, Michael Surace, Laura Dillon, Khan Baykaner. Leveraging graphs to do novel hypothesis and data-driven research using multiplex immunofluorescence images [abstract]. In: Proceedings of the AACR Virtual Special Conference on Artificial Intelligence, Diagnosis, and Imaging; 2021 Jan 13-14. Philadelphia (PA): AACR; Clin Cancer Res 2021;27(5_Suppl):Abstract nr PR-05.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.