Surrogate-guided sampling designs for classification of rare outcomes from electronic medical records data

Abstract

Scalable and accurate identification of specific clinical outcomes has been enabled by machine-learning applied to electronic medical record (EMR) systems. The development of automatic classification requires the collection of a complete labeled data set, where true clinical outcomes are obtained by human expert manual review. For example, the development of natural language processing algorithms requires the abstraction of clinical text data to obtain outcome information necessary for training models. However, if the outcome is rare then simple random sampling results in very few cases and insufficient information to develop accurate classifiers. Since large scale detailed abstraction is often expensive, time-consuming, and not feasible, more efficient strategies are needed. Under such resource constrained settings, we propose a class of enrichment sampling designs, where selection for abstraction is stratified by auxiliary variables related to the true outcome of interest. Stratified sampling on highly specific variables results in targeted samples that are more enriched with cases, which we show translates to increased model discrimination and better statistical learning performance. We provide mathematical details, and simulation evidence that links sampling designs to their resulting prediction model performance. We discuss the impact of our proposed sampling on both model development and validation. Finally, we illustrate the proposed 0 To whom correspondence should be addressed at wlktan [at] uw [dot] edu.

A Systematic Review of Natural Language Processing Applied to Radiology Reports

“…Ten studies use NLP to create specific cohorts for research purposes and six reported the performance of their tools. Out of these papers, the majority (n=8) created cohorts for specific medical conditions including fatty liver disease [Goldshtein et al, 2020, Redman et al, 2017 hepatocellular cancer [Sada et al, 2016], ureteric stones [Li and Elliot, 2019], vertebral facture [Tan and Heagerty, 2019], traumatic brain injury [Yadav et al, 2016, Mahan et al, 2019, and leptomeningeal disease secondary to metastatic breast cancer [Brizzi et al, 2019]. Five papers identified cohorts focused on particular radiology findings including ground glass opacities (GGO) [Van Haren et al, 2019], cerebral microbleeds (CMB) [Noorbakhsh-Sabet et al, 2018], pulmonary nodules [Gould et al, 2015], [Huhdanpaa et al, 2018], changes in the spine correlated to back pain [Bates et al, 2016] and identifying radiological evidence of people having suffered a fall.…”