Automatic extraction of cancer registry reportable information from free-text pathology reports using multitask convolutional neural networks

Abstract

Objective We implement 2 different multitask learning (MTL) techniques, hard parameter sharing and cross-stitch, to train a word-level convolutional neural network (CNN) specifically designed for automatic extraction of cancer data from unstructured text in pathology reports. We show the importance of learning related information extraction (IE) tasks leveraging shared representations across the tasks to achieve state-of-the-art performance in classification accuracy and computational efficiency. Materials and Methods Multitask CNN (MTCNN) attempts to tackle document information extraction by learning to extract multiple key cancer characteristics simultaneously. We trained our MTCNN to perform 5 information extraction tasks: (1) primary cancer site (65 classes), (2) laterality (4 classes), (3) behavior (3 classes), (4) histological type (63 classes), and (5) histological grade (5 classes). We evaluated the performance on a corpus of 95 231 pathology documents (71 223 unique tumors) obtained from the Louisiana Tumor Registry. We compared the performance of the MTCNN models against single-task CNN models and 2 traditional machine learning approaches, namely support vector machine (SVM) and random forest classifier (RFC). Results MTCNNs offered superior performance across all 5 tasks in terms of classification accuracy as compared with the other machine learning models. Based on retrospective evaluation, the hard parameter sharing and cross-stitch MTCNN models correctly classified 59.04% and 57.93% of the pathology reports respectively across all 5 tasks. The baseline models achieved 53.68% (CNN), 46.37% (RFC), and 36.75% (SVM). Based on prospective evaluation, the percentages of correctly classified cases across the 5 tasks were 60.11% (hard parameter sharing), 58.13% (cross-stitch), 51.30% (single-task CNN), 42.07% (RFC), and 35.16% (SVM). Moreover, hard parameter sharing MTCNNs outperformed the other models in computational efficiency by using about the same number of trainable parameters as a single-task CNN. Conclusions The hard parameter sharing MTCNN offers superior classification accuracy for automated coding support of pathology documents across a wide range of cancers and multiple information extraction tasks while maintaining similar training and inference time as those of a single task–specific model.

Deep Active Learning for Classifying Cancer Pathology Reports

“…Each pathology report in our dataset is associated with a unique tumor ID; the same tumor ID may be associated with one or more pathology reports. For each tumor ID, Certified Tumor Registrars (CTRs) manually assigned ground truth labels for key data elements such as cancer site and histology based off all data available for that tumor ID according to the SEER program coding and staging manual [2] . We note that these ground truth labels are at the tumor level rather than at the report level; as a consequence of this labelling scheme, tumor IDs associated with multiple pathology reports may have a tumor-level label that does not reflect the content within individual pathology reports.…”

“…Similar to our previous studies, we applied standard text pre-processing techniques to clean our corpus [2,3]. After excluding metadata (e.g., patient ID, registry ID) in cancer pathology reports, text was cleaned by removing any consecutive punctuation and by lowercasing all alphabetical characters.…”

“…We limit the maximum length for each cancer pathology reports 1,500 word tokens; [1] NJSCR is no longer in the SEER Program, but is included in the current data release. [2] https://seer.cancer.gov/tools/codingmanuals/index.html documents longer than 1500 are truncated and all documents shorter than 1500 are zero-padded.…”

“…For our classification model, we use a word-level text CNN because it is widely used for clinical NLP and text classification tasks [28,29]. The CNN architecture is implemented based off the same architecture we have used in previous studies [2,3]. The model hyperparameters are listed in Table 2.…”

“…The latest developments in natural language processing (NLP) have made notable progress in automating classification and information extraction from clinical texts. The current state-of-the-art techniques are generally deep learning (DL) architectures such as Convolutional Neural Networks (CNNs), which have been shown to outperform traditional machine learning techniques and rule-based approaches in clinical text applications [2][3][4]. However, a common drawback of DL models is that they tend to require a large amount of training data to achieve high performance.…”