Assessment of Deep Learning Using Nonimaging Information and Sequential Medical Records to Develop a Prediction Model for Nonmelanoma Skin Cancer

Abstract

IMPORTANCE A prediction model for new-onset nonmelanoma skin cancer could enhance prevention measures, but few patient data-driven tools exist for more accurate prediction. OBJECTIVE To use machine learning to develop a prediction model for incident nonmelanoma skin cancer based on large-scale, multidimensional, nonimaging medical information. DESIGN, SETTING, AND PARTICIPANTS This study used a database comprising 2 million randomly sampled patients from the Taiwan National Health Insurance Research Database from January 1, 1999, to December 31, 2013. A total of 1829 patients with nonmelanoma skin cancer as their first diagnosed cancer and 7665 random controls without cancer were included in the analysis. A convolutional neural network, a deep learning approach, was used to develop a risk prediction model. This risk prediction model used 3-year clinical diagnostic information, medical records, and temporal-sequential information to predict the skin cancer risk of a given patient within the next year. Stepwise feature selection was also performed to investigate important and determining factors of the model. Statistical analysis was performed from November 1, 2016, to October 31, 2018. MAIN OUTCOMES AND MEASURES Sensitivity, specificity, and area under the receiver operating characteristic (AUROC) curve were used to evaluate the performance of the models. RESULTS A total of 1829 patients (923 women [50.5%] and 906 men [49.5%]; mean [SD] age, 65.3 [15.7] years) with nonmelanoma skin cancer and 7665 random controls without cancer (3951 women [51.5%] and 3714 men [48.4%]; mean [SD] age, 47.5 [17.3] years) were included in the analysis. The 1-year incident nonmelanoma skin cancer risk prediction model using sequential diagnostic information and drug prescription information as a time-incorporated feature matrix could attain an AUROC of 0.89 (95% CI, 0.87-0.91), with a mean (SD) sensitivity of 83.1% (3.5%) and mean (SD) specificity of 82.3% (4.1%). Carcinoma in situ of skin (AUROC, 0.867;-2.80% loss) and other chronic comorbidities (eg, degenerative osteopathy [AUROC, 0.872;-2.32% loss], hypertension [AUROC, 0.879;-1.53% loss], and chronic kidney insufficiency [AUROC, 0.879;-1.52% loss]) served as more discriminative factors for the prediction. Medications such as trazodone, acarbose, systemic antifungal agents, statins, nonsteroidal anti-inflammatory drugs, and thiazide diuretics were the top-ranking discriminative features in the model; each led to more than a 1% decrease of the AUROC when eliminated individually (eg, trazodone AUROC, 0.868; −2.67% reduction; acarbose AUROC, 0.870; −2.50 reduction; and systemic antifungal agents AUROC, 0.875; −1.99 reduction). CONCLUSIONS AND RELEVANCE The findings of this study suggest that a risk prediction model may have potential predictive factors for nonmelanoma skin cancer. This model may help health care professionals target high-risk populations for more intensive skin cancer preventive methods.

How Can Artificial Intelligence Make Medicine More Preemptive?

“…However, current screening protocols are oversimplified and suffer from low compliance. More robust models are needed, for example, for the risk prediction for nonmelanoma skin cancer [ 10 ]. Screening procedures are often the first step leading to early interventions that are more cost-effective than intervening once symptoms appear.…”

Limitations of Deep Learning Attention Mechanisms in Clinical Research: Empirical Case Study Based on the Korean Diabetic Disease Setting

“…It may be desirable to test what attention results are produced when different values of degree of freedom are employed. Particularly, given that the medical field has various data types such as images, natural languages, and numerical values, attention results should be assessed according to the degree of freedom with consideration of the data characteristics [ 66 - 69 ].…”

“…By contrast, when the relationship between a small number of key variables and outcome is important, such as in the generation of targeted therapy [ 70 , 71 ], the importance should be focused on a few variables. However, to the best of our knowledge, most existing attention studies have not considered the control of the variable importance distribution [ 8 , 10 - 12 , 24 , 25 , 32 , 33 , 66 , 68 ]. Therefore, more studies on this subject are needed.…”

Machine Learning in Dermatology: Current Applications, Opportunities, and Limitations

“…Given that the model performed with an AUROC of 0.81 without any evaluation of the major associated risk factors or images, it has the potential for improving the diagnosis and management of NMSC. Another study predicted the likelihood of the development of NMSC by analyzing two million randomly sampled patients from the Taiwan National Health Insurance Research Database [103]. A CNN analyzed 3 years of clinical diagnostic information [i.e., International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) diagnosis and procedure codes and prescriptions; https:// www.cdc.gov/nchs/icd/icd9cm.htm] and temporal-sequential information (i.e., dates of clinical visits and days of prescriptions) to predict the development of NMSC of a given patient within the next year and achieved an AUROC of 0.89 [103].…”