Deep-insight visible neural network (DI-VNN) for improving interpretability of a non-image deep learning model by data-driven ontology

Abstract

We aimed to provide a framework that organizes internal properties of a convolutional neural network (CNN) model using non-image data to be interpretable by human. The interface was represented as ontology map and network respectively by dimensional reduction and hierarchical clustering techniques. The applicability is to implement a prediction model either to classify categorical or to estimate numerical outcome, including but not limited to that using data from electronic health records. This pipeline harnesses invention of CNN algorithms for non-image data while improving the depth of interpretability by data-driven ontology. However, the DI-VNN is only for exploration beyond its predictive ability, which requires further explanatory studies, and needs a human user with specific competences in medicine, statistics, and machine learning to explore the DI-VNN with high confidence. The key stages consisted of data preprocessing, differential analysis, feature mapping, network architecture construction, model training and validation, and exploratory analysis.

Questionnaire-free machine-learning method to predict depressive symptoms among community-dwelling older adults

“…It is because a more models in comparison would be more vulnerable to a multiple-testing effect relative to the number of datasets, i.e., the best model is found simply by chance [ 63 ]. To avoid such comparison, we considered three criteria for choosing algorithms in developing the models: (1) those commonly used in clinical prediction studies, i.e., logistic regression [ 58 ], which expects a linear predictor-outcome correlation; (2) those which commonly outperformed others (177 algorithms) across 121 datasets [ 64 ], which allow a non-linear predictor-outcome correlation; and (3) our proposed neural-network algorithm [ 65 ], which pursues moderate predictive performance and deeper interpretability. A sufficient sample size was also considered according to the PROBAST guidelines since a small sample size was vulnerable to overfitting [ 58 ].…”

“…In addition, we used a random-search method to tune values for the pre-defined hyperparameters. We also used those which were defined before conducting this study in a pre-registered protocol [ 65 ]. The randomness and pre-registration were deliberate to avoid a research bias, so-called “hypothesizing after the results are known (HARking)” [ 67 ].…”

“…This approach addresses criticisms of the CNN as a black-box model, i.e., it is unexplained which features and how these result in a particular prediction; yet, a CNN model can predict an outcome very well. Details about the DI-VNN pipeline were previously described elsewhere [ 65 ]. Some modifications of this pipeline were those by applying this procedure over 37 predictors and 19 PCs, resulting in 18 candidate features for DI-VNN.…”

“…These were centered using each average value after quantile-to-quantile normalization over all features among samples. To avoid HARking, we followed the same hyperparameter tuning approach, which was already pre-registered and thoroughly described elsewhere [ 65 ].…”

“…Our DI-VNN model explore ontological relationships among the predictors in predicting the outcome. A detailed technical explanation of the DI-VNN algorithm was previously described elsewhere [ 65 ]. Briefly, there were three steps: (1) differential analysis for feature pre-selection; (2) structural representation of features; and (3) CNN model training.…”