A Deep Neural Network Model using Random Forest to Extract Feature Representation for Gene Expression Data Classification

Abstract

In predictive model development, gene expression data is associated with the unique challenge that the number of samples (n) is much smaller than the amount of features (p). This “n ≪ p” property has prevented classification of gene expression data from deep learning techniques, which have been proved powerful under “n > p” scenarios in other application fields, such as image classification. Further, the sparsity of effective features with unknown correlation structures in gene expression profiles brings more challenges for classification tasks. To tackle these problems, we propose a newly developed classifier named Forest Deep Neural Network (fDNN), to integrate the deep neural network architecture with a supervised forest feature detector. Using this built-in feature detector, the method is able to learn sparse feature representations and feed the representations into a neural network to mitigate the overfitting problem. Simulation experiments and real data analyses using two RNA-seq expression datasets are conducted to evaluate fDNN’s capability. The method is demonstrated a useful addition to current predictive models with better classification performance and more meaningful selected features compared to ordinary random forests and deep neural networks.

Interpretation and Simplification of Deep Forest

“…(M) # Op. (M) AlexNets [37] 92.2 62.3 720 3DCNN-DAP [38] 92.4 70 174 M ultiple Inception [39] 93.2 12.36 23.7 3DIR with LSTM [40] 93.2 10.90 18.9 H-WRF [27] 92.6 0.25 0.0067 gcForest [7] 89.7 2.90 0.0381 FTDRF [8] 92.4 2.51 0.0233 LM RF (1.0) [24] 93.6 0.53 0.0060 iLM RF (0.8) 92.5 0.47 0.0059 iLM RF (0.7) 91.9 0.44 0.0059 Table 8. Comparison of accuracy (Acc.…”

Interpretation and Simplification of Deep Forest

“…An additional experiment was conducted on the CK+ dataset to test whether the proposed algorithm effectively recognizes the facial expressions, and the performance was compared with other state-of-the-art-methods, namely, an AlexNets-based FER approach [37]; a 3D CNN-based FER approach with deformable facial action parts constrained (3DCNN-DAP) [38]; a DNN-based approach that uses multiple inception layers [39]; a 3D Inception-ResNet (3DIR) with LSTM for the FER [40]; a fast FER based on a hierarchical weighted RF (H-WRF) [27]; three DRFbased methods, i.e., gcForest [7], FTDRF [8], and LMRF [24]; and the proposed iLMRF. The deep RF based methods, gcForest, FTDRF, and iLMRF, exploited a feature vector consisting of an 84-dimensional distance ratio and an 88dimensional angle ratio [27] without using the entire image.…”

“…Additional approaches exist in which convolutional neural networks (CNNs) and decision trees [27]- [29] are combined to integrate the DNN architecture with a supervised forest feature detector. However, these differ from ensemble-based approaches that use ensemble trees as a layer-by-layer connection without the use of backpropagation during learning.…”

Explainable Deep Learning for Augmentation of Small RNA Expression Profiles

“…Its disadvantage is that, it requires large amounts of training data (Li et al, 2019), is prone to overfit for small training sets and is difficult to biologically interpret (feature importance) (Webb, 2018). In (Kong and Yu, 2018) the RF and DL approaches were used in two stages. For the first stage, the RF approach was used to extract the most important features and then for the second stage, the DL approach was implemented for gene expression data classification based on the selected features.…”