Iterative Deep Neighborhood: A Deep Learning Model Which Involves Both Input Data Points and Their Neighbors

Abstract

Deep learning models, such as deep convolutional neural network and deep long-short term memory model, have achieved great successes in many pattern classification applications over shadow machine learning models with hand-crafted features. The main reason is the ability of deep learning models to automatically extract hierarchical features from massive data by multiple layers of neurons. However, in many other situations, existing deep learning models still cannot gain satisfying results due to the limitation of the inputs of models. The existing deep learning models only take the data instances of an input point but completely ignore the other data points in the dataset, which potentially provides critical insight for the classification of the given input. To overcome this gap, in this paper, we show that the neighboring data points besides the input data point itself can boost the deep learning model’s performance significantly and design a novel deep learning model which takes both the data instances of an input point and its neighbors’ classification responses as inputs. In addition, we develop an iterative algorithm which updates the neighbors of data points according to the deep representations output by the deep learning model and the parameters of the deep learning model alternately. The proposed algorithm, named “Iterative Deep Neighborhood (IDN),” shows its advantages over the state-of-the-art deep learning models over tasks of image classification, text sentiment analysis, property price trend prediction, etc.

Intelligent Early Fault Diagnosis of Space Flywheel Rotor System

“…) is the square norm of the difference between x1 and x2 , when E( x1 , x2 ) approaches 0, the gradient of E( x1 , x2 ) relative to the model parameters will disappear [39].…”

A Framework for Text Classification Using Evolutionary Contiguous Convolutional Neural Network and Swarm Based Deep Neural Network

“…Figure 1 illustrates the overview framework of our model. The model comprises a CNN model (Jeong et al, 2020 ), indicated as c , one concatenation layer, 1 Fully Connected (FC) layer, and one Softmax nonlinear transformation layer inspired by the work and architecture in (Liu et al, 2020 ). The dataflow in the model along with the function of these layers are explained as follows:…”

“…DE was used for finely adapting Naïve Bayesian Classifier (NBC) and used for text classification in Diab and El Hindi ( 2017 ). A multiple partially observed view for multilingual text categorization (Amini et al, 2009 ), an iterative deep neighborhood model for text classification (Liu et al, 2020 ), integrating bidirectional Long Term Short Memory (LSTM) with 2D max pooling for text classification (Zhou et al, 2016 ), Recurrent Neural Network (RNN) for text classification with multi-task learning (Liu et al, 2016 ), Recurrent CNN for text classification (Lai et al, 2015 ), and a character level convolutional network for text classification (Zhang et al, 2015 ) are some of the most famous deep learning works proposed in the literature. A ranking based deep learning representation for efficient text classification (Zheng et al, 2018 ), a hierarchical neural network document representation approach for text classification using three different models (Kowsari et al, 2017 ), a C-LSTM neural network for text classification (Zhou, 2015 ), and a neural attention model for leveraging contextual sentences for text classification (Yan, 2019 ) are again some of the works which help the research community to a great extent for further analysis.…”