Enhancing the stability of the deep neural network using a non-constant learning rate for data stream

Abstract

The data stream is considered the backbone of many real-world applications. These applications are most effective when using modern techniques of machine learning like deep neural networks (DNNs). DNNs are very sensitive to set parameters, the most prominent one is the learning rate. Choosing an appropriate learning rate value is critical because it is able to control the overall network performance. This paper presents a new developing DNN model using a multi-layer perceptron (MLP) structure that includes network training based on the optimal learning rate. Thereupon, this model consists of three hidden layers and does not adopt the stability of the learning rate but has a non-constant value (varying over time) to obtain the optimal learning rate which is able to reduce the error in each iteration and increase the model accuracy. This is done by deriving a new parameter that is added to and subtracted from the learning rate. The proposed model is evaluated by three streaming datasets: electricity, network security layer-knowledge discovery in database (NSL-KDD), and human gait database (HuGaDB) datasets. The results proved that the proposed model achieves better results than the constant model and outperforms previous models in terms of accuracy, where it achieved 88.16%, 98.67%, and 97.63% respectively.

DA-FER: Domain Adaptive Facial Expression Recognition

“…Unlike other deep learning methods, where the loss function depends only on feature extraction and final classification, domain adaptive learning involves calculating the difference between the features of the source domain and target domain [23]. Therefore, the learning rate of the domain adaptive network needs to be non-constant [24]. As the learning process continues, the loss function decreases, getting smaller and smaller.…”

Ataxia severity classification using enhanced feature selection and ranking optimization through machine learning model

“…Effective diagnosis and detection of neurologicaldisorders require the use of appropriate tools and approaches [7]. Wearable gadgets [8], video, and thermalcamera devices [9], microelectromechanical-sensor-units [10], as well as the kinect-sensor [11], are just some examples of the sensor technologies that have become increasingly common due to advancements in wireless technology as well as sensing technologies. Wearable-gait sensors, as recently shown in [12], are useful for investigating scale-for-the-assessment and rating-of-ataxia (SARA) behavior.…”