An Empirical Study on Generalizations of the ReLU Activation Function

Banerjee, Chaity; Mukherjee, Tathagata; Pasiliao, Eduardo L.

doi:10.1145/3299815.3314450

Cited by 58 publications

(40 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Though the conventional “s-functions” such as “ logsig ” or “ tansig ” are widely used, they suffer from the “data-saturation” issue ( Banerjee et al., 2019 ). In other words, when the range of the input for testing is significantly larger than that for the network training, the output of the “s-functions” will be close to 1 and exhibits negligible changes to the input variations.…”

Section: Methodsmentioning

confidence: 99%

“…Therefore, functions with an unbounded output range should be selected. Noting that the first-order derivative of the “ voltage vs time ” trajectory is important for calculating the IC trajectory, some widely used functions such as Reflected Linear Unit (ReLU) ( Banerjee et al., 2019 ) are not suitable, as they are not smooth at x = 0. In short, the activation function should be continuous, derivable, and exhibit no data saturation effect.…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Reconstruction of the incremental capacity trajectories from current-varying profiles for lithium-ion batteries

et al. 2021

View full text Add to dashboard Cite

Summary The reliable assessment of battery degradation is fundamental for safe and efficient battery utilization. As an important in situ health diagnostic method, the incremental capacity (IC) analysis relies highly on the low-noise constant-current profiles, which violates the real-life scenarios. Here, a model-free fitting process is reported, for the first time, to reconstruct the IC trajectories from noisy or even current-varying profiles. Based on the results from overall 22 batteries with three case studies, the errors of the peak positions in the reconstructed IC trajectories can be bounded within only 0.25%. With health indicators extracted from the reconstructed IC trajectories, the state of health can be readily determined from simple linear mappings, with estimation error lower than 1% only. By enabling the IC-based methods under complex load profiles, enhanced health assessment could be implemented to improve the reliability of the power systems and further promoting a more sustainable society.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Reconstruction of the incremental capacity trajectories from current-varying profiles for lithium-ion batteries

et al. 2021

View full text Add to dashboard Cite

show abstract

“…The role of Equation ( 5) is to sum all the maps related to the nucleus adjacent to the same spatial position, and the notation N is the integer assigned to all the cores in a layer. Therefore, unlike the standard model of the algorithm, it has a much smaller number of classes, even of those of interest such as cars, buses, bicycles [76,77]. In this direction, there are studies and niche systems such as Cyber-Physical, dedicated to the analysis of all types of bicycle, moped, or other devices in this category.…”

Section: Description Of External Environment and Practical Scenariosmentioning

confidence: 99%

Inter-Urban Analysis of Pedestrian and Drivers through a Vehicular Network Based on Hybrid Communications Embedded in a Portable Car System and Advanced Image Processing Technologies

Zadobrischi

Dimian

2021

Remote Sensing

View full text Add to dashboard Cite

Vehicle density and technological development increase the need for road and pedestrian safety systems. Identifying problems and addressing them through the development of systems to reduce the number of accidents and loss of life is imperative. This paper proposes the analysis and management of dangerous situations, with the help of systems and modules designed in this direction. The approach and classification of situations that can cause accidents is another feature analyzed in this paper, including detecting elements of a psychosomatic nature: analysis and detection of the conditions a driver goes through, pedestrian analysis, and maintaining a preventive approach, all of which are embedded in a modular architecture. The versatility and usefulness of such a system come through its ability to adapt to context and the ability to communicate with traffic safety systems such as V2V (vehicle-to-vehicle), V2I (vehicle-to-infrastructure), V2X (vehicle-to-everything), and VLC (visible light communication). All these elements are found in the operation of the system and its ability to become a portable device dedicated to road safety based on (radio frequency) RF-VLC (visible light communication).

show abstract

“…ReLU activation functions are used to introducing non-linearity into the ANN model so that the ANN can progressively learn more effective feature representations. ReLU is the most commonly used activation in all convolutional neural networks or deep learning models [31]. ReLU is a non-linear function that can backpropagate the errors easily.…”

Section: ) Deep Stacked Multilayered Perceptron (Ds-mlp)mentioning

confidence: 99%

Sensor-Based Human Activity Recognition Using Deep Stacked Multilayered Perceptron Model

et al. 2020

View full text Add to dashboard Cite

The recent development of machines exhibiting intelligent characteristics involves numerous techniques including computer hardware and software architecture development. Many different hardware devices, wearable sensors, machine learning, and deep learning model implementations are being applied in human activity recognition (HAR) applications in recent times. However, to develop high accuracy classification systems for activity recognition using low-cost hardware technology is of significant importance. To achieve this goal this study uses sensor data from two low-cost sensors, gyroscope and accelerometer along with the implementation of an Artificial Neural Network (ANN) based deep learning model for HAR. In particular, Deep Stacked Multilayered Perceptron (DS-MLP) has been proposed. In the implementation of DS-MLP, an ANN model has been used as a meta-learner while five MLP models have been used as base-learners. In this study, these base-learners and meta-learner have been combined using a stack ensemble technique. The performance evaluations have been done first on the applicability of individual base-models followed by the application of DS-MLP, the results prove the high accuracy of 97.3% and 99.4% for heterogeneous datasets used for testing. The performance of the proposed DS-MLP models has been compared to some existing machine learning classifiers and several state-of-the-art activity recognition systems. The comparative result analysis also proves that the proposed system performed better than these classification approaches in terms of important performance metrics such as accuracy, precision, recall, F-score, Cohen's Kappa, and Mathew correlation coefficient.

show abstract

An Empirical Study on Generalizations of the ReLU Activation Function

Cited by 58 publications

References 7 publications

Reconstruction of the incremental capacity trajectories from current-varying profiles for lithium-ion batteries

Reconstruction of the incremental capacity trajectories from current-varying profiles for lithium-ion batteries

Inter-Urban Analysis of Pedestrian and Drivers through a Vehicular Network Based on Hybrid Communications Embedded in a Portable Car System and Advanced Image Processing Technologies

Sensor-Based Human Activity Recognition Using Deep Stacked Multilayered Perceptron Model

Contact Info

Product

Resources

About