Gate-variants of Gated Recurrent Unit (GRU) neural networks

Dey, Rahul; Salemt, Fathi M.

doi:10.1109/mwscas.2017.8053243

Cited by 1,064 publications

(525 citation statements)

References 11 publications

Supporting

Mentioning

521

Contrasting

Unclassified

Order By: Relevance

“…By generating temporal representations from learning, LSTM has been successfully applied to speech recognition and machine translation. LSTM is similar to GRU we used in our residual block but LSTM has a higher computing cost [25].…”

Section: H a Comparative Studymentioning

confidence: 99%

LuNet: A Deep Neural Network for Network Intrusion Detection

Guo

2019

2019 IEEE Symposium Series on Computational Intelligence (SSCI)

View full text Add to dashboard Cite

One challenge for building a secure network communication environment is how to effectively detect and prevent malicious network behaviours. The abnormal network activities threaten users' privacy and potentially damage the function and infrastructure of the whole network. To address this problem, the network intrusion detection system (NIDS) has been used. By continuously monitoring network activities, the system can timely identify attacks and prompt counter-attack actions. NIDS has been evolving over years. The current-generation NIDS incorporates machine learning (ML) as the core technology in order to improve the detection performance on novel attacks. However, the high detection rate achieved by a traditional MLbased detection method is often accompanied by large falsealarms, which greatly affects its overall performance. In this paper, we propose a deep neural network, Pelican, that is built upon specially-designed residual blocks. We evaluated Pelican on two network traffic datasets, NSL-KDD and UNSW-NB15. Our experiments show that Pelican can achieve a high attack detection performance while keeping a much low false alarm rate when compared with a set of up-to-date machine learning based designs.

show abstract

Section: H a Comparative Studymentioning

confidence: 99%

LuNet: A Deep Neural Network for Network Intrusion Detection

Guo

2019

2019 IEEE Symposium Series on Computational Intelligence (SSCI)

View full text Add to dashboard Cite

show abstract

“…GRU: The GRU is a simplified version of the more complex LSTM unit that combines the input and forgets gates into a single update gate. It then merges both the cell and hidden states for faster operation (28). Equations 13-15 describe the mathematical operations inside the GRU neurons.…”

Section: Simple Rnnmentioning

confidence: 99%

Development and Evaluation of Recurrent Neural Network-Based Models for Hourly Traffic Volume and Annual Average Daily Traffic Prediction

Khan

Chowdhury

et al. 2019

Transportation Research Record

View full text Add to dashboard Cite

Word count: Abstract = 244 words, Text 7025 + 5 Tables 1250 + 17 Figures = 8275 words, Reference = 712 words Initial Paper ABSTRACTThe prediction of high-resolution hourly traffic volumes of a given roadway is essential for transportation planning. Traditionally, Automatic Traffic Recorders (ATR) are used to collect this hourly volume data. These large datasets are time series data characterized by long-term temporal dependencies and missing values. Regarding the temporal dependencies, all roadways are characterized by seasonal variations that can be weekly, monthly or yearly, depending on the cause of the variation. Regarding the missing data in a time-series sequence, traditional time series forecasting models perform poorly under the influence of seasonal variations. To address this limitation, robust, Recurrent Neural Network (RNN) based, multi-step ahead forecasting models are developed for time-series in this study. The simple RNN, the Gated Recurrent Unit (GRU) and the Long Short-Term Memory (LSTM) units are used to develop the model and evaluate its performance. Two approaches are used to address the missing value issue: masking and imputation, in conjunction with the RNN models. Six different imputation algorithms are then used to identify the best model. The analysis indicates that the LSTM model performs better than simple RNN and GRU models, and imputation performs better than masking to predict future traffic volume. Based on analysis using 92 ATRs, the LSTM-Median model is deemed the best model in all scenarios for hourly traffic volume and AADT prediction, with an average RMSE of 274 and MAPE of 18.91% for hourly traffic volume prediction and average RMSE of 824 and MAPE of 2.10% for AADT prediction. The hourly traffic volume roadway data is an important high-resolution dataset used to describe the operational characteristics of a transportation system. Accurate hourly traffic volumes can be utilized in calculating the Average Annual Daily Traffic (AADT). AADT is an essential parameter in many transportation models and decisions. Moreover, the prediction of future hourly traffic volumes of a given roadway is even more important than current data because it can be used to estimate future growth. Here, the volume growth factor of a roadway can be combined with other external data to predict the overall growth pattern of an area. Moreover, the high-resolution data provides insight into the factors contributing to growth, as it may be a gradual growth pattern or a sudden peak. The roadway volume can increase at a very specific time next year due to some special events, and a predictive model can predict this change. This means that the special event is a phenomenon that has occurred before. However, if a predictive model is unable to capture this event, then it is a new phenomenon that has not been previously observed. Therefore, the detection of special events or anomalies is also an application of high-resolution hourly volumes.Transportation planning is characterized by many projects that are relate...

show abstract

“…With regard to the process of feature extraction, most approaches use recurrent neural networks [26], typically implemented with LSTM [27] and GRU [28], to extract textual features. As for the visual features, convolutional neural networks [29] are used to obtain region features from image, among which VGG-net [30] and deep residual networks [31] are most common.…”

Section: A Feature Extraction and Representationmentioning

confidence: 99%

Cross-Modal Multistep Fusion Network with Co-Attention for Visual Question Answering

Lao

Guo

Wang

et al. 2018

Preprint

View full text Add to dashboard Cite

Visual question answering (VQA) is receiving increasing attention from researchers in both the computer vision and natural language processing fields. There are two key components in the VQA task: feature extraction and multi-modal fusion. For feature extraction, we introduce a novel co-attention scheme by combining Sentence-guide Word Attention (SWA) and Question-guide Image Attention (QIA) in a unified framework. To be specific, the textual attention SWA relies on the semantics of the whole question sentence to calculate contributions of different question words for text representation. For the multi-modal fusion, we propose a "Cross-modal Multistep Fusion (CMF)" network to generate multistep features and achieve multiple interactions for two modalities, rather than focusing on modeling complex interactions between two modals like most current feature fusion methods. To avoid the linear increase of the computational cost, we share the parameters for each step in the CMF. Extensive experiments demonstrate that the proposed method can achieve competitive or better performance than the state-of-the-art. INDEX TERMS visual question answering, cross-modal multistep fusion network, attention mechanism

show abstract

Gate-variants of Gated Recurrent Unit (GRU) neural networks

Cited by 1,064 publications

References 11 publications

LuNet: A Deep Neural Network for Network Intrusion Detection

LuNet: A Deep Neural Network for Network Intrusion Detection

Development and Evaluation of Recurrent Neural Network-Based Models for Hourly Traffic Volume and Annual Average Daily Traffic Prediction

Cross-Modal Multistep Fusion Network with Co-Attention for Visual Question Answering

Contact Info

Product

Resources

About