Language Modeling through Long-Term Memory Network

Nugaliyadde, Anupiya; Sohel, Ferdous; Wong, Kok Wai; Xie, Hong

doi:10.1109/ijcnn.2019.8851909

Cited by 12 publications

(5 citation statements)

References 64 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Such a model could also improve preference recalling as the previous user history will be contained in a separate memory. A forgetting mechanism (e.g., similar to the work of Kirkpatrick et al, 2017 ; Nugaliyadde et al, 2019 ; Wang et al, 2020a ; Rae et al, 2020 ) can be introduced to remove or compress old memories for increasing the efficiency of memory retrieval and reducing catastrophic forgetting.…”

Section: Discussionmentioning

confidence: 99%

Coffee With a Hint of Data: Towards Using Data-Driven Approaches in Personalised Long-Term Interactions

Irfan

Hellou²,

Belpaeme

2021

Front. Robot. AI

View full text Add to dashboard Cite

While earlier research in human-robot interaction pre-dominantly uses rule-based architectures for natural language interaction, these approaches are not flexible enough for long-term interactions in the real world due to the large variation in user utterances. In contrast, data-driven approaches map the user input to the agent output directly, hence, provide more flexibility with these variations without requiring any set of rules. However, data-driven approaches are generally applied to single dialogue exchanges with a user and do not build up a memory over long-term conversation with different users, whereas long-term interactions require remembering users and their preferences incrementally and continuously and recalling previous interactions with users to adapt and personalise the interactions, known as the lifelong learning problem. In addition, it is desirable to learn user preferences from a few samples of interactions (i.e., few-shot learning). These are known to be challenging problems in machine learning, while they are trivial for rule-based approaches, creating a trade-off between flexibility and robustness. Correspondingly, in this work, we present the text-based Barista Datasets generated to evaluate the potential of data-driven approaches in generic and personalised long-term human-robot interactions with simulated real-world problems, such as recognition errors, incorrect recalls and changes to the user preferences. Based on these datasets, we explore the performance and the underlying inaccuracies of the state-of-the-art data-driven dialogue models that are strong baselines in other domains of personalisation in single interactions, namely Supervised Embeddings, Sequence-to-Sequence, End-to-End Memory Network, Key-Value Memory Network, and Generative Profile Memory Network. The experiments show that while data-driven approaches are suitable for generic task-oriented dialogue and real-time interactions, no model performs sufficiently well to be deployed in personalised long-term interactions in the real world, because of their inability to learn and use new identities, and their poor performance in recalling user-related data.

show abstract

Section: Discussionmentioning

confidence: 99%

Coffee With a Hint of Data: Towards Using Data-Driven Approaches in Personalised Long-Term Interactions

Irfan

Hellou²,

Belpaeme

2021

Front. Robot. AI

View full text Add to dashboard Cite

show abstract

“…However, they face challenges such as gradient vanishing or exploding when dealing with long sequences. These challenges result in a limited memory span and the loss of early input information [45]. In order to address these challenges, long short-term memory (LSTM) is specifically designed to handle long-term dependencies in time series data by three gates and a cell state [46].…”

Section: Bidirectional Long Short-term Memorymentioning

confidence: 99%

CBF-IDS: Addressing Class Imbalance Using CNN-BiLSTM with Focal Loss in Network Intrusion Detection System

Peng,

Wu,

Xiao

2023

Applied Sciences

View full text Add to dashboard Cite

The importance of network security has become increasingly prominent due to the rapid development of network technology. Network intrusion detection systems (NIDSs) play a crucial role in safeguarding networks from malicious attacks and intrusions. However, the issue of class imbalance in the dataset presents a significant challenge to NIDSs. In order to address this concern, this paper proposes a new NIDS called CBF-IDS, which combines convolutional neural networks (CNNs) and bidirectional long short-term memory networks (BiLSTMs) while employing the focal loss function. By utilizing CBF-IDS, spatial and temporal features can be extracted from network traffic. Moreover, during model training, CBF-IDS applies the focal loss function to give more weight to minority class samples, thereby mitigating the impact of class imbalance on model performance. In order to evaluate the effectiveness of CBF-IDS, experiments were conducted on three benchmark datasets: NSL-KDD, UNSW-NB15, and CIC-IDS2017. The experimental results demonstrate that CBF-IDS outperforms other classification models, achieving superior detection performance.

show abstract

“…Deep neural network (DNN) is a popular ANN [33] and used for both mediumterm and long-term predictions. Recurrent neural network (RNN) and long shortterm memory (LSTM) network are the most used deep neural network (DNN), especially networks that adapt feedback loop from past inputs [49]. RNN and LSTM have surpassed other DNN models that do not employ feedback loops [50].…”

Section: Artificial Neural Networkmentioning

confidence: 99%

Forecasting peak energy demand for smart buildings

et al. 2020

View full text Add to dashboard Cite

Predicting energy consumption in buildings plays an important part in the process of digital transformation of the built environment, and for understanding the potential for energy savings. This also contributes to reducing the impact of climate change, where buildings need to increase their adaptability and resilience while reducing energy consumption and maintain user comfort. The use of Internet of Things devices for monitoring and control of energy consumption in buildings can take into account user preferences, event monitoring and building optimization. Detecting peak energy demand from historical building data can enable users to manage their energy use more efficiently, while also enabling real-time response strategies (including control and actuation) to known or future scenarios. Several statistical, time series, and machine learning techniques are proposed in this work to predict electricity consumption for five different building types, by using peak demand forecasting to achieve energy efficiency. We have used several indigenous and exogenous variables with a view to test different energy forecasting scenarios. The suggested techniques are evaluated for creating predictive models, including linear Regression, dynamic regression, ARIMA time series, exponential smoothing time series, artificial neural network, and deep neural network. We conduct the analysis on an energy consumption dataset of five buildings from 2014 until 2019. Our results show that for a day ahead prediction, the ARIMA model outperforms the other approaches with an accuracy of 98.91% when executed over a 168 h (1 week) of uninterrupted data for five government buildings.

show abstract

Language Modeling through Long-Term Memory Network

Cited by 12 publications

References 64 publications

Coffee With a Hint of Data: Towards Using Data-Driven Approaches in Personalised Long-Term Interactions

Coffee With a Hint of Data: Towards Using Data-Driven Approaches in Personalised Long-Term Interactions

CBF-IDS: Addressing Class Imbalance Using CNN-BiLSTM with Focal Loss in Network Intrusion Detection System

Forecasting peak energy demand for smart buildings

Contact Info

Product

Resources

About