Edge Caching for IoT Transient Data Using Deep Reinforcement Learning

Sheng, Shuran; Chen, Peng; Chen, Zhimin; Wu, Lenan; Jiang, Hao

doi:10.1109/iecon43393.2020.9255111

Cited by 6 publications

(3 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Kiani et al [ 28 ] dynamically resize cache memory according to the query load. Zhu et al [ 26 ], Sheng et al [ 13 ], and Nasehzadeh et al [ 36 ] provide evidence of cache-replacement strategies. There is a significant interest in leveraging machine learning (ML) techniques such as reinforcement learning (RL) using deep neural networks (DNN) to self-learn objective-oriented policies [ 30 ].…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Adaptive Context Caching for IoT-Based Applications: A Reinforcement Learning Approach

Weerasinghe

Zaslavsky

Loke

et al. 2023

Sensors

View full text Add to dashboard Cite

Making internet-of-things (IoT)-based applications context-aware demands large amounts of raw data to be collected, interpreted, stored, and reused or repurposed if needed from many domains and applications. Context is transient but interpreted data can be distinguished from IoT data in many aspects. Managing context in cache is a novel area of research that has been given very little attention. Performance metric-driven adaptive context caching (ACOCA) can have a profound impact on the performance and cost efficiency of context-management platforms (CMPs) when responding to context queries in realtime. Our paper proposes an ACOCA mechanism to maximize both the cost and performance efficiency of a CMP in near realtime. Our novel mechanism encompasses the entire context-management life cycle. This, in turn, distinctively addresses the problems of efficiently selecting context for caching and managing the additional costs of context management in the cache. We demonstrate that our mechanism results in long-term efficiencies for the CMP that have not been observed in any previous study. The mechanism employs a novel, scalable, and selective context-caching agent implemented using the twin delayed deep deterministic policy gradient method. It further incorporates an adaptive context-refresh switching policy, a time-aware eviction policy, and a latent caching decision management policy. We point out in our findings that the additional complexity of adaptation introduced to the CMP through ACOCA is significantly justified, considering the cost and performance gains achieved. Our algorithm is evaluated using a real-world inspired heterogeneous context-query load and a data set based on parking-related traffic in Melbourne, Australia. This paper presents and benchmarks the proposed scheme against traditional and context-aware caching policies. We demonstrate that ACOCA outperforms the benchmarks in both cost and performance efficiency, i.e., up to 68.6%, 84.7%, and 67% more cost efficient compared to traditional data caching policies to cache context, redirector mode, and context-aware adaptive data caching under real-world-like circumstances.

show abstract

Section: Related Workmentioning

confidence: 99%

“…First, context information cannot be cached based on the write-once-read-many concept, as in data caching, due to transiency. Context needs to be refreshed [ 11 , 12 ], similar to cached IoT data [ 13 , 14 ]. Refreshing incurs a recurring cost of processing and data retrieval during cache residence.…”

Section: Introductionmentioning

confidence: 99%

Adaptive Context Caching for IoT-Based Applications: A Reinforcement Learning Approach

Weerasinghe

Zaslavsky

Loke

et al. 2023

Sensors

View full text Add to dashboard Cite

show abstract

“…There also exist works showing the applicability of genetic algorithms to cache replacement [39]. The main criticism of the usage of ML-based approaches to large caches is that the learning process can take a significant amount of time and computational resources [40]. Moreover, quick adaptation to changes in load is also challenging for such methods.…”

Section: Related Workmentioning

confidence: 99%

Refresh Rate-Based Caching and Prefetching Strategies for Internet of Things Middleware

Medvedev,

Hassani,

Belov

et al. 2023

Sensors

View full text Add to dashboard Cite

One of the research directions in Internet of Things (IoT) is the field of Context Management Platforms (CMPs) which is a specific type of IoT middleware. CMPs provide horizontal connectivity between vertically oriented IoT silos resulting in a noticeable difference in how IoT data streams are processed. As these context data exchanges can be monetised, there is a need to model and predict the context metrics and operational costs of this exchange to provide relevant and timely context in a large-scale IoT ecosystem. In this paper, we argue that caching all transient context information to satisfy this necessity requires large amounts of computational and network resources, resulting in tremendous operational costs. Using Service Level Agreements (SLAs) between the context providers, CMP, and context consumers, where the level of service imperfection is quantified and linked to the associated costs, we show that it is possible to find efficient caching and prefetching strategies to minimize the context management cost. So, this paper proposes a novel method to find the optimal rate of IoT data prefetching and caching. We show the main context caching strategies and the proposed mathematical models, then discuss how a correctly chosen proactive caching strategy and configurations can help to maximise the profit of CMP operation when multiple SLAs are defined. Our model is accurate up to 0.0016 in Root Mean Square Percentage Error against our simulation results when estimating the profits to the system. We also show our model is valid using the t-test value tending to 0 for all the experimental scenarios.

show abstract

Process Framework for Edge Computing Systems

Vora

2024

Lecture Notes in Networks and Systems

View full text Add to dashboard Cite

Edge Caching for IoT Transient Data Using Deep Reinforcement Learning

Cited by 6 publications

References 16 publications

Adaptive Context Caching for IoT-Based Applications: A Reinforcement Learning Approach

Adaptive Context Caching for IoT-Based Applications: A Reinforcement Learning Approach

Refresh Rate-Based Caching and Prefetching Strategies for Internet of Things Middleware

Process Framework for Edge Computing Systems

Contact Info

Product

Resources

About