Contextual Learning for Content Caching With Unknown Time-Varying Popularity Profiles via Incremental Clustering

Abstract

With the rapid development of social networks and high-quality video sharing services, the demand for delivering large quantity and high quality contents under stringent endto-end delay requirement is increasing. To meet this demand, we study the content caching problem modelled as a Markov decision process in the network edge server when the popularity profiles are unknown and time-varying. In order to adapt to the changing trends of content popularity, a context-aware popularity learning algorithm is proposed. We prove that the learning error of this scheme is sublinear in the number of requests. In light of the learned popularities, a reinforcement learningbased caching scheme is designed on top of the state-actionreward-state-action algorithm with a function approximation. A reactive caching algorithm is also proposed to reduce the complexity. The time complexities of both the caching schemes are studied to demonstrate their feasibility in real time systems and a theoretical analysis is performed to prove that the cache hit rate of the reactive caching algorithm asymptotically converges to the optimal cache hit rate. Finally the simulations are presented to demonstrate the superiority of the proposed algorithms.

Adaptive Context Caching for IoT-Based Applications: A Reinforcement Learning Approach

“…Further to the complexity, sensed data originating from IoT devices (from which the context is derived) are also transient [ 11 , 26 ]. Previous work in adaptive data caching took data lifetime [ 11 ], properties of network queueing [ 17 ], popularity [ 3 , 4 ], and/or cost of caching [ 26 ] to make adaptive decisions. The problem however is that these parameters cannot be considered in isolation when managing context information compared to data caching.…”

“…A significant body of research has been performed in the area of data caching and surveyed [ 29 , 30 ]. There exists a considerable number of promising techniques investigated in adaptive data caching, often referred to as context-aware data caching [ 4 , 15 , 19 , 30 ]. Interested readers are referred to our survey which compares and contrasts data caching with context caching and provides a concrete definition for adaptive context caching [ 15 ].…”

“…As we indicated in Section 3 , context-aware caching in the literature has primarily dealt with maximizing the HR of a system [ 3 , 4 , 20 ]. Khargaria et al [ 27 ] also implemented a context-caching algorithm based on the popularity of context entities.…”

“…The literature points to caching the popular data [ 3 , 4 , 5 ] such as the timetables to improve the response time of the application. However, it may not reflect the changes and interruptions.…”

Content Popularity Prediction Based on Quantized Federated Bayesian Learning in Fog Radio Access Networks

“…In [12], the authors proposed a user preference model to predict content popularity with model parameters learned by online gradient descent (OGD), which was proved to be superior to traditional caching schemes in [6]- [8]. In [13], the authors proposed two online prediction models for content popularity, namely the popularity prediction model and the Grassman prediction model, where the unconstrained coefficients for linear prediction were obtained by solving a constrained non-negative least squared method. In [14], the authors investigated the effect of time on the prediction of popularity by dynamically weighting the feature sequences, mainly using dynamic weighting techniques to model the changes in the importance of the feature sequences over time.…”