Joint Status Sampling and Updating for Minimizing Age of Information in the Internet of Things

Abstract

The effective operation of time-critical Internet of things (IoT) applications requires real-time reporting of fresh status information of underlying physical processes. In this paper, a real-time IoT monitoring system is considered, in which the IoT devices sample a physical process with a sampling cost and send the status packet to a given destination with an updating cost. This joint status sampling and updating process is designed to minimize the average age of information (AoI) at the destination node under an average energy cost constraint at each device. This stochastic problem is formulated as an infinite horizon average cost constrained Markov decision process (CMDP) and transformed into an unconstrained Markov decision process (MDP) using a Lagrangian method. For the single IoT device case, the optimal policy for the CMDP is shown to be a randomized mixture of two deterministic policies for the unconstrained MDP, which is of threshold type. This reveals a fundamental tradeoff between the average AoI of the destination and the sampling and updating costs. Then, a structureaware optimal algorithm to obtain the optimal policy of the CMDP is proposed and the impact of the wireless channel dynamics is studied while demonstrating that channels having a larger mean channel gain and less scattering can achieve better AoI performance. For the case of multiple IoT devices, a low-complexity distributed suboptimal policy is proposed with the updating control at the destination and the sampling control at each IoT device. Then, an online learning algorithm is developed to obtain this policy, which can be implemented at each IoT device and requires only the local knowledge and small signaling from the destination. The proposed learning algorithm is shown to converge almost surely to the suboptimal policy. Simulation results show the structural properties of the optimal policy for the single IoT device case; and show that the proposed policy for multiple IoT devices outperforms a zero-wait baseline policy, with average AoI reductions reaching up to 33%.A preliminary version of this work [1] was submitted for conference publication. Index TermsInternet of things, status update, age of information, Markov decision processes, structural analysis, distributed stochastic learning.

A Reinforcement Learning Framework for Optimizing Age of Information in RF-Powered Communication Systems

“…Afterwards, a series of works [5]- [12] aimed at characterizing the average AoI and its variations (e.g., Peak Age-of-Information (PAoI) [8]- [10] and Value of Information of Update (VoIU) [11]) for adaptations of the queueing model studied in [4]. Another direction of research [13]- [33] focused on employing AoI as a performance metric for different communication systems that deal with time critical information while having limited resources, e.g., multi-server information-update systems [14], broadcast networks [15]- [17], multi-hop networks [18], cognitive networks [19], unmanned aerial vehicle (UAV)assisted communication systems [20]- [22], IoT networks [2], [23], [24], ultra-reliable low-latency vehicular networks [25], multicast networks [26], decentralized random access schemes [32], and multi-state time-varying networks [33]. Particularly, the objective of this research direction was to characterize optimal policies that minimize average AoI, referred to as ageoptimal polices, by applying different tools from optimization theory.…”

On the Role of Age of Information in the Internet of Things

“…The system state is then defined as the combination of all different states of IoT devices. In addition, the AoI value for each process at the destination node is assumed to be upper bounded by a finite value which can be chosen to be arbitrarily large [12]. This value signifies that the information is too stale to be of any use at the destination node.…”

Minimum Age of Information in the Internet of Things With Non-Uniform Status Packet Sizes

“…where Pr[X ′ |X, w] is given by (4). Note that J n (X, w) is related to the right-hand side of the Bellman equation in (13). For each X, RVIA can be used to find V n (X) according to:…”

“…The work in [13] studies the joint design of the status sampling and updating processes to minimize the average AoI for an IoT monitoring system under an energy constraint at each device. In particular, [13] proposes optimal and suboptimal polices for the cases of a single device and multiple devices, respectively. Different from [8]- [13] where the transmission of the status update is assumed to be always successful, the works in [14]- [18] consider that the status update may get lost during the transmission to the destination.…”

“…In this work, we do not explicitly consider the energy limitations on the IoT devices, due to the development of energy harvesting and battery storage technologies[30],[31]. However, it is possible to extend the analytical framework to the scenario in which there are energy constraints for the IoT devices, by following a constrained MDP approach in our previous work[13].…”