Partially Observable Markov Decision Processes (POMDPs) and Wireless Body Area Networks (WBAN): A Survey

Abstract

Wireless body area network (WBAN) is a promising candidate for future health monitoring system. Nevertheless, the path to mature solutions is still facing a lot of challenges that need to be overcome. Energy efficient scheduling is one of these challenges given the scarcity of available energy of biosensors and the lack of portability. Therefore, researchers from academia, industry and health sectors are working together to realize practical solutions for these challenges. The main difficulty in WBAN is the uncertainty in the state of the monitored system. Intelligent learning approaches such as a Markov Decision Process (MDP) were proposed to tackle this issue. A Markov Decision Process (MDP) is a form of Markov Chain in which the transition matrix depends on the action taken by the decision maker (agent) at each time step. The agent receives a reward, which depends on the action and the state. The goal is to find a function, called a policy, which specifies which action to take in each state, so as to maximize some utility functions (e.g., the mean or expected discounted sum) of the sequence of rewards. A partially Observable Markov Decision Processes (POMDP) is a generalization of Markov decision processes that allows for the incomplete information regarding the state of the system. In this case, the state is not visible to the agent. This has many applications in operations research and artificial intelligence. Due to incomplete knowledge of the system, this uncertainty makes formulating and solving POMDP models mathematically complex and computationally expensive. Limited progress has been made in terms of applying POMPD to real applications. In this paper, we surveyed the existing methods and algorithms for solving POMDP in the general domain and in particular in Wireless body area network (WBAN). In addition, the papers discussed recent real implementation of POMDP on practical problems of WBAN. We believe that this work will provide valuable insights for the newcomers who would like to pursue related research in the domain of WBAN.

A Secure Medical Information Management System for Wireless Body Area Networks

“…Wireless body area networks (WBANs) consist of small wearable or implantable sensors on, near, or in a human body. By collecting vital sign parameters and activities from a human body, WBANs can enable continuous health monitoring and provide portable, real-time and ubiquitous healthcare services, and offer numerous practical and innovative applications to improve healthcare quality [1][2][3][4][5][6][7][8][9]. The security of WBANs is critical for those healthcare applications because the sensitive biometric data have to be exchanged over insecure networks.…”

A Lightweight Integrity Authentication Scheme based on Reversible Watermark for Wireless Body Area Networks

“…In general, a Wireless Body Area Network (WBAN) consists of several small wearable or implantable sensors on/near/in a human body and a data aggregator to collect and process the sensor data. By collecting a person's biometric data, the continuous, real-time and ubiquitous health monitoring system with a WBAN can improve the quality of healthcare services [1][2][3][4][5]. However, the sensitive biometric data transmitted over the broadcast wireless channels in WBAN could be attacked easily, therefore how to ensure the integrity of data is one of the major security issues in WBAN [6][7].…”