A Deep Hierarchical Reinforcement Learning Algorithm in Partially Observable Markov Decision Processes

Le, Tuyen P.; Vien, Ngo Anh; Chung, T. J.

doi:10.1109/access.2018.2854283

Cited by 48 publications

(23 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The Q-learning Markov decision process (MDP) algorithm was used under the constraint to achieve the minimum computation latency, communication latency, and network latency by allocating data packets to different processors of virtual machines. Q-learning MDP is a mathematical framework for modeling decision-making and observations by collecting feedback from past experience in a dynamic environment [43]. The proposed approach requires a Q-learning MDP to account for the dynamic behavior of the IoT-fog-cloud system [23, 43].…”

Section: Methodsmentioning

confidence: 99%

“…Q-learning MDP is a mathematical framework for modeling decision-making and observations by collecting feedback from past experience in a dynamic environment [43]. The proposed approach requires a Q-learning MDP to account for the dynamic behavior of the IoT-fog-cloud system [23, 43]. The IoT-fog-cloud system was unable to predict the transition probabilities and rewards because of dynamically changing incoming data packet requests at fog nodes.…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

An analytical model to minimize the latency in healthcare internet-of-things in fog computing environment

et al. 2019

View full text Add to dashboard Cite

Fog computing (FC) is an evolving computing technology that operates in a distributed environment. FC aims to bring cloud computing features close to edge devices. The approach is expected to fulfill the minimum latency requirement for healthcare Internet-of-Things (IoT) devices. Healthcare IoT devices generate various volumes of healthcare data. This large volume of data results in high data traffic that causes network congestion and high latency. An increase in round-trip time delay owing to large data transmission and large hop counts between IoTs and cloud servers render healthcare data meaningless and inadequate for end-users. Time-sensitive healthcare applications require real-time data. Traditional cloud servers cannot fulfill the minimum latency demands of healthcare IoT devices and end-users. Therefore, communication latency, computation latency, and network latency must be reduced for IoT data transmission. FC affords the storage, processing, and analysis of data from cloud computing to a network edge to reduce high latency. A novel solution for the abovementioned problem is proposed herein. It includes an analytical model and a hybrid fuzzy-based reinforcement learning algorithm in an FC environment. The aim is to reduce high latency among healthcare IoTs, end-users, and cloud servers. The proposed intelligent FC analytical model and algorithm use a fuzzy inference system combined with reinforcement learning and neural network evolution strategies for data packet allocation and selection in an IoT–FC environment. The approach is tested on simulators iFogSim (Net-Beans) and Spyder (Python). The obtained results indicated the better performance of the proposed approach compared with existing methods.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

An analytical model to minimize the latency in healthcare internet-of-things in fog computing environment

et al. 2019

View full text Add to dashboard Cite

show abstract

“…Optimizing this function in the process of mapping an unknown environment, where the objective model and the time needed to build it are unknown, is still under research. Though the process of predicting the future impact of an action is computationally expensive, there are recent advancements by using spectral techniques [ 163 ] and deep learning [ 164 ].…”

Section: On Going Developmentsmentioning

confidence: 99%

Active Mapping and Robot Exploration: A Survey

Lluvia

Lazkano

Ansuategi

2021

Sensors

View full text Add to dashboard Cite

Simultaneous localization and mapping responds to the problem of building a map of the environment without any prior information and based on the data obtained from one or more sensors. In most situations, the robot is driven by a human operator, but some systems are capable of navigating autonomously while mapping, which is called native simultaneous localization and mapping. This strategy focuses on actively calculating the trajectories to explore the environment while building a map with a minimum error. In this paper, a comprehensive review of the research work developed in this field is provided, targeting the most relevant contributions in indoor mobile robotics.

show abstract

“…The conceptual backbone of the model is that the representation of environmental dynamics is organized as a hierarchy of time scales (Kiebel, Daunizeau, and Friston 2008). Such modelling approaches have been proposed in cognitive control in the context of hierarchical reinforcement learning (HRL), (e.g., Botvinick and Weinstein 2014;Holroyd and McClure 2015) and are naturally also an increasingly relevant topic in artificial intelligence research (e.g., Bacon and Precup 2018;Pang et al 2019;Le, Vien, and Chung 2018;Mnih et al 2015). In general, HRL models are based on the idea that action sequences can be chunked and represented as a new temporally extended state, (see also Maisto, Donnarumma, and Pezzulo 2015) for a probabilistic modelling alternative.…”

Section: Uncertainty and A Hierarchy Of Time Scalesmentioning

confidence: 99%

Meta-control of the exploration-exploitation dilemma emerges from probabilistic inference over a hierarchy of time scales

Marković

Goschke

Kiebel

2019

Preprint

View full text Add to dashboard Cite

Cognitive control is typically understood as a set of mechanisms which enable humans to reach goals that require integrating the consequences of actions over longer time scales. Importantly, using routine beheavior or making choices beneficial only at a short time scales would prevent one from attaining these goals. During the past two decades, researchers have proposed various computational cognitive models that successfully account for behaviour related to cognitive control in a wide range of laboratory tasks. As humans operate in a dynamic and uncertain environment, making elaborate plans and integrating experience over multiple time scales is computationally expensive, the specific question of how uncertain consequences at different time scales are integrated into adaptive decisions remains poorly understood. Here, we propose that precisely the problem of integrating experience and forming elaborate plans over multiple time scales is a key component for better understanding how human agents solve cognitive control dilemmas such as the exploration-exploitation dilemma. In support of this conjecture, we present a computational model of probabilistic inference over hidden states and actions, which are represented as a hierarchy of time scales. Simulations of goal-reaching agents instantiating the model in an uncertain and dynamic task environment show how the exploration-exploitation dilemma may be solved by inferring meta-control states which adapt behaviour to changing contexts.

show abstract

A Deep Hierarchical Reinforcement Learning Algorithm in Partially Observable Markov Decision Processes

Cited by 48 publications

References 27 publications

An analytical model to minimize the latency in healthcare internet-of-things in fog computing environment

An analytical model to minimize the latency in healthcare internet-of-things in fog computing environment

Active Mapping and Robot Exploration: A Survey

Meta-control of the exploration-exploitation dilemma emerges from probabilistic inference over a hierarchy of time scales

Contact Info

Product

Resources

About