Action Space Shaping in Deep Reinforcement Learning

Kanervisto, Anssi; Scheller, Christian; Hautamäki, Ville

doi:10.1109/cog47356.2020.9231687

Cited by 71 publications

(43 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The set of all valid actions in a given environment is called as an action space abbreviated as S [47]. Some environment such as Atari and Go have discrete action spaces, where only finite number of actions are available to the agent [48]. Other environments have continuous action spaces, such as, where an agent controls a robot in a physical world [49].…”

Section: A Basic Conceptmentioning

confidence: 99%

Reinforcement learning based resource management for fog computing environment: Literature review, challenges, and open issues

et al. 2022

View full text Add to dashboard Cite

show abstract

Section: A Basic Conceptmentioning

confidence: 99%

Reinforcement learning based resource management for fog computing environment: Literature review, challenges, and open issues

et al. 2022

View full text Add to dashboard Cite

show abstract

“…The majority of found applications in the field of DRL are evaluated in game-like environments [32]. To the best of our knowledge and based on scanning and searching using Google Scholar, very few applications of DRL for HFS scheduling problems could be found.…”

Section: Related Workmentioning

confidence: 99%

“…The goal is eventually to present an encoding similar to a video game, in which an agent has access to either one or two controllers and can interact with the environment such as a game. This encoding is motivated by the fact that most of the applications in the field of deep reinforcement learning are conducted in game-like environments [32].…”

Section: Action Space Observation Space and Reward Functionmentioning

confidence: 99%

“…This type of action space is used when the agents must produce multiple independent discrete actions to interact with the environment [39]. For further discussion on the multidiscrete action space formulation, one can refer to the works presented in [32,39]. These actions are then used to select different allocation and sequencing algorithms on the first and fourth processing stages over time for solving the scheduling problem.…”

Section: Action Spacementioning

confidence: 99%

See 1 more Smart Citation

Deep Reinforcement Learning Techniques For Solving Hybrid Flow Shop Scheduling Problems: Proximal Policy Optimization (PPO) and Asynchronous Advantage Actor-Critic (A3C)

Nahhas¹,

Kharitonov²,

Turowski³

2022

Proceedings of the Annual Hawaii International Conference on System Sciences

View full text Add to dashboard Cite

Well-studied scheduling practices are fundamental for the successful support of core business processes in any manufacturing environment. Particularly, the Hybrid Flow Shop (HFS) scheduling problems are present in many manufacturing environments. The current advances in the field of Deep Reinforcement Learning (DRL) attracted the attention of both practitioners and academics to investigate their adoption beyond synthetic game-like applications. Therefore, we present an approach that is based on DRL techniques in conjunction with a discrete event simulation model to solve a real-world four-stage HFS scheduling problem. The main narrative behind the presented concepts is to expose a DRL agent to a gamelike environment using an indirect encoding. Two types of DRL techniques namely, Proximal Policy Optimization (PPO) and Asynchronous Advantage Actor-Critic (A3C), are evaluated for solving problems of different complexity. The computational results suggest that the DRL agents successfully learn appropriate policies for solving the investigated problem. In addition, the investigation shows that the agent can adjust their policies when we expose them to a different problem. We further evaluate the approach to solving problem instances published in the literature to establish a comparison.

show abstract

“…The compatibility of the two types of action space can effectively adapt to a variety of reinforcement learning algorithms. Due to the low precision requirements of the filtering threshold for small and medium-scale datasets, discretization of actions can reduce the number of action explorations [47] while meeting the basic accuracy requirements, which ensures efficient access to high-performance areas. For the dataset with large-scale neighbors, a large number of discrete actions that meet the high-precision requirements will affect the learning effect [17].…”

mentioning

confidence: 99%

Reinforced Neighborhood Selection Guided Multi-Relational Graph Neural Networks

PengHao

ZhangRuitong

DouYingtong

et al. 2021

ACM Trans. Inf. Syst.

View full text Add to dashboard Cite

Graph Neural Networks (GNNs) have been widely used for the representation learning of various structured graph data, typically through message passing among nodes by aggregating their neighborhood information via different operations. While promising, most existing GNNs oversimplify the complexity and diversity of the edges in the graph and thus are inefficient to cope with ubiquitous heterogeneous graphs, which are typically in the form of multi-relational graph representations. In this article, we propose RioGNN , a novel Reinforced, recursive, and flexible neighborhood selection guided multi-relational Graph Neural Network architecture, to navigate complexity of neural network structures whilst maintaining relation-dependent representations. We first construct a multi-relational graph, according to the practical task, to reflect the heterogeneity of nodes, edges, attributes, and labels. To avoid the embedding over-assimilation among different types of nodes, we employ a label-aware neural similarity measure to ascertain the most similar neighbors based on node attributes. A reinforced relation-aware neighbor selection mechanism is developed to choose the most similar neighbors of a targeting node within a relation before aggregating all neighborhood information from different relations to obtain the eventual node embedding. Particularly, to improve the efficiency of neighbor selecting, we propose a new recursive and scalable reinforcement learning framework with estimable depth and width for different scales of multi-relational graphs. RioGNN can learn more discriminative node embedding with enhanced explainability due to the recognition of individual importance of each relation via the filtering threshold mechanism. Comprehensive experiments on real-world graph data and practical tasks demonstrate the advancements of effectiveness, efficiency, and the model explainability, as opposed to other comparative GNN models.

show abstract

Action Space Shaping in Deep Reinforcement Learning

Cited by 71 publications

References 9 publications

Reinforcement learning based resource management for fog computing environment: Literature review, challenges, and open issues

Reinforcement learning based resource management for fog computing environment: Literature review, challenges, and open issues

Deep Reinforcement Learning Techniques For Solving Hybrid Flow Shop Scheduling Problems: Proximal Policy Optimization (PPO) and Asynchronous Advantage Actor-Critic (A3C)

Reinforced Neighborhood Selection Guided Multi-Relational Graph Neural Networks

Contact Info

Product

Resources

About