Hierarchical Deep Q-Network from imperfect demonstrations in Minecraft

Skrynnik, Alexey; Staroverov, Aleksey; Aitygulov, Ermek; Aksenov, Kirill; Davydov, Vasilii; Panov, Aleksandr I.

doi:10.1016/j.cogsys.2020.08.012

Cited by 16 publications

(11 citation statements)

References 2 publications

(2 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, the application of emotional model was limited because it adopted symbolic cognitive state and decision-making behavior. In literature [ 22 ], the author proposed an improved emotional interaction model, but it still did not take into account the influence of individual personality characteristics on the emotional model, so it was still limited when it was used to solve decision-making problems. Khare and Bajaj [ 23 ] put forward a multimodal emotion recognition model based on visual signals and physiological signals, which combines facial expression features and ECG features in series at feature level to form multimodal features.…”

Section: Related Workmentioning

confidence: 99%

Emotional Analysis Model for Social Hot Topics of Professional Migrant Workers

Pang

Bao

2022

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

Text makes up a large portion of network data because it is the vehicle for people’s direct expression of emotions and opinions. How to analyze and mine these emotional text data has become a hot topic of concern in academia and industry in recent years. The online LDA (Latent Dirichlet Allocation) model is used in this paper to train the social hot topic data of professional migrant workers on the same time slice, and the subtopic evolution and intensity are obtained. The topic development is divided into four categories, and the classification model is created using SVM (Support Vector Machine). Instead of decision makers, a virtual human with sensibility and rationality is built using a hierarchical emotional cognitive model to solve multiobjective optimization problems interactively. It analyzes human body structure and emotional signals, and then combines them with visual and physiological signals to create multimodal emotional data. An example is used to demonstrate the effectiveness of the proposed model.

show abstract

Section: Related Workmentioning

confidence: 99%

Emotional Analysis Model for Social Hot Topics of Professional Migrant Workers

Pang

Bao

2022

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

show abstract

“…There are lots of relevant studies. For example, HDQfD [31] utilizes the hierarchical structure of expert trajectories, presenting a structured taskdependent replay buffer and an adaptive prioritizing technique to gradually erase poor-quality expert data from the buffer. HDQfD won the first place in 2019.…”

Section: Related Workmentioning

confidence: 99%

SEIHAI: A Sample-efficient Hierarchical AI for the MineRL Competition

Mao¹,

Wang²,

Hao³

et al. 2021

Preprint

View full text Add to dashboard Cite

The MineRL competition is designed for the development of reinforcement learning and imitation learning algorithms that can efficiently leverage human demonstrations to drastically reduce the number of environment interactions needed to solve the complex ObtainDiamond task with sparse rewards. To address the challenge, in this paper, we present SEIHAI, a Sample-efficient Hierarchical AI, that fully takes advantage of the human demonstrations and the task structure. Specifically, we split the task into several sequentially dependent subtasks, and train a suitable agent for each subtask using reinforcement learning and imitation learning. We further design a scheduler to select different agents for different subtasks automatically. SEIHAI takes the first place in the preliminary and final of the NeurIPS-2020 MineRL competition.

show abstract

“…Otherwise, the agent would waste training time by constantly switching between moving forward and backward, effectively jittering in place rather than exploring the environment. Removed actions maybe set to "always on", which was a popular transformation in the Minecraft MineRL competition, where always executing "attack" helped the agents to learn gathering resources [10], [27], [28].…”

Section: Action Space Shaping In Video Game Environmentsmentioning

confidence: 99%

“…These actions are often discretized, either by splitting them into a set of bins, or by defining three discrete choices: negative, zero and positive. This is especially common with camera rotation, where agents can only choose to turn the camera left/right and up/down at a fixed rate per step [10], [27], [28], [36]. A downside is that this turning rate is a hyper-parameter, which requires tuning.…”

Section: Action Space Shaping In Video Game Environmentsmentioning

confidence: 99%

“…Since the resulting action space combinatorially explodes quickly with an increasing MultiDiscrete space, this is usually combined with removing some of the actions. This can be either dropping unnecessary actions as described above, manually selecting the allowed combinations (as done in MineRL [10], [27]) or by limiting maximum number of pressed buttons (as done in ViZDoom [30], [31]). This transformation is intuitively motivated by the assumption that it is easier to learn a single large policy than multiple small policies, as well as technical limitations of some of the algorithms.…”

Section: Action Space Shaping In Video Game Environmentsmentioning

confidence: 99%

See 1 more Smart Citation

Action Space Shaping in Deep Reinforcement Learning

Kanervisto¹,

Scheller²,

Hautamäki³

2020

Preprint

View full text Add to dashboard Cite

Reinforcement learning (RL) has been successful in training agents in various learning environments, including videogames. However, such work modifies and shrinks the action space from the game's original. This is to avoid trying "pointless" actions and to ease the implementation. Currently, this is mostly done based on intuition, with little systematic research supporting the design decisions. In this work, we aim to gain insight on these action space modifications by conducting extensive experiments in video-game environments. Our results show how domain-specific removal of actions and discretization of continuous actions can be crucial for successful learning. With these insights, we hope to ease the use of RL in new environments, by clarifying what action-spaces are easy to learn.

show abstract

Hierarchical Deep Q-Network from imperfect demonstrations in Minecraft

Cited by 16 publications

References 2 publications

Emotional Analysis Model for Social Hot Topics of Professional Migrant Workers

Emotional Analysis Model for Social Hot Topics of Professional Migrant Workers

SEIHAI: A Sample-efficient Hierarchical AI for the MineRL Competition

Action Space Shaping in Deep Reinforcement Learning

Contact Info

Product

Resources

About