SEIHAI: A Sample-Efficient Hierarchical AI for the MineRL Competition

Mao, Hangyu; Wang, Chao; Hao, Xiaotian; Mao, Yihuan; Lu, Yao; Wu, Chengjie; Hao, Jianye; Liu, Dong; Tang, Pingzhong

doi:10.1007/978-3-030-94662-3_3

Cited by 8 publications

(6 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Some previous works leverage hierarchical reinforcement learning framework to develop embodied agents. For example, SEIHAI [30] splits a long-horizon task into several subtasks, trains a suitable agent for each subtask, and designs a scheduler to invoke these agents. JueWu-MC [28] follows a similar idea but equipped the agent with action-aware representation learning capability.…”

Section: Related Workmentioning

confidence: 99%

Open-World Multi-Task Control Through Goal-Aware Representation Learning and Adaptive Horizon Prediction

Cai¹,

Wang²,

Ma³

et al. 2023

Preprint

View full text Add to dashboard Cite

We study the problem of learning goal-conditioned policies in Minecraft, a popular, widely accessible yet challenging open-ended environment for developing human-level multi-task agents. We first identify two main challenges of learning such policies: 1) the indistinguishability of tasks from the state distribution, due to the vast scene diversity, and 2) the non-stationary nature of environment dynamics caused by partial observability. To tackle the first challenge, we propose Goal-Sensitive Backbone (GSB) for the policy to encourage the emergence of goal-relevant visual state representations. To tackle the second challenge, the policy is further fueled by an adaptive horizon prediction module that helps alleviate the learning uncertainty brought by the non-stationary dynamics. Experiments on 20 Minecraft tasks show that our method significantly outperforms the best baseline so far; in many of them, we double the performance. Our ablation and exploratory studies then explain how our approach beat the counterparts and also unveil the surprising bonus of zero-shot generalization to new scenes (biomes). We hope our agent could help shed some light on learning goal-conditioned, multi-task agents in challenging, open-ended environments like Minecraft. The code is released at https://github.com/CraftJarvis/ MC-Controller.

show abstract

Section: Related Workmentioning

confidence: 99%

Open-World Multi-Task Control Through Goal-Aware Representation Learning and Adaptive Horizon Prediction

Cai¹,

Wang²,

Ma³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Table 2 shows the final ranking of the intro and research tracks, respectively. The overall winner of the research track outperformed last year's top solution (Mao et al, 2021) by a large margin. 5 Solutions in the intro track reached notably higher scores (top three average of 479.3 vs. 48.5 for research Round 1), due to fewer restrictions on usable techniques.…”

Section: Research Track Submissions Overviewmentioning

confidence: 99%

MineRL Diamond 2021 Competition: Overview, Results, and Lessons Learned

Kanervisto¹,

Milani²,

Ramanauskas³

et al. 2022

Preprint

View full text Add to dashboard Cite

Reinforcement learning competitions advance the field by providing appropriate scope and support to develop solutions toward a specific problem. To promote the development of more broadly applicable methods, organizers need to enforce the use of general techniques, the use of sample-efficient methods, and the reproducibility of the results. While beneficial for the research community, these restrictions come at a cost-increased difficulty. If the barrier for entry is too high, many potential participants are demoralized. With this in mind, we hosted the third edition of the MineRL ObtainDiamond competition, MineRL Diamond 2021, with a separate track in which we permitted any solution to promote the participation of newcomers. With this track and more extensive tutorials and support, we saw an increased number of submissions. The participants of this easier track were able to obtain a diamond, and the participants of the harder track progressed the generalizable solutions in the same task.

show abstract

“…It designed a two-stage pipeline, first mastering the prerequisite skills with parameterization trick, and then learning a meta controller to execute the instructions. Moving to solve complex long-horizon tasks in Minecraft, works (Lin et al, 2021;Mao et al, 2022;Oh et al, 2017) explored the hierarchical architecture. In recent years, influenced by the trend of large-scale pre-training paradigms, a group of researchers have emerged, who are utilizing vast amounts of internet knowledge to train intelligent agents.…”

Section: Minecraft Agentsmentioning

confidence: 99%

Fast and accurate DNASeq Variant Calling workflow composed of LUSH toolkit

Wang¹,

Zhang²,

Wang³

et al. 2023

Preprint

View full text Add to dashboard Cite

BackgroundWhole genome sequencing (WGS) is becoming increasingly prevalent for molecular diagnosis, staging and prognosis because of its declining costs and the ability to detect nearly all genes associated with a patient’s disease. The currently widely accepted variant calling pipeline, GATK, is limited in terms of its computational speed and efficiency, which cannot meet the growing analysis needs.MethodsIn this study, we propose a fast and accurate DNASeq variant calling workflow that is purely composed of tools from LUSH toolkit. The LUSH pipeline is highly optimized for the WGS pipeline based on SOAPnuke, BWA and GATK which can be deployed on any general-purpose CPU-based computing system. We validated the accuracy, speed and scalability of the LUSH pipeline on several standard WGS datasets.ResultsOur test results show that the LUSH pipeline and the GATK pipeline are highly consistent in terms of accuracy, achieving over 99% precision and recall on NA12878. For speed, the LUSH pipeline completes 30x WGS data in 1.6 hours, which is about 17x faster than the GATK pipeline. From BAM to VCF, LUSH_HC even takes only 12 minutes, about 76x faster than GATK. Moreover, the LUSH pipeline shows favorable scalability in terms of thread and sequencing depth.ConclusionThe LUSH pipeline provides far superior computational speed to GATK while maintaining a high level of accuracy comparable to that of GATK, which greatly facilitates bedside analysis of acute patients, large-scale cohort data analysis, and variant calling in crop breeding programs.

show abstract

SEIHAI: A Sample-Efficient Hierarchical AI for the MineRL Competition

Cited by 8 publications

References 12 publications

Open-World Multi-Task Control Through Goal-Aware Representation Learning and Adaptive Horizon Prediction

Open-World Multi-Task Control Through Goal-Aware Representation Learning and Adaptive Horizon Prediction

MineRL Diamond 2021 Competition: Overview, Results, and Lessons Learned

Fast and accurate DNASeq Variant Calling workflow composed of LUSH toolkit

Contact Info

Product

Resources

About