Simulation-based optimization of StarCraft tactical AI through evolutionary computation

Othman, Nasri Bin; Decraene, James; Cai, Wentong; Hu, Nan; Low, Malcolm Yoke Hean; Gouaillard, Alexandre

doi:10.1109/cig.2012.6374182

Cited by 25 publications

(8 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Real-time strategy (RTs) games usually have a time-varying scene, which is different from board games [9]. In many traditional RTs games, StarCraft has a large number of players and a large number of competitions, which requires different countermeasures, tactics and even control techniques, so it has attracted the attention of international scholars [10]. StarCraft provides a relatively ideal environment for AI research in video games and it can achieve confrontation decision-making via a computer program.…”

Section: Multi-agent Confrontation In the Real-time Strategy Gamementioning

confidence: 99%

A Confrontation Decision-Making Method with Deep Reinforcement Learning and Knowledge Transfer for Multi-Agent System

2020

Symmetry

View full text Add to dashboard Cite

In this paper, deep reinforcement learning (DRL) and knowledge transfer are used to achieve the effective control of the learning agent for the confrontation in the multi-agent systems. Firstly, a multi-agent Deep Deterministic Policy Gradient (DDPG) algorithm with parameter sharing is proposed to achieve confrontation decision-making of multi-agent. In the process of training, the information of other agents is introduced to the critic network to improve the strategy of confrontation. The parameter sharing mechanism can reduce the loss of experience storage. In the DDPG algorithm, we use four neural networks to generate real-time action and Q-value function respectively and use a momentum mechanism to optimize the training process to accelerate the convergence rate for the neural network. Secondly, this paper introduces an auxiliary controller using a policy-based reinforcement learning (RL) method to achieve the assistant decision-making for the game agent. In addition, an effective reward function is used to help agents balance losses of enemies and our side. Furthermore, this paper also uses the knowledge transfer method to extend the learning model to more complex scenes and improve the generalization of the proposed confrontation model. Two confrontation decision-making experiments are designed to verify the effectiveness of the proposed method. In a small-scale task scenario, the trained agent can successfully learn to fight with the competitors and achieve a good winning rate. For large-scale confrontation scenarios, the knowledge transfer method can gradually improve the decision-making level of the learning agent.

show abstract

Section: Multi-agent Confrontation In the Real-time Strategy Gamementioning

confidence: 99%

A Confrontation Decision-Making Method with Deep Reinforcement Learning and Knowledge Transfer for Multi-Agent System

2020

Symmetry

View full text Add to dashboard Cite

show abstract

“…Although AlphaGo is a milestone in the goal of general artificial intelligence, in general, the problems it represents are still relatively simple compared with the real world [2]. Therefore, in recent years, researchers have focused on real-time strategy (RTS) games [3][4][5][6], such as Dota2 [7] and StarCraft [8,9], which represent dynamic games of asymmetric information beyond the Go game.…”

Section: Introductionmentioning

confidence: 99%

Research and Implementation of Intelligent Decision Based on a Priori Knowledge and DQN Algorithms in Wargame Environment

et al. 2020

View full text Add to dashboard Cite

The reinforcement learning problem of complex action control in a multi-player wargame has been a hot research topic in recent years. In this paper, a game system based on turn-based confrontation is designed and implemented with state-of-the-art deep reinforcement learning models. Specifically, we first design a Q-learning algorithm to achieve intelligent decision-making, which is based on the DQN (Deep Q Network) to model complex game behaviors. Then, an a priori knowledge-based algorithm PK-DQN (Prior Knowledge-Deep Q Network) is introduced to improve the DQN algorithm, which accelerates the convergence speed and stability of the algorithm. The experiments demonstrate the correctness of the PK-DQN algorithm, it is validated, and its performance surpasses the conventional DQN algorithm. Furthermore, the PK-DQN algorithm shows effectiveness in defeating the high level of rule-based opponents, which provides promising results for the exploration of the field of smart chess and intelligent game deduction.

show abstract

“…These advantages have made war games a favourite basis for research in artificial intelligence (AI), statistics, human cognition, machine learning and strategy (e.g. [25][26][27][28][29]). We feel that war games can have as much or an even greater benefit to understanding biological concepts such as animal contests and weapon evolution.…”

Section: Introductionmentioning

confidence: 99%

Cybernetic combatants support the importance of duels in the evolution of extreme weapons

et al. 2020

View full text Add to dashboard Cite

A current evolutionary hypothesis predicts that the most extreme forms of animal weaponry arise in systems where combatants fight each other one-to-one, in duels. It has also been suggested that arms races in human interstate conflicts are more likely to escalate in cases where there are only two opponents. However, directly testing whether duels matter for weapon investment is difficult in animals and impossible in interstate conflicts. Here, we test whether superior combatants experience a disproportionate advantage in duels, as compared with multi-combatant skirmishes, in a system analogous to both animal and military contests: the battles fought by artificial intelligence agents in a computer war game. We found that combatants with experimentally improved fighting power had a large advantage in duels, but that this advantage deteriorated as the complexity of the battlefield was increased by the addition of further combatants. This pattern remained under the two different forms of the advantage granted to our focal artificial intelligence (AI) combatants, and became reversed when we switched the roles to feature a weak focal AI among strong opponents. Our results suggest that one-on-one combat may trigger arms races in diverse systems. These results corroborate the outcomes of studies of both animal and interstate contests, and suggest that elements of animal contest theory may be widely applicable to arms races generally.

show abstract

Simulation-based optimization of StarCraft tactical AI through evolutionary computation

Cited by 25 publications

References 11 publications

A Confrontation Decision-Making Method with Deep Reinforcement Learning and Knowledge Transfer for Multi-Agent System

A Confrontation Decision-Making Method with Deep Reinforcement Learning and Knowledge Transfer for Multi-Agent System

Research and Implementation of Intelligent Decision Based on a Priori Knowledge and DQN Algorithms in Wargame Environment

Cybernetic combatants support the importance of duels in the evolution of extreme weapons

Contact Info

Product

Resources

About