Creating Autonomous Adaptive Agents in a Real-Time First-Person Shooter Computer Game

Wang, Di; Tan, Ah-Hwee

doi:10.1109/tciaig.2014.2336702

Cited by 55 publications

(51 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In a certain way, this AI never truly understands the goal of the game nor the multiple distinct components that can cause a player to win or lose a game. Wang & Tang [8] tackles briefly these problems by creating two distinct learning processes: learning the appropriate behavior and focusing on optimal weapon selection.…”

Section: Related Workmentioning

confidence: 99%

Narrow Artificial Intelligence with Machine Learning for Real-Time Estimation of a Mobile Agent’s Location Using Hidden Markov Models

Beaulac

Larribe

2017

International Journal of Computer Games Technology

View full text Add to dashboard Cite

We propose to use a supervised machine learning technique to track the location of a mobile agent in real time. Hidden Markov Models are used to build artificial intelligence that estimates the unknown position of a mobile target moving in a defined environment. This narrow artificial intelligence performs two distinct tasks. First, it provides real-time estimation of the mobile agent's position using the forward algorithm. Second, it uses the Baum-Welch algorithm as a statistical learning tool to gain knowledge of the mobile target. Finally, an experimental environment is proposed, namely a video game that we use to test our artificial intelligence. We present statistical and graphical results to illustrate the efficiency of our method.

show abstract

Section: Related Workmentioning

confidence: 99%

Narrow Artificial Intelligence with Machine Learning for Real-Time Estimation of a Mobile Agent’s Location Using Hidden Markov Models

Beaulac

Larribe

2017

International Journal of Computer Games Technology

View full text Add to dashboard Cite

show abstract

“…A fusion ART network can also be considered as a building block for more complex memory or cognitive architecture. In [16], it is used as the building block for modelling episodic memory comprising memory formation forgetting, and then consolidation processes. This model of episodic memory has been demonstrated to handle memory tasks in a fast-paced real-time first-person shooter video game environment [17].…”

Section: Budhitama Subagdja Dynamic and Incremental Exploration Stramentioning

confidence: 99%

Dynamic and Incremental Exploration Strategy in Fusion Adaptive Resonance Theory for Online Reinforcement Learning

Subagdja

2016

Jurnal Ilmu Komputer dan Informasi

View full text Add to dashboard Cite

One of the fundamental challenges in reinforcement learning is to setup a proper balance between exploration and exploitation to obtain the maximum cummulative reward in the long run. Most protocols for exploration bound the overall values to a convergent level of performance. If new knowledge is inserted or the environment is suddenly changed, the issue becomes more intricate as the exploration must compromise the pre-existing knowledge. This paper presents a type of multi-channel adaptive resonance theory (ART) neural network model called fusion ART which serves as a fuzzy approximator for reinforcement learning with inherent features that can regulate the exploration strategy. This intrinsic regulation is driven by the condition of the knowledge learnt so far by the agent. The model offers a stable but incremental reinforcement learning that can involve prior rules as bootstrap knowledge for guiding the agent to select the right action. Experiments in obstacle avoidance and navigation tasks demonstrate that in the configuration of learning wherein the agent learns from scratch, the inherent exploration model in fusion ART model is comparable to the basic E-greedy policy. On the other hand, the model is demonstrated to deal with prior knowledge and strike a balance between exploration and exploitation. Keywords: Reinforcement Learning, Exploration Strategy, Adaptive Resonance Theory AbstrakSalah satu permasalahan mendasar dari Reinforcement Learning adalah mengatur keseimbangan antara eksplorasi dan eksploitasi untuk mendapatkan ganjaran (reward) maksimal secara kumulatif dalam jangka waktu yang lama. Ketika pengetahuan awal diikutsertakan, masalah muncul karena eksplorasi yang dilakukan harus dikompromikan dengan pengetahuan sebelumnya yang telah dipelajari. Makalah ini menampilkan salah satu jenis jaringan saraf tiruan adaptive resonance theory (ART) berkanal ganda yang dikenal juga dengan sebutan fusion ART yang juga merupakan aproksimator Fuzzy untuk reinforcement learning dengan kemampuan meregulasi strategi eksplorasi sebagai sifat dasarnya. Model ini menawarkan proses pembelajaran yang stabil tetapi inkremental serta mampu melibatkan pengetahuan awal yang memilih aksi yang benar. Eksperimen menggunakan navigasi dan menghindari rintangan sebagai domain masalah menunjukkan bahwa konfigurasi pembelajaran menggunakan sifat dasar untuk meregulasi eksplorasi sebanding dengan metoda umum yang menggunakan aturan Egreedy. Di lain pihak, model yang diusulkan ini juga menunjukkan kemampuan dalam menggunakan pengetahuan awal serta mencapai keseimbangan dalam eksplorasi dan eksploitasi pengetahuan.

show abstract

“…The real-time collection of realistic crowdsourced data can also unveil opportunities in the area of Artificial Intelligence, such as modeling human-like behavior [9][10][11] for social and socio-economic studies as well as for marketing and/or entertainment purposes. The latter is traditionally linked to game AI [12,13], which as a term is mainly used for describing non player characters (NPCs).…”

Section: Introductionmentioning

confidence: 99%

“…(see e.g. [10,11,14]). However, most of these modern games rely on desktop-like game-playing [11] utilizing data based on the human's decisions only and ignoring spatio-temporal or other sensory information.…”

Section: Introductionmentioning

confidence: 99%

“…[10,11,14]). However, most of these modern games rely on desktop-like game-playing [11] utilizing data based on the human's decisions only and ignoring spatio-temporal or other sensory information. Here it is important to note that the year 2011 has been extremely important for the Computing field, as the number of smartphones exceeded for the first time in history the number of all types of Personal Computers combined (i.e., Notebooks, Tablets, Netbooks and Desktops), marking the beginning of the post-PC era 2 and therefore the beginning of real-time Smartphone gaming.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Human-Like Agents for a Smartphone First Person Shooter Game Using Crowdsourced Data

Kronis

Konstantinidis

Papadopoulos

2013

IFIP Advances in Information and Communication Technology

View full text Add to dashboard Cite

The evolution of Smartphone devices with their powerful computing capabilities and their ever increasing number of sensors has recently introduced an unprecedented array of applications and games. The Smartphone users who are constantly moving and sensing are able to provide large amounts of opportunistic/participatory data that can contribute to complex and novel problem solving, unfolding in this way the full potential of crowdsourcing. Crowdsourced data can therefore be utilized for optimally modeling human-like behavior and improving the realizablity of AI gaming. In this study, we have developed an Augmented Reality First Person Shooter game, coined AR Shooter, that allows the crowd to constantly contribute their game play along with various spatio-temporal information. The crowdsourced data are used for modeling the human player's behavior with Artificial Neural Networks. The resulting models are utilized back to the game's environment through AI agents making it more realistic and challenging. Our experimental studies have shown that our AI agents are quite competitive, while being very difficult to distinguish from human players.

show abstract

Creating Autonomous Adaptive Agents in a Real-Time First-Person Shooter Computer Game

Cited by 55 publications

References 36 publications

Narrow Artificial Intelligence with Machine Learning for Real-Time Estimation of a Mobile Agent’s Location Using Hidden Markov Models

Narrow Artificial Intelligence with Machine Learning for Real-Time Estimation of a Mobile Agent’s Location Using Hidden Markov Models

Dynamic and Incremental Exploration Strategy in Fusion Adaptive Resonance Theory for Online Reinforcement Learning

Human-Like Agents for a Smartphone First Person Shooter Game Using Crowdsourced Data

Contact Info

Product

Resources

About