Building Controllers for Tetris

Thiéry, Christophe; Scherrer, Bruno

doi:10.3233/icg-2009-32102

Cited by 26 publications

(22 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Even if not described in the paper, extension of ℓ 1 -PBR to the state-action value function approximation is straightforward. In the future, we plan to perform a deeper theoretical study of the proposed approach (the analysis of [7] in the case of ℓ 2 -penalized LSTD can be a lead) and to apply it to control problems (notably Tetris [23] should be an interesting application, as features are quite interpretable).…”

Section: Resultsmentioning

confidence: 99%

ℓ1-Penalized Projected Bellman Residual

Geist

Scherrer

2012

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

Abstract. We consider the task of feature selection for value function approximation in reinforcement learning. A promising approach consists in combining the Least-Squares Temporal Difference (LSTD) algorithm with ℓ1-regularization, which has proven to be effective in the supervised learning community. This has been done recently whit the LARS-TD algorithm, which replaces the projection operator of LSTD with an ℓ1-penalized projection and solves the corresponding fixed-point problem. However, this approach is not guaranteed to be correct in the general offpolicy setting. We take a different route by adding an ℓ1-penalty term to the projected Bellman residual, which requires weaker assumptions while offering a comparable performance. However, this comes at the cost of a higher computational complexity if only a part of the regularization path is computed. Nevertheless, our approach ends up to a supervised learning problem, which let envision easy extensions to other penalties.

show abstract

Section: Resultsmentioning

confidence: 99%

ℓ1-Penalized Projected Bellman Residual

Geist

Scherrer

2012

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

show abstract

“…The original Tetris had been a popular benchmark in RL [16], but it became hard to study when the best policies started achieving the level of tens of millions cleared lines on average [23]. Learning and evaluating such controllers is computationally expensive.…”

Section: Game Descriptionmentioning

confidence: 99%

High-Dimensional Function Approximation for Knowledge-Free Reinforcement Learning

Jaśkowski

Szubert

Liskowski

et al. 2015

Proceedings of the 2015 Annual Conference on Genetic and Evolutionary Computation

View full text Add to dashboard Cite

SZ-Tetris, a restricted version of Tetris, is a difficult reinforcement learning task. Previous research showed that, similarly to the original Tetris, value function-based methods such as temporal difference learning, do not work well for SZ-Tetris. The best performance in this game was achieved by employing direct policy search techniques, in particular the cross-entropy method in combination with handcrafted features. Nonetheless, a simple heuristic hand-coded player scores even higher. Here we show that it is possible to equal its performance with CMA-ES (Covariance Matrix Adaptation Evolution Strategy). We demonstrate that further improvement is possible by employing systematic n-tuple network, a knowledge-free function approximator, and VD-CMA-ES, a linear variant of CMA-ES for high dimension optimization. Last but not least, we show that a large systematic n-tuple network (involving more than 4 million parameters) allows the classical temporal difference learning algorithm to obtain similar average performance to VD-CMA-ES, but at 20 times lower computational expense, leading to the best policy for SZ-Tetris known to date. These results enrich the current understanding of difficulty of SZ-Tetris, and shed new light on the capabilities of particular search paradigms when applied to representations of various characteristics and dimensionality.

show abstract

“…In the literature, many authors have introduced different features to synthesize the state of the game, [19] gives a good review of these features. In this work we use eight features six of them are the features introduced by Dellacherie [7] for his artificial player, and two were introduced by [20].…”

Section: Previous Workmentioning

confidence: 99%

On the evolution of artificial Tetris players

Boumaza

2009

2009 IEEE Symposium on Computational Intelligence and Games

View full text Add to dashboard Cite

In the paper, we focus the use of evolutionary algorithms to learn strategies to play the game of Tetris. We describe the problem and discuss the nature of the search space. We present experiments to illustrate the learning process of our artificial player, and provide a new procedure to speed up the learning time. The results we present compare with the best known artificial player, and show how our evolutionary algorithm is able to rediscover player strategies previously published. Finally we provide some ideas to improve the performance of artificial Tetris players.

show abstract

Building Controllers for Tetris

Cited by 26 publications

References 13 publications

ℓ1-Penalized Projected Bellman Residual

ℓ1-Penalized Projected Bellman Residual

High-Dimensional Function Approximation for Knowledge-Free Reinforcement Learning

On the evolution of artificial Tetris players

Contact Info

Product

Resources

About