Action learning versus strategy learning

Hanaki, Nobuyuki

doi:10.1002/cplx.20036

Cited by 3 publications

(4 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We have compared the standard action-based models, namely, WFP and RL with a more modern strategy-based approach [Hanaki, 2004;Ioannou, Romero, 2014]. We significantly extended this approach by focusing on partial, «elementary strategies» instead of complete automata.…”

Section: Discussionmentioning

confidence: 99%

“…Conditional strategy-based models also relate to the experimental paper on finite automata, (first proposed by Aumann, 1981) by Hanaki (2004), who introduces conditional learning strategies into the model of learning (see also [Ioannou, Romero, 2014]).…”

Section: Related Literaturementioning

confidence: 99%

“…Optimal behavior against such players should be more than simple actions and, according, to Hanaki (2004), it may suffice to make strategies that condirion actions to th history of the previous round as first proposed by Aumann (1981). For the 2×2 games, the set of strategies constructed in that way provides a manageable learning model because the total number of distinct decision rules does not exceed 2 5 = 32, but for larger games the number of such rules grows too quickly to be tractable by a human agent 2 .…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

How to Learn to Defeat Noisy Robot in Rock-Paper-Scissors Game: An Exploratory Study

Chernov

2020

HSE Economic Journal

View full text Add to dashboard Cite

This paper studies learning in strategic environment using experimental data from the Rock-Paper-Scissors game. In a repeated game framework, we explore the response of human subjects to uncertain behavior of strategically sophisticated opponent. We model this opponent as a robot who played a stationary strategy with superimposed noise varying across four experimental treatments. Using experimental data from 85 subjects playing against such a stationary robot for 100 periods, we show that humans can decode their strategies, on average outperforming the random response to such a robot by 17%. Further, we show that human ability to recognize such strategies decreases with exogenous noise in the behavior of the robot. Further, we fit learning data to classical Reinforcement Learning (RL) and Fictitious Play (FP) models and show that the classic action-based approach to learning is inferior to the strategy-based one. Unlike the previous papers in this field, e.g. Ioannou, Romero (2014), we extend and adapt the strategy-based learning techniques to the 3×3 game. We also show, using a combination of experimental and ex-post survey data, that human participants are better at learning separate components of an opponent's strategy than in recognizing this strategy as a whole. This decomposition offers them a shorter and more intuitive way to figure out their own best response. We build a strategic extension of the classical learning models accounting for these behavioral phenomena.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Related Literaturementioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

How to Learn to Defeat Noisy Robot in Rock-Paper-Scissors Game: An Exploratory Study

Chernov

2020

HSE Economic Journal

View full text Add to dashboard Cite

show abstract

“…e reward sparsity problem makes the training process be much more difficult and slower because it cannot get a timely and effective reward. To solve this problem, many solutions such as reward reshaping, hierarchical DRL, and curriculum learning are proposed in recent years [12][13][14][15][16][17][18]. Among these methods, the intrinsic reward mechanism plays an important role.…”

Section: Introductionmentioning

confidence: 99%

AIBPO: Combine the Intrinsic Reward and Auxiliary Task for 3D Strategy Game

Cao

Wang

et al. 2021

Complexity

View full text Add to dashboard Cite

In recent years, deep reinforcement learning (DRL) achieves great success in many fields, especially in the field of games, such as AlphaGo, AlphaZero, and AlphaStar. However, due to the reward sparsity problem, the traditional DRL-based method shows limited performance in 3D games, which contain much higher dimension of state space. To solve this problem, in this paper, we propose an intrinsic-based policy optimization (IBPO) algorithm for reward sparsity. In the IBPO, a novel intrinsic reward is integrated into the value network, which provides an additional reward in the environment with sparse reward, so as to accelerate the training. Besides, to deal with the problem of value estimation bias, we further design three types of auxiliary tasks, which can evaluate the state value and the action more accurately in 3D scenes. Finally, a framework of auxiliary intrinsic-based policy optimization (AIBPO) is proposed, which improves the performance of the IBPO. The experimental results show that the method is able to deal with the reward sparsity problem effectively. Therefore, the proposed method may be applied to real-world scenarios, such as 3-dimensional navigation and automatic driving, which can improve the sample utilization to reduce the cost of interactive sample collected by the real equipment.

show abstract

Data-driven dynamic decision models

Nay

Gilligan

2015

2015 Winter Simulation Conference (WSC)

View full text Add to dashboard Cite

This article outlines a method for automatically generating models of dynamic decision-making that both have strong predictive power and are interpretable in human terms. This is useful for designing empirically grounded agent-based simulations and for gaining direct insight into observed dynamic processes. We use an efficient model representation and a genetic algorithmbased estimation process to generate simple approximations that explain most of the structure of complex stochastic processes. This method, implemented in C ++ and R, scales well to large data sets. We apply our methods to empirical data from human subjects game experiments *

show abstract

Action learning versus strategy learning

Abstract: This article seeks to ascertain whether the strategy-learning model of Hanaki, Sethi, Erev, and Peterhansl (2003)

Cited by 3 publications

References 14 publications

How to Learn to Defeat Noisy Robot in Rock-Paper-Scissors Game: An Exploratory Study

How to Learn to Defeat Noisy Robot in Rock-Paper-Scissors Game: An Exploratory Study

AIBPO: Combine the Intrinsic Reward and Auxiliary Task for 3D Strategy Game

Data-driven dynamic decision models

Contact Info

Product

Resources

About