Modeling human multitasking behavior in video games through modular reinforcement learning

Guo, Sihang; Masetty, Bharath; Zhang, Ruohan; Ballard, Dana H.; Hayhoe, Mary

doi:10.1167/jov.20.11.1552

Cited by 2 publications

(2 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There is also mounting evidence for separated value functions in the human brain ( 84 , 85 ) and that decision dynamics are best modeled using competing value components ( 86 ). Others have proposed that opposed serotonin and dopamine learning systems reflect competition between optimistic and pessimistic behavioral policies ( 87 ) and that human behavior can be fit best by assuming it reflects modular reinforcement learning ( 88 , 89 ). Our work provides an explanation for these findings, which are all consistent with the idea that different objectives compete for behavioral expression in parallel ( 90 ).…”

Section: Discussionmentioning

confidence: 99%

Having multiple selves helps learning agents explore and adapt in complex changing worlds

Dulberg

Dubey

Berwian

et al. 2023

Proc. Natl. Acad. Sci. U.S.A.

View full text Add to dashboard Cite

Satisfying a variety of conflicting needs in a changing environment is a fundamental challenge for any adaptive agent. Here, we show that designing an agent in a modular fashion as a collection of subagents, each dedicated to a separate need, powerfully enhanced the agent’s capacity to satisfy its overall needs. We used the formalism of deep reinforcement learning to investigate a biologically relevant multiobjective task: continually maintaining homeostasis of a set of physiologic variables. We then conducted simulations in a variety of environments and compared how modular agents performed relative to standard monolithic agents (i.e., agents that aimed to satisfy all needs in an integrated manner using a single aggregate measure of success). Simulations revealed that modular agents a) exhibited a form of exploration that was intrinsic and emergent rather than extrinsically imposed; b) were robust to changes in nonstationary environments, and c) scaled gracefully in their ability to maintain homeostasis as the number of conflicting objectives increased. Supporting analysis suggested that the robustness to changing environments and increasing numbers of needs were due to intrinsic exploration and efficiency of representation afforded by the modular architecture. These results suggest that the normative principles by which agents have adapted to complex changing environments may also explain why humans have long been described as consisting of “multiple selves.”

show abstract

Section: Discussionmentioning

confidence: 99%

Having multiple selves helps learning agents explore and adapt in complex changing worlds

Dulberg

Dubey

Berwian

et al. 2023

Proc. Natl. Acad. Sci. U.S.A.

View full text Add to dashboard Cite

show abstract

“…There is also mounting evidence for separated value functions in the human brain (75, 76), and that decision dynamics are best modelled using competing value components (77). Others have proposed that opposed serotonin and dopamine learning systems reflect competition between optimistic and pessimistic behavioral policies (78), and that human behavior can be fit best by assuming it reflects modular reinforcement learning (79, 80). Our work provides a normative basis for these findings, which are all consistent with the idea that different objectives compete for behavioral expression in parallel (81).…”

Section: Discussionmentioning

confidence: 99%

Having “multiple selves” helps learning agents explore and adapt in complex changing worlds

Dulberg

Dubey

Berwian

et al. 2022

Preprint

View full text Add to dashboard Cite

Satisfying a variety of conflicting needs in a changing environment is a fundamental challenge for any adaptive agent. Here, we show that designing an agent in a modular fashion as a collection of sub-agents, each dedicated to a separate need, powerfully enhanced the agent's capacity to satisfy its overall needs. We used the formalism of deep reinforcement learning to investigate a biologically relevant multi-objective task: continually maintaining homeostasis of a set of physiologic variables. We then conducted simulations in a variety of environments and compared how modular agents performed relative to standard monolithic agents (i.e., agents that aimed to satisfy all needs in an integrated manner using a single aggregate measure of success). Simulations revealed that modular agents: a) exhibited a form of exploration that was intrinsic and emergent rather than extrinsically imposed; b) were robust to changes in non-stationary environments, and c) scaled gracefully in their ability to maintain homeostasis as the number of conflicting objectives increased. Supporting analysis suggested that the robustness to changing environments and increasing numbers of needs were due to intrinsic exploration and efficiency of representation afforded by the modular architecture. These results suggest that the normative principles by which agents have adapted to complex changing environments may also explain why humans have long been described as consisting of 'multiple selves'.

show abstract

Modeling human multitasking behavior in video games through modular reinforcement learning

Cited by 2 publications

References 0 publications

Having multiple selves helps learning agents explore and adapt in complex changing worlds

Having multiple selves helps learning agents explore and adapt in complex changing worlds

Having “multiple selves” helps learning agents explore and adapt in complex changing worlds

Contact Info

Product

Resources

About