Protection from uncertainty in the exploration/exploitation trade-off.

Abstract

The exploration/exploitation trade-off (EE trade-off) describes how, when faced with several competing alternatives, decision-makers must often choose between a known good alternative (exploitation) and one or more unknown but potentially more rewarding alternatives (exploration). Prevailing theory on how humans perform the EE trade-off states that uncertainty is a major motivator for exploration: the more uncertain the environment, the more exploration that will occur. The current paper examines whether exploratory behaviour in both choice and attention may be impacted differently depending on whether uncertainty is onset suddenly (unexpected uncertainty), or more slowly (expected uncertainty). It is shown that when uncertainty was expected, participants tended to explore less with their choices, but not their attention, than when it was unexpected. Crucially, the impact of this "protection from uncertainty" on exploration only occurred when participants had an opportunity to learn the structure of the task prior to experiencing uncertainty. This suggests that the interaction between uncertainty and exploration is more nuanced than simply more uncertainty leading to more exploration, and that attention and choice behaviour may index separate aspects of the EE trade-off.

The role of intolerance of uncertainty when solving the exploration-exploitation dilemma

“…In addition, the task had a low level of uncertainty, as the stimuli and probabilities were fixed, something that may have resulted in low levels of exploration and as such low variability in the data. Future tasks could increase the level of uncertainty, with for example different stimuli presented across trials, may increase such uncertainty and lead to more exploratory behaviors, and as such more variability in the data (Morriss et al, 2019;Walker et al, 2021). Lastly, our findings could point to the absence of a relation between IU and EED, at least on how these variables were operationalized here.…”

Variability and harshness shape flexible strategy-use in support of the constrained flexibility framework

“…Support for the CFF comes from diverse fields. For example, in decision-making and foraging tasks where participants must choose to either exploit a current resource or explore alternatives, more exploratory or information-seeking behavior is observed in variable compared to stable contexts 45 – 49 and this effect is more pronounced in less predictable decision-environments 50 – 52 . Although the impact of harshness on elective flexibility is much less studied, a pronounced suppression of explorative behavior in response to risk is observed in both humans (e.g., loss-aversion bias 53 ) and nonhuman animals 54 .…”

Variability and harshness shape flexible strategy-use, support for the Constrained Flexibility Framework

“…Support for the CFF comes from diverse elds. For example, in decision-making and foraging tasks where participants must choose to either exploit a current resource or explore alternatives, more exploratory or information-seeking behavior is observed in variable compared to stable contexts [42][43][44][45][46] and this effect is more pronounced in less predictable decision-environments [47][48][49] . Although the impact of harshness on elective exibility is much less studied, a pronounced suppression of explorative behavior in response to risk is observed in both humans (e.g., loss-aversion bias 50 ) and nonhuman animals 51 .…”