In multialternative risky choice, we are often faced with the opportunity to allocate our limited information-gathering capacity between several options before receiving feedback. In such cases, we face a natural trade-off between breadth—spreading our capacity across many options—and depth—gaining more information about a smaller number of options. Despite its broad relevance to daily life, including in many naturalistic foraging situations, the optimal strategy in the breadth–depth trade-off has not been delineated. Here, we formalize the breadth–depth dilemma through a finite-sample capacity model. We find that, if capacity is small (∼10 samples), it is optimal to draw one sample per alternative, favoring breadth. However, for larger capacities, a sharp transition is observed, and it becomes best to deeply sample a very small fraction of alternatives, which roughly decreases with the square root of capacity. Thus, ignoring most options, even when capacity is large enough to shallowly sample all of them, is a signature of optimal behavior. Our results also provide a rich casuistic for metareasoning in multialternative decisions with bounded capacity using close-to-optimal heuristics.

When facing many options, we narrow down our focus to very few of them. Although behaviors like this can be a sign of heuristics, they can actually be optimal under limited cognitive resources. Here we study the problem of how to optimally allocate limited sampling time to multiple options, modelled as accumulators of noisy evidence, to determine the most profitable one. We show that the effective sampling capacity of an agent increases with both available time and the discriminability of the options, and optimal policies undergo a sharp transition as a function of it. For small capacity, it is best to allocate time evenly to exactly five options and to ignore all the others, regardless of the prior distribution of rewards. For large capacities, the optimal number of sampled accumulators grows sub-linearly, closely following a power law for a wide variety of priors. We find that allocating equal times to the sampled accumulators is better than using uneven time allocations. Our work highlights that multi-alternative decisions are endowed with breadth-depth tradeoffs, demonstrates how their optimal solutions depend on the amount of limited resources and the variability of the environment, and shows that narrowing down to a handful of options is always optimal for small capacities.

When facing many options, we narrow down our focus to very few of them. Although behaviors like this can be a sign of heuristics, they can actually be optimal under limited cognitive resources. Here, we study the problem of how to optimally allocate limited sampling time to multiple options, modeled as accumulators of noisy evidence, to determine the most profitable one. We show that the effective sampling capacity of an agent increases with both available time and the discriminability of the options, and optimal policies undergo a sharp transition as a function of it. For small capacity, it is best to allocate time evenly to exactly five options and to ignore all the others, regardless of the prior distribution of rewards. For large capacities, the optimal number of sampled accumulators grows sublinearly, closely following a power law as a function of capacity for a wide variety of priors. We find that allocating equal times to the sampled accumulators is better than using uneven time allocations. Our work highlights that multialternative decisions are endowed with breadth–depth tradeoffs, demonstrates how their optimal solutions depend on the amount of limited resources and the variability of the environment, and shows that narrowing down to a handful of options is always optimal for small capacities.

23 24 25 26 Acknowledgements: This work is supported by the Howard Hughes Medical Institute 27 (HHMI, ref 55008742), MINECO (Spain; BFU2017-85936-P) and ICREA Academia 28 (2016) to R.M.-B.; by NIH (DA037229) to B.Y.H.; by a scholar award from the James 29 S. McDonnell Foundation (grant# 220020462) to J.D.Abstract 34 Decision-makers are often faced with limited information about the outcomes of 35 their choices. Current formalizations of uncertain choice, such as the explore-exploit 36 dilemma, do not apply well to decisions in which search capacity can be allocated to each 37 option in variable amounts. Such choices confront decision-makers with the need to 38 tradeoff between breadth -allocating a small amount of capacity to each of many options 39 -and depth -focusing capacity on a few options. We formalize the breadth-depth 40 dilemma through a finite sample capacity model. We find that, if capacity is smaller than 41 4-7 samples, it is optimal to draw one sample per alternative, favoring breadth. However, 42 for larger capacities, a sharp transition is observed, and it becomes best to deeply sample 43 a very small fraction of alternatives, that decreases with the square root of capacity. Thus, 44 ignoring most options, even when capacity is large enough to shallowly sample all of 45 them, reflects a signature of optimal behavior. Our results also provide a rich casuistic 46 for metareasoning in multi-alternative decisions with bounded capacity. 47 48 49 50 51 52 53 54 55 56 57The breadth-depth (BD) dilemma is a ubiquitous problem in decision-making. 58Consider the example of going to graduate school, where one can enroll in many courses 59 in many topics. Let us assume that the goal is to determine the single one topic that is 60 most relevant, the one that will grant us a job. Should I enroll in few courses in many 61 topics -breadth search-at the risk of not learning enough about any topic to tell which 62 one is the best? Or should I enroll in many courses in very few topics -depth search-at 63 the risk of missing the really exciting topic for the future? One crucial element of this 64 type of decision is that the allocation of resources (time, in this case) needs to be done in 65 advance, before feedback is received (before classes start). Also, once decided, the 66 strategy cannot be changed on the fly, as doing so would be very costly. The BD dilemma 67 is popular in tree search algorithms (Horowitz and Sahni, 1978;Korf, 1985) and in 68 optimizing menu designs (Miller, 1981). It is also one faced by humans and other 69 foragers in many situations, as when we plan, schedule, or invest with finite resources. It 70 is remarkable that the bulk of research on the BD has been in fields outside of psychology 71 (e.g. (Halpert, 1958;Schwartz et al., 2009;Turner et al., 2002). We believe that one 72 reason is the lack of standard sets of tools for thinking about the problem and separating 73 it from other dilemmas. 74Many features of the BD dilemma warrant its study in isolation. First, BD 75 decisions are...

Intrinsic motivation generates behaviors that do not necessarily lead to immediate reward, but help exploration and learning. Here we show that agents having the sole goal of maximizing occupancy of future actions and states, that is, moving and exploring on the long term, are capable of complex behavior without any reference to external rewards. We find that action-state path entropy is the only measure consistent with additivity and other intuitive properties of expected future action-state path occupancy. We provide analytical expressions that relate the optimal policy with the optimal state-value function, from where we prove uniqueness of the solution of the associated Bellman equation and convergence of our algorithm to the optimal state-value function. Using discrete and continuous state tasks, we show that 'dancing', hide-and-seek and a basic form of altruistic behavior naturally result from entropy seeking without external rewards. Intrinsically motivated agents can objectively determine what states constitute rewards, exploiting them to ultimately maximize action-state path entropy.