How do people predict a random walk? Lessons for models of human cognition

Abstract

Repeated forecasts of changing targets are a key aspect of many everyday tasks, from predicting the weather to financial markets. Random walks provide a particularly simple and informative case study, as new values represent random deviations from the preceding value only, with further previous points being irrelevant. Moreover, random walks often hold simple rational solutions in which predictions should repeat the most recent state, and hence replicate the properties of the target. In previous experiments, however, we have found that human forecasters do not adhere to this standard, showing systematic deviations from the properties of a random walk such as excessive volatility and extreme movements between subsequent predictions. We suggest that such deviations reflect general statistical signatures of human cognition displayed across multiple tasks, offering a window into underlying cognitive mechanisms. Using these deviations as new criteria, we here explore several cognitive models for predicting random walks drawn from various approaches developed in the existing literature, including Bayesian, error-based learning, autoregressive and sampling mechanisms. These models are contrasted to determine which best accounts for the particular statistical features displayed by experimental participants. We find support for sampling models in both aggregate and individual fits, suggesting that these variations are attributable to the use of inherently stochastic prediction systems. We thus argue that variability in predictions is driven by computational noise in the decision making process, rather than ``late'' noise at the output stage.

Noise in Cognition: Bug or Feature?

“…The multiple chains of MC 3 introduce human-like long-range autocorrelations (see Figure 3D 3 ), and surprisingly, in both fixed target and random walk target tasks, this local sampling algorithm also produces almost no autocorrelations in the changes between estimates, but does show autocorrelations in the magnitude of these changes (see Figure 3E 3 ; Zhu, Sundh, et al, 2021). MC 3 better fit the overwhelming majority of participants in a price prediction task than non-sampling models of human behavior (Spicer, Zhu, et al, 2022a). Castillo et al (2022) showed that 'volitional' random generation could be explained as resulting from a related MCMC algorithm.…”

“…But these sources could potentially be minor contributors, as argued by Drugowitsch et al (2016). Relatedly, Spicer, Zhu, et al (2022a) showed that almost all of their participants did not seem to exhibit any response noise (i.e., sampling algorithms without response noise fit the data better than sampling algorithms with response noise) after 0.37% of responses were excluded as outliers. However, sensory noise is still useful for explaining Weber's Law scaling of noise with stimulus magnitudes, as it is not obvious why that would arise from the sampling from hypotheses, and refining models of computational noise will in the future allow for more accurate partitioning of the different sources of noise.…”

“…The remaining chains are useful for exploring the distribution, because, when appropriate, one of these remaining chains can swap positions with the MCMC chain sampling from the correct target distribution. In this way, MC 3 can make occasional large jumps across the hypothesis space, allowing it also produce heavy-tailed changes even when sampling from a fixed Gaussian distribution (see Figure 3C 3 ; Zhu et al, 2022), or when the target follows a Gaussian random walk in either time interval estimation tasks or price prediction tasks (Spicer, Zhu, et al, 2022a;Zhu, Sundh, et al, 2021).…”

Comparing serial reproduction and serial prediction of random walk

“…The same four versions of the models in experiment 1 were fitted to the serial prediction data in experiment 2. There were other models such as the MCMC model (Spicer et al, 2022) that may explain human's prediction of random walk better stepwise, but our interest was in the serial effects rather than the accuracy of prediction, and thus we did not implement and extend such models. We encourage future work to explore serial extensions of the random walk prediction models.…”

“…That said, if humans used the optimal strategy for making predictions, the outcomes of serial reproductions and serial predictions of random walk series would have been similar. In fact, there was good evidence that humans' predictions of random walk series deviate from this optimal strategy (Spicer, Zhu, Chater, & Sanborn, 2022;Zhu, Spicer, Sanborn, & Chater, 2021), but none of them so far focused on the serial effects of random walk predictions.…”