Learners presumably attempt to allocate their study time to maximize reward, yet in some contexts, their study choices are driven by reading biases that would not maximize reward. For instance, when presented with items in a horizontal array that are worth different values if correctly recalled, learners will often first select the leftmost item (i.e., a reading bias), even when it is associated with the lowest value. In four experiments, we investigated the degrees to which various factors cause learners to shift to agenda-based regulation. On each trial, participants were presented with three cues and a point value (1, 3, or 5) for each. The participants could select any cue for study (in which case, its target would be presented) in any order. In Experiment 1, participants either selected items for study under time pressure or were given unlimited time to select items. Not limiting selection time increased the likelihood that higher-valued items would be prioritized for study, but reading biases still influenced item selection. In Experiment 2, participants could select only one item per trial, and higher-valued items were prioritized even more for study, but not exclusively so. In Experiments 3 and 4, we ruled out a lack of motivation and inaccurate task beliefs as explanations for why participants would sometimes choose lower-valued items. The results demonstrate the influence of a pervasive reading bias on learners' item selections, but as importantly, they show that a shift toward agenda use occurs when habitual responding cannot maximize reward.