Anticipated and experienced rewards modulate cognitive control and attention, but the mechanisms of these modulations are poorly understood. We compared neuronal responses in monkey dorsolateral prefrontal cortex and parietal area 7A, two areas strongly implicated in visual attention and working memory, related to probabilistic rewards signaled by familiar visual cues. Neurons in both areas encoded the magnitude, probability and expected value (EV) of the reward signaled by the cue. Strikingly, neurons also encoded across-trial memories of recent rewards, which, although statistically irrelevant to a trial's expectations, correlated with the monkeys' behavioral sensitivity to reward history. Finally, upon outcome delivery, neurons combined responses to the experienced outcome with renewed sensitivity to EV and reward history, allowing for population-level decoding of reward prediction errors (RPEs) relative to the trial's EV and reward history. Frontal and parietal areas explicitly encode expected and experienced outcomes and provide information relevant to computing model-free and model-based RPEs.