While word predictability from sentence context is typically investigated by cloze completion probabilities (CCP), it can be more deeply understood by relying on language models (LMs), allowing to define the three key components of memory: Memory starts with experience as implemented by a text corpus, here defined by Wikipedia capturing general knowledge and (movie) subtitles approximating social interactions. LMs then consolidate a long-term memory structure from experience, as addressed by n-gram, topics and recurrent neural network (RNN) models. Retrieval was investigated by predicting fixation durations from an English and a German reading sample. Item-level regressions showed greater correlations of LMs with single-fixation duration (SFD), gaze duration (GD) and total viewing time (TVT) than CCP. When predicting each fixation case separately using generalized additive models, three LMs together always performed better than CCP. When testing single LMs against the typically-sized English CCP sample (N = 30), LMs usually performed better than CCP (8 vs. 3). The larger German CCP sample (N = 272), however, often performed better than single LMs (4 vs. 2). Subtitles-trained n-gram probabilities of present (and last) words allowed for reliable predictions of all fixation durations. Wikipedia-trained topic probabilities of the last and present word allow for reliable predictions of late GD and TVT effects. The present word predictions of RNNs were less sensitive to training-corpus choice and are recommendable if a single LM is used. Moreover, its reliable next word probability effects make it most suitable to address parafoveal preview and top-down predictions.