Although the hippocampus has been implicated in both the temporal organization of memories and association of scene elements, some theoretical accounts posit that the role of the hippocampus in episodic memory is largely atemporal. In this study, we set out to explore this discrepancy by identifying hippocampal activity patterns related to scene construction while participants performed a temporal order memory task. Participants in the fMRI scanner were shown a sequence of photographs, each consisting of a central object and a contextual background scene. On each retrieval trial, participants were shown a pair of the original photographs (FULL), objects from the scenes without the background (OBJ), or background contexts without the main foreground object (BACK). In the temporal order judgment (TOJ) task, participants judged the temporal order of the pair of scenes; in the Viewing trials, two identical scenes were shown without any task. First, we found that the anterior hippocampus—particularly the CA1 and subiculum—showed similar patterns of activation between the BACK and OBJ conditions, suggesting that scene construction occurred spontaneously during both TOJ and Viewing. Furthermore, neural markers of scene construction in the anterior hippocampus did not apply to incorrect trials, showing that successful temporal memory retrieval was functionally linked to scene construction. In the cortex, time‐processing areas, such as the supplementary motor area and the precuneus, and scene‐processing areas, such as the parahippocampal cortex, were activated and functionally connected with the hippocampus. Together, these results support the view that the hippocampus is concurrently involved in scene construction and temporal organization of memory and propose a model of hippocampal episodic memory that takes both processes into account.