Memory deficits are common in epilepsy patients. In these patients, the interictal EEG commonly shows interictal epileptiform discharges (IEDs). While IEDs are associated with transient cognitive impairments, it remains poorly understood why this is. We investigated the effects of human (male and female) hippocampal IEDs on single-neuron activity during a memory task in patients with medically refractory epilepsy undergoing depth electrode monitoring. We quantified the effects of hippocampal IEDs on single-neuron activity and the impact of this modulation on subjectively declared memory strength. Across all recorded neurons, the activity of 50 of 728 neurons were significantly modulated by IEDs, with the strongest modulation in the medial temporal lobe (33 of 416) and in particular the right hippocampus (12 of 58). Putative inhibitory neurons, as identified by their extracellular signature, were more likely to be modulated by IEDs than putative excitatory neurons (19 of 157 vs 31 of 571). Behaviorally, the occurrence of hippocampal IEDs was accompanied by a disruption of recognition of familiar images only if they occurred up to 2 s before stimulus onset. In contrast, IEDs did not impair encoding or recognition of novel images, indicating high temporal and task specificity of the effects of IEDs. The degree of modulation of individual neurons by an IED correlated with the declared confidence of a retrieval trial, with higher firing rates indicative of reduced confidence. Together, these data link the transient modulation of individual neurons by IEDs to specific declarative memory deficits in specific cell types, thereby revealing a mechanism by which IEDs disrupt medial temporal lobe-dependent declarative memory retrieval processes.