The zinc finger antiviral protein (ZAP) is a broad inhibitor of virus replication. Its best-characterized function is to bind CpG dinucleotides present in viral RNA and, through the recruitment of TRIM25, KHNYN and other cellular RNA degradation machinery, target them for degradation or prevent their translation. ZAP’s activity requires the N-terminal RNA binding domain that selectively binds CpG-containing RNA. However, much less is known about the functional contribution of the remaining domains. Using ZAP-sensitive and ZAP-insensitive human immunodeficiency virus type I (HIV-1), we show that the catalytically inactive poly-ADP-ribose polymerase (PARP) domain of the long ZAP isoform (ZAP-L) is essential for CpG-specific viral restriction. Mutation of a crucial cysteine in the C-terminal CaaX box that mediates S-farnesylation and, to a lesser extent, the inactive catalytic site triad within the PARP domain, disrupted the activity of ZAP-L. Addition of the CaaX box to ZAP-S partly restored antiviral activity, explaining why ZAP-S lacks CpG-dependent antiviral activity despite conservation of the RNA-binding domain. Confocal microscopy confirmed the CaaX motif mediated localization of ZAP-L to vesicular structures and enhanced physical association with intracellular membranes. Importantly, the PARP domain and CaaX box together modulate the interaction between ZAP-L and its cofactors TRIM25 and KHNYN, implying that its proper subcellular localisation is required to establish an antiviral complex. The essential contribution of the PARP domain and CaaX box to ZAP-L’s CpG-directed antiviral activity was further confirmed by inhibition of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) replication. Thus, compartmentalization of ZAP-L on intracellular membranes provides an essential effector function in the ZAP-L-mediated antiviral activity.