Primordial black holes formed through the collapse of cosmological density fluctuations have been hypothesised as contributors to the dark matter content of the Universe. At the same time, their mergers could contribute to the recently observed population of gravitational-wave sources. We investigate the mergers of primordial black holes in small clusters of ∼ 30 objects in the absence of initial binaries. Binaries form dynamically through Newtonian gravitational interactions. These binaries act as heat sources for the cluster, increasing the cluster's velocity dispersion, which inhibits direct mergers through gravitational-wave two-body captures. Meanwhile, three-body encounters of tight binaries are too rare to tighten binaries sufficiently to allow them to merge through gravitational-wave emission. We conclude that in the absence of initial binaries, merger rates of primordial black holes are at least an order of magnitude lower than previously suggested, and are thus insufficient to allow for gravitational-wave detections of such sources. Conversely, gravitational-wave observations cannot constrain the contribution of primordial black holes to the dark matter density in this scenario.