The existence of black holes (BHs) with masses in the range between stellar remnants and supermassive BHs has only recently become unambiguously established. GW190521, a gravitational wave signal detected by the LIGO/Virgo Collaboration, provides the first direct evidence for the existence of such intermediate-mass BHs (IMBHs). This event sparked and continues to fuel discussion on the possible formation channels for such massive BHs. As the detection revealed, IMBHs can form via binary mergers of BHs in the “upper mass gap” (≈40–120 M ⊙). Alternatively, IMBHs may form via the collapse of a very massive star formed through stellar collisions and mergers in dense star clusters. In this study, we explore the formation of IMBHs with masses between 120 and 500 M ⊙ in young, massive star clusters using state-of-the-art Cluster Monte Carlo models. We examine the evolution of IMBHs throughout their dynamical lifetimes, ending with their ejection from the parent cluster due to gravitational radiation recoil from BH mergers, or dynamical recoil kicks from few-body scattering encounters. We find that all of the IMBHs in our models are ejected from the host cluster within the first ∼500 Myr, indicating a low retention probability of IMBHs in this mass range for globular clusters today. We estimate the peak IMBH merger rate to be  ≈ 2 Gpc − 3 yr − 1 at redshift z ≈ 2.