Aims. To investigate the impact of radio jets during the formation epoch of their massive host galaxies, we present an analysis of two massive, log M stellar /M ∼ 10.6 and 11.3, compact radio galaxies at z = 3.5, TNJ0205+2242 and TNJ0121+1320. Their small radio sizes (R ≤ 10 kpc) are most likely a sign of youth. In particular, we compare their radio properties and gas dynamics with those in well extended radio galaxies at high redshift, which show strong evidence for powerful, jet-driven outflows of significant gas masses (M ∼ 10 9−10 M ). Methods. Our analysis combines rest-frame optical integral-field spectroscopy obtained with SINFONI on the VLT with existing radio imaging, CO(4-3) emission line spectra, and rest-frame UV longslit spectroscopy. Results.[OIII]λ5007 line emission is compact in both galaxies and lies within the region defined by the radio lobes. For TNJ0205+2242, the Lyα profile narrows significantly outside the jet radius, indicating the presence of a quiescent halo. TNJ0121+1320 has two components at a projected relative distance of ∼10 kpc and a velocity offset of ∼300 km s −1 , measured from the [OIII]λ5007 velocity map. This suggests that the fainter component is orbiting around the more massive, radio-loud galaxy. If motions are gravitational, this implies a dynamical mass of 2 × 10 11 M for the radio-loud component. Conclusions. The dynamical mass, molecular gas mass measured from the CO line emission, and radio luminosity of these two compact radio galaxies imply that compact radio sources may well develop large-scale, energetic outflows as observed in extended radio galaxies, with the potential of removing significant fractions of the ISM from the host galaxy. The absence of luminous emission line gas extending beyond the radio emission in these sources agrees with the observed timescales and outflow rates in extended radio galaxies, and adds further evidence that the energetic, large-scale outflows observed in extended radio sources (Nesvadba et al. 2006, ApJ, 650, 693) are indeed the result of influence of the radio jet.