Little is known about the effect of perennial biomass crops (PBCs) removal on soil C dynamics. The belowground biomass (BGB) that is composed by plant belowground organs (PBO) such as rhizomes in the herbaceous PBCs and stumps in woody PBCs should be considered, together with fine roots (FR), as a huge input of exogenous organic matter (EOM) that is incorporated into the soil at the reversion. In this study, we mimic the incorporation of BGB of PBCs through a soil-residues incubation under controlled conditions to investigate the effects of adding FR and PBO (at real field rates) on soil C and N mineralization dynamics, and to understand decomposition controlling factors. A modified RothC model version, encompassing a better description of decomposable (DEOM) and resistant (REOM) pools, was fitted to C mineralization curves of respiration measured by CO2 evolution in incubated soil to quantify partitioning factors and decomposition rates of PBCs BGB components. After 1 month, PBO showed higher mineralization rates (498 µg CO2-C gsoil−1) than FR (196 µg CO2-C gsoil−1), with black locust having the highest amount of C respired (38% of added C). The emission peak occurred within 3 days from the beginning of the experiment for PBO and after 1 day for FR. Generally, according to the modified version of RothC model, PBO had higher proportion of REOM than FR, except for black locust. The decomposition constant rates from the optimized RothC model were higher for PBO (kDEOM: 20.9 y−1, kREOM: 12.1 y−1) than FR (kDEOM: 0.4 y−1, kREOM: 0.1 y−1), indicating that FR are less decomposable than PBO. The C/N ratio is not the main controlling factor of decomposition when residue N is not a limiting factor, while the availability of easily decomposable substrates (DEOM/REOM ratio) and cell-wall composition decomposition is a strong predictor of C and N mineralization of these EOM types. The explicit inclusion of crop-specific DEOM/REOM ratios within RothC or a similar soil C model will help to improve the predictions of long-term C sequestration trajectories (half-life > 30 years) associated with PBCs cultivation, especially when dismission of such perennial cropping systems is addressed.