The proper management of diversity is essential to the success of Evolutionary Algorithms. Specifically, methods that explicitly relate the amount of diversity maintained in the population to the stopping criterion and elapsed period of execution, with the aim of attaining a gradual shift from exploration to exploitation, have been particularly successful.However, in the area of Genetic Programming (GP), the performance of this design principle has not been studied. In this paper, a novel GP variant, Genetic Programming with Dynamic Management of Diversity (GP-DMD), is presented. GP-DMD applies this novel design principle through a replacement strategy that combines penalties based on distance-like functions with a multi-objective Pareto selection based on maximizing accuracy and minimizing tree size. The experimental validation based on the symbolic regression task considers several state-of-the-art approaches and shows clear improvements both in terms of accuracy and simplicity. The effects of GP-DMD on the dynamics of the population are also analyzed, revealing the reasons for the superiority of GP-DMD. As in other fields of Evolutionary Computation, this design principle contributes significantly to the area of GP.