In this paper, a novel Constellation Extension (CE) based approach is presented to address the high Peak-to-Average Power Ratio (PAPR) problem at the transmitter side, which is an important drawback of Orthogonal Frequency Division Multiplexing (OFDM) systems. This new proposal is formulated as a Mixed Integer Non-Linear Programming (MINLP) optimization problem, which employs Generalized Benders Decomposition (GBD) and Branch-and-Bound (BB) methods to determine the most adequate extension factor and the optimum set of input symbols to be extended within a proper quarter-plane of the constellation. The optimum technique based on GBD, denoted as Generalized Benders Decomposition for Constellation Extension (GBDCE), provides a bound with relevant improvement in terms of PAPR reduction compared with other CE techniques, although it may exhibit slow convergence. To avoid excessive processing time in practical systems, the sub-optimum Branch-and-Bound for Constellation Extension (BBCE) scheme is proposed. Simulation results show that BBCE achieves a significant PAPR reduction, providing a good trade-off between complexity and performance. We also show that the BBCE scheme performs satisfactorily in terms of Power Spectral Density (PSD) and Bit Error Rate (BER) in the presence of a non-linear High Power Amplifier (HPA).