In acceptor doped ferroelectrics and in ferroelectric films and nanocomposites, defect dipoles, strain gradients, and the electric boundary conditions at interfaces and surfaces often impose internal bias fields. In this work we delicately study the impact of internal bias fields on the electrocaloric effect (ECE), utilizing an analytical model and ab initio-based molecular dynamics simulations. We reveal the complex dependency of the ECE on field protocol and relative strength of internal and external fields. The internal fields may even reverse the sign of the response (inverse or negative ECE). We explore the transition between conventional and inverse ECE and discuss reversible and irreversible contributions to the field-induced specific entropy change. Most importantly, we predict design routes to optimize the cooling and heating response for small external fields by the combination of internal field strengths and the field loading protocol. arXiv:1805.04380v2 [cond-mat.mtrl-sci] 14 May 2018 P end Pinit EdP , with P init and P end the initial and final polarization, respectively. For positive E, the work is thus positive if |P | increases, while negative if |P | decreases, and vice versa for negative E. Outside the hysteretic region (e.g. between A and B or E and