Recent study has shown that a non-singular oscillating potential -a feature of Infinite Derivative Gravity (IDG) theories -matches current experimental data better than the standard GR potential. In this work we show that this non-singular oscillating potential can be given by a wider class of theories which allows the defocusing of null rays, and therefore geodesic completeness. We consolidate the conditions whereby null geodesic congruences may be made past-complete, via the Raychaudhuri Equation, with the requirement of a non-singular Newtonian potential in an IDG theory. In doing so, we examine a class of Newtonian potentials characterised by an additional degree of freedom in the scalar propagator, which returns the familiar potential of General Relativity at large distances.