We calculate the two loop hard correction to the photon self-energy in an electron-positron plasma (EPP) for arbitrary soft momenta. This provides the only missing ingredient to obtain the Hard Thermal Loop (HTL) effective Lagrangian at next-to-leading order (NLO), and the full photon propagator at the same order. This result can be easily extended to obtain the soft photon propagator in a quark gluon plasma. We use the Keldysh representation of the real time formalism in the massless fermion limit, and dimensional regularization (DR) to regulate any ultraviolet (UV), infrared (IR) or collinear divergences that appear in the intermediate steps of the calculation. In the limit of soft photon momenta, our result is finite. It not only provides an O(α) correction to the Debye mass, but also a new non-local structure. A consistent regularization of radial and angular integrals is crucial to get this new structure. As an application we calculate the plasmon dispersion relations at NLO.