We discuss the properties of bound states in finite-bandwidth waveguide QED beyond the rotating wave approximation or excitation-number-conserving light-matter coupling models. Therefore, we extend the standard calculations to a broader range of light-matter strengths, in particular, in the so-called ultrastrong coupling regime. We do this using the polaron technique. Our main results are as follows: We compute the spontaneous emission rate, which is renormalized as compared with the Fermi golden rule formula. We generalize the existence criteria for bound states, their properties, and their role in the qubit thermalization. We discuss effective spin-spin interactions through both vacuum fluctuations and bound states. Finally, we sketch a perfect state-transfer protocol among distant emitters mediated by bound states.