The exact matrix elements for transitions from the ground state to some excited state of a nonrelativistic hydrogenic atom, accompanied by the emission or absorption of a real or virtual photon, are here obtained in closed form. The matrix elements are simple rational functions of the photon energy if the excited state is bound, or have a simple transcendental factor if the excited state is unbound. It follows from the form of these elements that the contribution of each such process to the second-order radiative correction to the ground state energy level is always finite, without renormalization. It also follows that the sum of these contributions over all bound intermediate states is finite, while the sum over all unbound intermediate states diverges at most logarithmically without renormalization.