The family of Willmore immersions from a Riemann surface into S n+2 can be divided naturally into the subfamily of Willmore surfaces conformally equivalent to a minimal surface in R n+2 and those which are not conformally equivalent to a minimal surface in R n+2 . On the level of their conformal Gauss maps into Gr 1,3 (R 1,n+3 ) = SO + (1, n + 3)/SO + (1, 3) × SO(n) these two classes of Willmore immersions into S n+2 correspond to conformally harmonic maps for which every image point, considered as a 4-dimensional Lorentzian subspace of R 1,n+3 , contains a fixed lightlike vector or where it does not contain such a "constant lightlike vector". Using the loop group formalism for the construction of Willmore immersions we characterize in this paper precisely those normalized potentials which correspond to conformally harmonic maps containing a lightlike vector. Since the special form of these potentials can easily be avoided, we also precisely characterize those potentials which produce Willmore immersions into S n+2 which are not conformal to a minimal surface in R n+2 . It turns out that our proof also works analogously for minimal immersions into the other space forms.