A Willmore surface y : M → S n+2 has a natural harmonic oriented conformal Gauss map Gry : M → SO + (1, n + 3)/SO(1, 3) × SO(n), which maps each point p ∈ M to its oriented mean curvature 2-sphere at p. An easy observation shows that all conformal Gauss maps of Willmore surfaces satisfy a restricted nilpotency condition which will be called "strongly conformally harmonic." The goal of this paper is to characterize those strongly conformally harmonic maps from a Riemann surface M to SO + (1, n + 3)/SO + (1, 3) × SO(n) which are the conformal Gauss maps of some Willmore surface in S n+2 . It turns out that generically the condition of being strongly conformally harmonic suffices to be associated to a Willmore surface. The exceptional case will also be discussed.