We study boundary regularity of maps from two-dimensional domains into manifolds which are critical with respect to a generic conformally invariant variational functional and which, at the boundary, enter perpendicularly into a support manifold. For example, harmonic maps, or H-surfaces, with a partially free boundary condition.In the interior it is known, by the celebrated work of Rivière, that these maps satisfy a system with an antisymmetric potential, from which one can derive regularity of the solution. We show that these maps satisfy along the boundary a system with a nonlocal antisymmetric boundary potential which contains information from the interior potential and the geometric Neumann boundary condition. We then proceed to show boundary regularity for solutions to such systems.