We study boundary regularity of maps from two-dimensional domains into manifolds which are critical with respect to a generic conformally invariant variational functional and which, at the boundary, intersect perpendicularly with a support manifold. For example, harmonic maps, or H-surfaces, with a partially free boundary condition. In the interior it is known, by the celebrated work of Rivière, that these maps satisfy a system with an antisymmetric potential, from which one can derive the interior regularity of the solution. Avoiding a reflection argument, we show that these maps satisfy along the boundary a system of equations which also exhibits a (nonlocal) antisymmetric potential that combines information from the interior potential and the geometric Neumann boundary condition. We then proceed to show boundary regularity for solutions to such systems.