We develop the existence and regularity theory for the generalized Neumann problem for Yang-Mills connections. This is the most general boundary value problem for connections on a compact manifold with smooth boundary, with geometric meaning. It is obtained by reflecting the base manifold across its boundary and lifting this action non-trivially to the bundle. The prescribed lifting corresponds to a geometric invariant, which is similar to the monopole number. When this invariant is non-zero, there exist non-trivial solutions of the generalized Neumann problem. We prove the existence of non-trivial solutions over the 3-dimensional disk D 3 and over the 4-dimensional manifold D 3 × S 1 . We outline the procedure for finding non-trivial examples of solutions over more general manifolds of dimension 3 and 4.