We construct an explicit family of modular iterated integrals which involves cusp forms. This leads to a new method of producing "invariant versions" of iterated integrals of modular forms. The construction will be based on an extension of higher-order modular forms which, in contrast to the standard higher-order forms, applies to general Fuchsian groups of the first kind and, as such, is of independent interest.