We generalize Milnor link invariants to all types of knotted surfaces in 4-space, and more generally to all codimension 2 embeddings. This is achieved by using the notion of cut-diagram, which is a higher dimensional generalization of Gauss diagrams, associated to codimension 2 embeddings in Euclidian spaces. We define a notion of group for cut-diagrams, which generalizes the fundamental group of the complement, and we extract Milnor-type invariants from the successive nilpotent quotients of this group. We show that the latter are invariant under an equivalence relation called cut-concordance, which encompasses the topological notion of concordance. We give several concrete applications of the resulting Milnor concordance invariants for knotted surfaces, comparing their relative strength with previously known concordance invariants, and providing realization results. We also obtain classification results for Spun links up to link-homotopy, as well as a criterion for a knotted surface to be ribbon. Finally, the theory of cut-diagrams is further investigated, heading towards a combinatorial approach to the study of surfaces in 4-space.