It is shown that the vacuum expectation values W(C,,..., C,) of products of the traces of the path-ordered phase factors P exp[ig$,,A, (x)dx2] are multiplicatively renormalizable in all orders of perturbation theory. Here A, ( x ) are the vector gauge field matrices in the non-Abelian gauge theory with gauge group U ( N ) or SU(IIr), and C, are loops (closed paths). When the loops are smooth (i.e., differentiable) and simple (i.e., non-self-intersecting), it has been shown that the generally divergent loop functions Wbecome finite functions w when expressed in terms of the renormalized coupling constant and multiplied by the factors e-KL(ci), where K is linearly divergent and L(C, ) is the length of C,. It is proved here that the loop functions remain multiplicatively renormalizable even if the curves have any finite number of cusps (points of nondifferentiability) or cross points (points of self-intersection). If C, is a loop which is smooth and simple except for a single cusp of angle y , then W, (C,) = Z (~)~ ( C , ) is finite for a suitable renormalization factor Z(yj which depends on y but on no other characteristic of C,. This statement is made precise by introducing a regularization, or via a loop-integrand subtraction scheme specified by a normalization condition W, (c?) = 1 for an arbitrary but fixed loop cy. Next, if Cp is a loop which is smooth and simple except for a cross point of angles@, then @(cp) must be renormalized together with the loop functions of associated sets S> = [C',,..., Cpr ) (i = 2,-.,I) of loops C b which coincide with certain parts of Cp3C:. Then W, (Sb) = Z V)@(S$) is finite for a suitable matrix.ZtJ(p). Finally, for a loop with r cross points of angles @l,...,fl, and s cusps of angles yl,-.,y,, the corresponding renormalization matrices factorize locally as Zilj I@ ,)... Ztdr (p,)Z(y ,)-. Z(y, ).