To manipulate orbital angular momentum (OAM) carried by light beams, there is a great interest in designing various optical elements from the deep-ultraviolet to the microwave. Normally, the OAM variation introduced by optical elements can be attributed to two terms, namely the dynamic and geometric phases. Up till now, the dynamic contribution induced by optical elements has been clearly recognized. However, the contribution of geometric phase still seems obscure, especially considering the vector vortex beams. In this work, an analytical formula is derived to fully describe the OAM variation introduced by the nonabsorbing optical elements, which perform space-variant polarization-state manipulations. It is found that the geometric contribution can be further divided into two parts: one is directly related to optical elements and the other one explicitly relies solely on the vortices before and after the transformations. Based on this result, the same OAM variation can be achieved with different combinations of the dynamic and/or geometric contributions. With numerical simulations, it is shown that transformation of the optical vortices can be fully and flexibly designed with a family of optical elements. We believe that these results are helpful to understand the effect of optical elements and offer a new perspective to design the optical elements for manipulating the OAM carried by light beams. * dkzhang@outlook.com † x-feng@tsinghua.edu.cn to the closed trajectory |a → |b → |b † J → |a † J → |a , where |a † J (|b † J ) holds the Stokes vector S a † J (b † J ) = S a(b) − 2(S a(b) · S J )S J