We develop an extension of the infinite swapping and partial infinite swapping techniques [N. Plattner, J. D. Doll, P. Dupuis, H. Wang, Y. Liu, and J. E. Gubernatis, J. Chem. Phys. 135, 134111 (2011)] to curved spaces. Furthermore, we test the performance of infinite swapping and partial infinite swapping in a series of flat spaces characterized by the same potential energy surface model. We develop a second order variational algorithm for general curved spaces without the extended Lagrangian formalism to include holonomic constraints. We test the new methods by carrying out NVT classical ensemble simulations on a set of multidimensional toroids mapped by stereographic projections and characterized by a potential energy surface built from a linear combination of decoupled double wells shaped purposely to create rare events over a range of temperatures.