We extend the newly proposed probability-changing cluster (PCC) Monte Carlo algorithm to the study of systems with the vector order parameter. Wolff's idea of the embedded cluster formalism is used for assigning clusters. The Kosterlitz-Thouless (KT) transitions for the two-dimensional (2D) XY and q-state clock models are studied by using the PCC algorithm. Combined with the finite-size scaling analysis based on the KT form of the correlation length, ξ ∝ exp(c/ T /TKT − 1), we determine the KT transition temperature and the decay exponent η as TKT = 0.8933(6) and η = 0.243(5) for the 2D XY model. We investigate two transitions of the KT type for the 2D q-state clock models with q = 6, 8, 12, and for the first time confirm the prediction of η = 4/q 2 at T1, the low-temperature critical point between the ordered and XY-like phases, systematically.