We study the evolution of photon polarization during the photon-axion conversion process with focusing on the magnetic field configuration dependence. Most previous studies have been carried out in a conventional model where a network of magnetic domains is considered and each domain has a constant magnetic field. We investigate a more general model where a network of domains is still assumed, but each domain has a helical magnetic field. We find that the asymptotic behavior does not depend on the configuration of magnetic fields. Remarkably, we analytically obtain the asymptotic values of the variance of polarization in the conventional model. When the helicity is small, we show that there appears the damped oscillating behavior in the early stage of evolution.Moreover, we see that the constraints on the axion coupling and the cosmological magnetic fields using polarization observations are affected by the magnetic field configuration. This is because the different transient behavior of polarization dynamics is caused by the different magnetic field configuration. Recently, [C. Wang and D. Lai, J. Cosmol. Astropart. Phys. 06 (2016) 006.] claimed that the photon-axion conversion in helical model behaves peculiarly. However, our helical model gives much closer predictions to the conventional discontinuous magnetic field configuration model.