We present a comprehensive structural study of the charge-orbital ordering and magnetic phase transitions observed in the A-site ordered SmBaMn 2 O 6 perovskite combining synchrotron radiation x-ray powder diffraction and symmetry-adapted modes analysis. In SmBaMn 2 O 6 , successive phase transitions in charge, spin, and lattice degrees of freedom take place with decreasing temperature at T CO1 ≈ 380 K, T CO2 ≈ 190 K, and T N ≈ 250 K. The main difference between the two charge-ordered phases concerns the stacking sequence along the c axis, which is double for the high temperature charge-ordered phase and has led to controversy in the literature. We show that both charge-ordered phases are pseudosymmetric with respect to the ideal undistorted tetragonal structure of A-site ordered RBaMn 2 O 6 perovskites and lead to two nonequivalent Mn sites. However, the charge segregation stabilizes at about 0.35e − in the low temperature charge-ordered phase, clearly below the nominal separation of one charge unit between Mn 3+ and Mn 4+ and undergoes a prominent increase in the high temperature charge-ordered phase when warming above ≈250 K. The two Mn sites are anisotropic in both charge-ordered phases but the analysis of the active modes discloses that only the low temperature charge-ordered phase displays a Jahn-Teller-like distortion for one of the Mn sites. In addition, this low temperature charge-ordered phase has polar symmetry compatible with ferroelectricity along the a axis.