Ba 2 M 3 (C 2 O 4 ) 3 (OH) 4 •3H 2 O (M = Ni, Co) are synthesized by the traditional hydrothermal method. The basic magnetic unit of this system is a spin hexamer constituted by two non-coplanar triangular rings. The magnetic spin hexamers were connected with each other by oxalate groups, forming a twodimensional framework along the ab plane. Susceptibility, magnetization, and specific heat have been measured to study the magnetism of this system. Ba 2 Ni 3 (C 2 O 4 ) 3 (OH) 4 •3H 2 O (Ni compound) exhibits two magnetic transitions at T 1 = 20 K and T 2 = 12 K with Weiss temperature θ W = −135.65 K. However, Ba 2 Co 3 (C 2 O 4 ) 3 (OH) 4 •3H 2 O (Co compound) undergoes one magnetic transition at T 1 = 10 K with Weiss temperature θ W = −85.03 K. It is found that both compounds are frustrated antiferromagnets with the frustrated factor of f = 6.8 for the Ni compound and f = 8.5 for the Co compound. The possible origin of spin frustration is discussed.