We introduce a notion of ternary distributive algebraic structure, give examples, and relate it to the notion of a quandle. Classification is given for low order structures of this type. Constructions of such structures from ternary bialgebras are provided. We also describe ternary distributive algebraic structures coming from groups and give examples from vector spaces whose bases are elements of a finite ternary distributive set. We introduce a cohomology theory that is analogous to Hochschild cohomology and relate it to a formal deformation theory of these structures.