We identify and discuss the ground state of a quantum magnet on a triangular lattice with bonddependent Ising-type spin couplings, that is, a triangular analog of the Kitaev honeycomb model. The classical ground-state manifold of the model is spanned by decoupled Ising-type chains, and its accidental degeneracy is due to the frustrated nature of the anisotropic spin couplings. We show how this subextensive degeneracy is lifted by a quantum order-by-disorder mechanism and study the quantum selection of the ground state by treating short-wavelength fluctuations within the linked cluster expansion and by using the complementary spin-wave theory. We find that quantum fluctuations couple next-nearest-neighbor chains through an emergent four-spin interaction, while nearest-neighbor chains remain decoupled. The remaining discrete degeneracy of the ground state is shown to be protected by a hidden symmetry of the model. Frustrated magnets, systems in which every pairwise exchange interaction cannot be simultaneously satisfied, are characterized by accidental degeneracies between various order patterns 1 . Often, these accidental degeneracies are lifted via an order-by-disorder mechanism, driven by thermal and/or quantum fluctuations, selecting an unique ground state 2-4 . In highly frustrated quantum magnets, those with extensive degeneracy, e.g., the isotropic spin one-half kagomé and pyrochlore antiferromagnets (AF), the order-by-disorder mechanism is inactive and they remain disordered down to the lowest temperatures, realizing so-called quantum spin liquids (QSL) in their ground states 1 .In 30 and quantum 31,32 spins has been studied numerically. The obtained rich phase diagram includes a Z 2 -vortex crystal phase near the AF Heisenberg limit, and a nematic phase of decoupled Ising chains with subextensive degeneracy at the Kitaev limit 30,31 . In addition, a chiral spin-liquid phase has been proposed close to the antiferromagnetic Kitaev limit 32 . Here, we study analytically the Kitaev model on the triangular lattice and solve the puzzle of its ground state by analyzing the effects of quantum fluctuations within both the linked-cluster expansion, 33 combined with degenerate perturbation theory, and the linear spin-wave theory. We show that such a deceptively simple model, once realized on a triangular lattice, becomes the host of very interesting and unexpected order-by-disorder effects such as: the quantum selection of the easy axes, the emergence of a specific four-spin interaction, the reduction of the sub-extensive degeneracy of the nematic ground state manifold down to a discrete one protected by a hidden symmetry of the model.
I. THE MODELWe consider a triangular lattice lying in the (1, 1, 1) plane of the spin-quantization frame [see Fig. 1(a)] and label by (γ)(= x, y, z) its three non-equivalent NN bonds spanned by the lattice vectors a x = 1 /2, − √ 3 /2 , a y = 1 /2, √ 3 /2 and a z = (1, 0), respectively. On a (γ)-bond, the one perpendicular to the γ spin-quantization axis, only the S γ i components of...