We have performed high-field magnetization and ESR measurements on Ba3CoSb2O9 single crystals, which approximates the two-dimensional (2D) S = 1/2 triangular-lattice Heisenberg antiferromagnet. For an applied magnetic field H parallel to the ab-plane, the entire magnetization curve including the plateau at one-third of the saturation magnetization (Ms) is in excellent agreement with the results of theoretical calculations except a small step anomaly near (3/5)Ms, indicative of a theoretically undiscovered quantum phase transition. However, for H c, the magnetization curve exhibits a cusp near Ms/3 owing to the weak easy-plane anisotropy and the 2D quantum fluctuation. From a detailed analysis of the collective ESR modes observed in the ordered state, combined with the magnetization process, we have determined all the magnetic parameters including the interlayer and anisotropic exchange interactions.PACS numbers: 75.10. Jm, 75.45.+j, 75.60.Ej, Over the past decades, there has been considerable interest in frustrated quantum magnets, owing to a rich variety of exotic quantum phenomena [1][2][3]. For classical spins with an antiferromagnetic coupling, a geometric frustration suppresses the long-range ordering, leading to a degenerate ground state. The degeneracy can be destroyed by quantum fluctuations, which emerge not only through an interplay of strong geometric frustration, low dimensionality, and small spin, but also through the application of a magnetic field. Despite intensive research efforts, the detailed mechanism of the quantum effects, e.g., the ground state property [4,5], has still been highly controversial.One macroscopic manifestation of the quantum phenomena is the stabilization of the "up-up-down" spin structure under a magnetic field, predicted for a twodimensional (2D) triangular-lattice Heisenberg antiferromagnet (TLHAF) with a small spin [6,7]. In a magnetization process, the nonclassical anomaly appears as a plateau in a finite field range at one-third of the saturation magnetization M s , hereafter referred to as the M s /3 plateau. In a classical picture, a monotonic increase in the magnetization is expected up to M s . A number of theoretical approaches for explaining the quantum mechanism of the M s /3 plateau have been proposed [8][9][10][11][12][13][14]. Thus far, however, few numbers of definite experimental results reserved judgment on the issue. This is mainly due to the experimental difficulty in growing the model material, let alone in observing the M s /3 plateau purely driven by quantum fluctuations. In fact, most of the TLHAFs ever studied, such as Cs 2 CuBr 4 , [15,16] have a distorted triangular lattice, which induces an antisymmetric Dzyaloshinsky-Moriya (DM) interaction.It is believed that the spin state in the lower-field range above the higher edge field of the M s /3 plateau is the 2 : 1 canted coplanar state that is a continuous variant of the up-up-down state [7][8][9][10][11]. However, whether the 2 : 1 canted coplanar state is stable up to the saturation or a new quan...